Previously if we do not write automated tests on a web application project we felt that it is totally fine, as long as it is completed with zero or minimum amount of bugs and delivered on time. The clients also did not demand automated tests, and even if we offered it they would reject the idea as the cost of the project would increase.
That might work with smaller project that can be completed in less than 2 months. But on bigger projects, without automated tests the future will be painful for both us IT shops and clients. Bugs occur and reoccur. The application slows down faster than expected after the application is launched and its user base grows. The application does not work on Internet Explorer, etc.
Admittedly, we experienced those nightmares. We learned from our mistakes and thus now quality is non-negotiable. If we build a web application, then it has to meet these five criteria:
1. Automated test on the code. Ruby on Rails in particular makes writing automated tests easier. The minimum test coverage needs to reach 80%. When this is the case bugs will be minimized and the chance of a bug reoccuring because of a change is suppressed down to zero.
2. Automated test on the front-end, using Selenium. We’ll write use cases and that all use cases must be covered.
3. Cross browser compatibility. By default we will support IE7 and above, but not IE6, because even Google Apps has phased it out, and within a year IE6 users will dwindle even more. I know that (lots of) developers and entrepreneurs want IE to be removed from the face of the earth, but that isnt going to happen. IE7 is an animal, IE6 is a beast. And yet according to W3C statistics, currently there are still more IE6 users compared to Opera + Safari combined! So whether or not to support IE6 depends on the client (some still insist that we support and test on IE6 as well).
4. NewRelic doesnt complain. We can install the free version and make sure there is no warning in its report. At least then we know our Ruby code and our database queries arent slowing things down.
5. Flying grades on Yslow measurement on the major webpages. Without us knowing it, most application bottleneck may come from front-end stuff (HTML/CSS , Javascript, Images, #HTTP requests, etc) instead of the back-end stuff (server / Ruby code and database queries). So it is important that if a client chooses to supply the HTML/CSS, that us vendors make sure the client knows what constitutes a good front-end code. Btw not all Yslow grades should / can be A; for example, a client may or may not want to use a CDN hosting, or that not all javascripts and css can be external.
I believe both IT vendors and clients must want to follow these minimum five quality levels. Even higher quality web application is possible. For example, we have not yet talked about javascript-degradability (so users without javascript enabled can access and use the site), and we have also not yet talked about usability (for color-blind or people with other sight problems). But that may not be a good investment unless the client is Google or Yahoo 🙂
2 comments
Chandra
April 4, 2011 at 10:48 am
Salam kenal, saya Chandra, web developer. Saya telah terlibat dalam sejumlah proyek web dan sangat tertarik dengan selenium yang Bapak bahas.
Menurut Bapak, dengan selenium biaya proyek akan bertambah. Kalau boleh tahu, rata-rata biaya proyek bertambah berapa persen dengan menggunakan selenium? Dan bagaimana meyakinkan client untuk menambah biaya demi selenium? Apakah source code selenium termasuk dalam deliverable proyek?
Terima kasih.
Ikin Wirawan
April 4, 2011 at 1:48 pm
Halo Chandra, tentu saja dengan lebih banyak testing yang kita lakukan, akan memakan lebih banyak waktu. Waktu = mandays = cost. Maka cost akan bertambah.
Bila Chandra sudah faham konsep BDD / TDD, maka biasanya penulisan test menambah sekitar 20-30%, namun ini berupa investasi yang bagus karena ke depannya kode yang dibuat akan lebih bersih dan terbebas dari bugs, sehingga save cost di masa datang (jadi ini Investasi, bukan Cost).
Source code Selenium bisa diberikan ke client, terserah deal kamu. Semoga membantu!