Author: Niklas Back, Test Automation
I have been involved in several test automation project over the past years, with various results. Some projects were huge successes while others were total disasters. What I have noticed is that there are some common denominators in the successful projects.
Something we need to understand is that the difference between working as a test automation engineer and as a programmer is subtle.
- Usually the same programming language is used. There are many recording tools available, especially when it comes to GUI testing, each of them running their own language and syntax. It might be enticing to deploy one of them but then you are neglecting the existing knowledgebase and programing competence. If it is possible, both test and development should use the same development tools and programing language.
- The same code base is used, and the requirement on the code quality is the same. This is especially true if it is an agile project. In this case, the test code is tightly coupled with the developers’ code. Older test code will probably not run on later builds and vice versa.
This means that you have to involve the programmers, you have to erase the barrier between test and development.
When you involve the programmers in the testing process you will get a better understanding of what needs to be tested, the programmers can assist you in making the application more testable, they can assist you in making stubs and drivers. In addition, they might even acquire the mind-set of a tester. Moreover, programmers are usually quite good at programming. So do not hesitate to ask for their assistance if you are stuck.
Another success factor is to get all testers involved. Just like in development, the more people that scrutinizes the test code, the better the code quality. The key is to avoid smart coding, and by involving more testers in the code base, you are forced to write simple and understandable code. The cynosure here is KISS (Keep It Simple Stupid). By involving all testers, you will distribute the knowledge of the test tool and remove the dependency on one person.
You also need to get the full support from management. There seems to be a misconception that by introducing test automation, it will be possible to cut costs on testing, but this is incorrect. What many project managers do not realize is that they have to consider the long-term costs. Test automation is more expensive compared to having a group of manual testers, and what most project managers does not realize is the difference in speed and quality between manual testing and automated testing.
What we need to remember is that it is human to make mistakes. Having a group of testers performing the same tests day out and day in makes them less alert, and they will start to make mistakes and more bugs will slip through, that has to be addressed later, perhaps at the customers site (and this can be very expensive, e.g. in embedded systems). In addition, if the developers are forced to wait for the testers to finish their manual testing it will slow down the development pace, and they will also write larger chunks of code that may have to be rolled back, because the new code will break something else. By implementing test automation on all levels (Unit, Integration, System and GUI testing), the developers can feel more confident when refactoring the code, and they will get a quicker feedback if the code they just wrote broke something else. The stakeholders will have greater confidence in code quality and the release cycle is much shorter (think CI and CD) meaning that bug fixes and minor enhancements can be deployed much quicker. So to summarize it all: If you look at the total cost for the software life cycle, test automation is much cheaper compared to manual testing.
Do not overdo the test automation, the more tests you write, the more code you have to maintain. The tricky part is to know when to automate and when to test manually. If it takes five minutes to run the test manually, and you will run it once a month, then there is no point spending weeks to automate the
test. Also, prune your test cases when the code evolves. Perhaps one of the newer test cases are covering several old test cases.
Lastly, do not neglect manual testing, because this is how you find most of the bugs (in fact automated tests from integration level and upwards, should only find few defects) and to my experience, exploratory testing finds far more bugs then scripted testing.
