Author: Monika Wiklund, Test Manager
How do you find the right tool for test automation? Where do you start?
It can be difficult to find a ready tool for a specific situation, either because there is no tool that fits or because there is no budget for buying expensive tools. If this is the case, and you decide to make your own tool, a good start is to remember this simple rule of “keep it simple” and to try to create something that is good enough and solves the immediate problem.
At one of my previous work places, we had a system that was based on a mix of asynchronous events that were performed by the users in the system and several different types of system events that were scheduled to perform at certain times during the day, week, month or year. To test this system, we wanted to “fast forward” it to verify that each system event had performed as specified and that the functionality was consistent over time.
Since the system was Linux based, we thought that it would be possible to achieve this by speeding up the system clock by hacking the Linux kernel time drivers as a general solution to speed up long automated test sessions. When we started to research that solution, we quickly concluded that it was very complicated and difficult to get to work. Hence, we searched for an alternate mechanism based on the actual needs of our testing.
We identified two main requirements:
1) to be able to manually skip to the next scheduled event to perform some kind of manual intervention before initiating the next time skip.
2) to be able to make the system automatically skip to a specific time in the future while pausing the skip at predetermined time stamps to allow processing to occur.
An additional key insight was that we had control over when the processing occurred – all processing events were well defined in time. No nondeterministic events were present in the system except for the manual interventions in requirement one.
Our conclusion was that we did not have to speed up the system clock linearly: it was good enough to simply skip ahead to a few seconds before each event was scheduled to take place. When the clock reached the time of the event, the system triggered and performed its processing. After the event was completed, we could skip ahead to a few seconds before the next scheduled event, and so on. This simplified the problem a lot, and we created a very simple 78 lines long python script that achieved exactly what was needed. The script is command line based and uses a crontab-like file that contains the schedule of the system. Based on this information it sets the system clock to a specified time just ahead of the event and waits long enough for the event to finish before it continues.
Some initial experimenting was required to get the right amount of delay before and after each event, but once we got the timing right, we had a very simple and powerful tool for performing long term testing of the system.
Of course, this only works if you know approximately how long time each event takes and add some margin for safety. An obvious improvement would be to monitor the system state to determine when to continue, but the simple solution turned out to be good enough.
We also discussed automating the manual interventions, but I left the work place shortly after and do not know if the tool was evolved or not.
This little story is just an example of how a simple solution can go a long way for solving something that might be perceived as a difficult problem. Test automation does not have to be difficult if you start simple and take it one step at a time.
Identify the most pressing need to start from and make sure that you can solve this problem first, then continue with the next.
