Author: Stina Leijonhufvud, Test manager / Consultant manager
This in an article on how much the company culture and unwritten rules of a workplace can affect both how a requirement is formulated and how it is perceived.
The unwritten rules of the users and/or the company writing the code will affect a lot of things; not least the perception of what is a bug and what is an improvement, and how the System Under Test (SUT) should work. Another aspect is the fact that there might be a difference in how the developers and testers think the SUT will be used and how the system is actually used by the users.
The Relation Between Requirements and Tests
In software testing, we need to know or find the requirements on the SUT before we can design good tests to fully verify that the system works as it should. Traditional development methodologies required that all requirements should be defined before starting development, and that they should be concise, unambiguous and not least testable. With agile methodologies taking over, the requirements and pieces of code get smaller and inherently evolve over time, but generally you still use the requirements to design test cases (whether they are automatic or manual).
In real life my experience is that documented requirements might actually be missing altogether, or if they exist they might be very vague or cover a too broad spectrum. This is especially true when replacing an existing system with a new – especially if it is planned as a “big bang” (yes this still happens). An example of this would be the, actually quite common, requirement(s) that can be simplified to:
“The new system should work like the old system, but better”
As you can understand this is not easily testable as we’d have to thoroughly test the whole system (which could take a very long time) and we would also have to know which differences are to be perceived as bugs and which are to be seen as improved features.
To create good tests the tester has to try to identify all these limitations and possibilities in order to create the most effective tests. Sometimes, however, it might even be easier to be a creative and value adding tester when the requirements are vague or missing, as you do not have to argue whether a requirement is actually correct or not, and just refer to how the system works.
The Effects of Unwritten Rules on Testing
Whether you are lucky and have a very good set of well-defined and unambiguous requirements or if the requirements are few (or too many) , unclear or even contradictory, you need to reflect on whether there might be any unwritten rules that will impact the usage of the SUT and how you should test the system.
In my spare time I am (among other things) a scoutmaster, and I was looking out for inspirations for future meetings when I read about an exercise called “the Hulungs”. The idea is to separate the team into two groups. One group is defined as the natives – The Hulungs – and the other group is set to be a group of international engineers. The Hulungs wants to build a bridge to get “over there” but do not know how, so the team of engineers are sent out to help them with this.
All of this seems extremely simple in theory – at least for the “international engineers” as they just have to tell the Hulungs how to build a bridge (only limitation is that they cannot help with actually building the bridge and a time frame). However the Hulungs have a set of very specific communication habits:
- Anyone speaking without pinching their nose cannot be heard (and is subsequently ignored)
- Hulungs never say no. They always say yes, even when they mean no.
- Some of the tools are forbidden for a female or a male respectively
- All of the above are unwritten rules and may not / cannot be explained to the engineers.
Just imagine being one of the engineers, and the frustration you must feel when talking to the Hulungs, at least from the start…
How the Hulungs Can Help Your Testing
So, before defining your test cases as “ready”; I suggest you sit down and contemplate the Hulungs for a while and then look at the requirements and/or the system again. Especially if the requirements are written by, or for, a set of non-technical users. (Generally speaking requirements for technical systems used by technicians have better requirements, but that is probably due to the fact that the work culture of the users and developers are quite close.)
The important issues to think about are:
- Is there anything in the requirement that could be ambiguous or understood differently (do the writer pinch his nose or not?)
- Is there anything that might actually mean something else if you look at it from another perspective (do yes actually mean no?)
I bet that at least a few of the requirements or test cases can do with some clarification.
So in the future -> any time you get in touch with some requirements:
- Read the requirements through and play with the system (if possible)
- Think about the Hulungs and their view of the world
- Read the requirements and play with the system again.
It will make you a better tester.
