When you need to stop testing
One little girl said: "I know how to tell you " bananabananabanana... " per letter, but I do not know when I should stop".
The beauty of the ideal "scenario" task is that we always know when to stop. The last note, the last line of the dialogue in the script, the last piece of empty space on the canvas means that the end of the work is close. If you do just what tell you scenarios and finish, then you stop. If you notice a problem - you stop. If you have questions / curious ideas - you stop.
Buy how to decide that this stop will be the final one?
When you cover with checks some new functional it is hard to stop. When QA execute TCs and they report more and more bugs after fixing then it is hard to stop.
So how do we determine when to stop?
The first step on this path is the realization that we can not be sure that we have completed our work. Any attempt to find an answer to the question of when to stop will lead you to paradox. From one point of view you will spend a lot of time on some tasks which will not lead you to any good results in scope of time spending. From another point of view you should not leave enemies (bugs) behind your back (there is always customer support with a lot of free testers customers but still).
In order to answer these questions, we will have to analyze testing from start to end.
So main reasons to stop are:
1. Time's up! The most common reason to finish testing: our time allocated for it has expired.
2. A dead horse. The program is too stricter for further testing to make sense. We understand that everything can change so much that the results of our testing will turn into a pumpkin.
3. Mission accomplished. We finish the testing, according to client and it was his decision to stop testing.
4. The mission was canceled. Our client asked us to stop testing. Perhaps we went beyond the budget, or the project was canceled, or whatever else was the reason. Regardless of the reason, we must stop
5. I'm stuck! We stop, because something prevents us. We do not have the necessary information (many complain that they can not test without a specification, for example). The system has a blocking bug, and we can not get to the area that we want to test. We do not have the necessary equipment or the necessary tools. We do not have enough experience to perform a specialized test.
The main reason to stop is the decision of your client. Everything is related to guy who pays you money. It is not possible to find all bugs. It is not possible to make 100% coverage. But you should never stop unless there is heavy reason for it as mentioned above.
Let us take a look on the reasons that might be look like “stop reason” but they are not:
1. You can not find new bugs. If you can not find any bugs it means that you are bad tester. Each software has bugs. So try harder
2. You think that mission is completed and you did your job well. Forget about it. Your work just can not be completed in terms of testing. You can be stopped only by your client. There is always work to do.
3. If you stuck. For example you currently do not have access to database or to the bug tracking system. Try to brainstorm with your teammates and you will find some new scenarios which were not checked previously.
4. Stop word. You do not have any stop word in this case. It is a different game.
Pickles. Sometimes you will have periods when there are no obvious tasks or you are waiting for response from devs etc. Ok next tip is not so fair but in such case you will avoid some unwanted questions from client. Be clever and do some pickles. Time to time you will see real trivial bugs which will not hurt the system. Note them and report time to time when need to. It is not cheating since previously you checked more important scenarios and reported more critically bugs. Just do not forget to report them all.
If your project is oriented to release by your will then it is the different story when to stop testing and do release.
First of all the whole responsibility is on you. What you should do to minimize fall speed in fail case? (“to do list” which is located below is relevant even if you do not decide when to stop but it is mandatory when you do)
1. Request all specifications that you need
2. Ask all questions to relevant person and make sure that you received good answers. Make sure that those conversations are not private and all relevant persons are part of them
3. Discuss new functional via calls
4. Set up priorities for each check that you will do. Remember that positive scenarios always (ALWAYS) have higher priority than negative checks.
5. Try to understand what regression you should do when new functional has arrived.
6. Define estimate time as soon as possible for each of your tasks
7. Before stop testing make sure that client was informed about what part was not tested and what bugs do you have opened right now.
8. Try to combine Manual and Automation testing.
“They told me I had a schizophrenia, I told them I had a bicycle” ©
Be sure that your client understands you and that you both understand the goal of testing the project. If your language is different from client’s language make all your statements simple and straight. The worst enemy of testing is misunderstanding.
Let us summary up all those tips and to do lists. Your job as QA is not only to find errors. Your job is also to set up priorities correctly and to understand what and when you should do.
Be cool but also be warm. Bad bugs to you.