Regression testing automate or no automate?
#Prelude (The marketing)
“If you want a job done right, do it yourself.” This definition is usually true but not always. Here we'll talk about boring part of testing - regression testing. And real slogan will be “If you want a regression done right, automate it.” Applications nowadays are often large and complex. They include several levels and many modules with complex relations. Therefore, mobile testing, testing of websites or testing of software requires a lot of time and resources.To simplify such processes, automation is often used. With its help, thousands of test cases can be performed without human intervention. Automated testing becomes the most cost-effective and convenient when applied to regression tests.
1. Running regression tests manually takes a long time.
Performing checks manually, testers will never finish their job on time. Currently, applications are large and complex, and perform various functions. That's why it takes a lot of time to check if everything works as it should.
2. People tend to make mistakes performing a large number of similar boring operations, everyone can make a mistake. Also, a tester may decide to skip some tests because they have always been successful or because, in his opinion, changes in the system do not affect that module.As a result, the test coverage will be insufficient and serious bugs may remain unidentified.
3. Automated testing requires less labor than manual testing.If the tests are automated, the testing company can run the program for testing, for example, after lunch, and it will complete the execution of thousands of boring tests by the next morning.To perform the same work manually, several testers will have to work for several days.
#Interlude1 (The half-truth)
You've just read the text above. Now forget it. Computers currently never do job better than a real human.
The main goal of testing is finding bugs. The truth is that manual testing and automated are two completely different processes, more precisely, two ways to perform the same process. Their dynamics is different, and the types of errors that they try to discover also differ. According to this, there's no sense to compare them literally either by costs or by number of detected errors. Besides, there are many hidden and specific factors involved in the real comparison. Therefore, the best way to assess process - is the context of a series of real projects on software development security. That's why I recommend that automation be regarded as one of the parts of a diverse search for the best testing strategy, and not as the activity that dominates the testing process, or remains The only way to conduct tests.
#Interlude2 (The automate addiction)
You will definitely fall in the pit of automation. To measure your falling speed come up to automation meaningfully. Just as there can be high-quality software, there can also be quality testing automation. To create a high-quality automated test, you must be very careful. This way is strewn with errors. It is necessary to keep the following key principles in mind all the time:
- Keep a clear distinction between automation and the process that is automated. The test process should be presented in such a way that it is easy to analyze and be the basis for automation.
- Think of your automation as a basic set of tests that can be used in addition to manual testing, but do not replace it.
- Carefully select your test instruments. Collect information from other testers and organizations. Get familiar with the trial versions of the test tools you like before buying them.
- Before choosing or developing tools which influences the test management you should be very careful. A good management system can really help make the test case more convenient for analysis, review and modification.
- Make sure that the results of each test suite run are displayed in the test progress report, which contains information which tests were successfully passed and which ones were not, in comparison with the errors already found. The report should also contain information about the changes made or improvements in automated tests. I consider such a report to be an obligatory material for analyzing how much automation is cost-effective.
- Make sure that the product under test is mature enough so that efforts to maintain automated tests while making changes to the product do not outweigh the benefits of automation.
#Postlude (The truth)
Test automation is a very great idea. The secret, how to make it a good investment, is to think firstly about testing, and only secondly about its automation. If testing is the goal in understanding the quality of software, then automation is only a mean to achieve this goal. You can learn about it from advertising, but this is just one of many strategies that support effective testing.