Software Testing Types: Detailed overview of different testing types
An efficient QA process must be comprehensive and multifaceted. It is essential to check software inside out and from different angles. As a result, tests of the most diverse types are executed to ensure unparalleled quality and compliance with users' needs and expectations. In this article, we would like to highlight the importance of software testing one more time and outline diverse approaches to software testing along with its numerous types.
Importance of Software Testing
Even for non-experts, it is obvious that without proper testing of an application, its success is light years away. So it is no coincidence that testing is an essential stage of the software development life cycle. Bug detection and assurance of high product quality are the most evident advantages of software testing. Still, they are not the only ones. In essence, it is just the tip of the iceberg. If to consider software testing more fundamentally, it is possible to reveal the whole scope of indirect advantages:
Early bug detection is a great way to save a fortune. Therefore, you can reduce expenses on bug-fixing and avoid expenses related to possible court claims and penalties for non-compliance with norms and regulations.
Proper software testing helps to reveal defects in a timely manner. This, in its turn, simplifies the identification of root causes and streamlines the bug-fixing process.
High user satisfaction
The high quality of software paves the way to users' hearts, whereas great user satisfaction results in high user acquisition and user retention rates that have a positive impact on your revenue.
High return on investment
As long as software testing helps to cut down development costs, avoid additional expenses and increase revenue, it is a viable option to provide a high ROI.
Under the conditions of severe competition, it is essential to have advantages and stand out from the rivals. Unmatched quality can be your core merit and distinctive feature.
Reputation is equally important for both startups and well-established companies since it is a well-known fact that reputation is a thing that is hard to build and easy to lose. In these terms, it is essential to have a good impression from the very beginning. Software testing is all about meeting your users' needs and expectations. That is why if reputation is one of your main concerns, software testing is not a thing to be neglected.
The modern conception of high product quality also suggests app reliability in the context of security. The thing is that most modern applications collect, store, and use the most diverse sensitive data of their users. Consequently, it is more than essential to provide secure data protection and prevent any hacker attacks. Software testing also includes security testing that while being properly executed helps to reveal possible security loopholes and suggest efficient improvements.
The main types of software testing
Software testing implies the whole set of testing activities. QA specialists try out an app in different ways using the most varied testing techniques along with diverse testing practices. There are multiple classifications of testing types. We differentiate testing types by the following six categories:
Functional vs non-functional testing
The first category is the most extensive and interesting to our mind. Let's go into the nitty-gritty.
Functional testing is exactly what it says on the tin. It concentrates on the functional aspects of a tested application by checking whether all features have been realized and whether they have been realized in the right way. Functional requirements are leveraged to identify expected test results for further comparison with the actual ones.
Non-functional testing covers the whole bunch of non-functional aspects that also contribute to software quality and user satisfaction, i.e. security, performance, scalability, usability, reliability, etc. Non-functional tests are conducted only after the completion of functional tests, allowing a team to polish the AUT. Non-functional testing is not about whether an app works or does not work, but rather about how well it works.
There are diverse types of non-functional tests. Let's have a closer look at each one.
GUI testing is executed to verify whether all UI elements like icons, sliders, text fields, buttons, checkboxes, etc. work and look as expected. Fonts, colors, and element location are also checked in the course of GUI testing.
The aim of interoperability testing is to ensure a flawless function of a tested application with other applications and services.
Compatibility testing aims to verify whether an application works equally well across different devices, hardware, software, OS, networks, web browsers, etc.
There are two approaches to compatibility testing which give rise to the following types of testing:
Backward compatibility testing verifies app compatibility with earlier versions of different hardware and/or software.
Forward compatibility testing assesses the prospects of an app to work smoothly on future versions. It is rather challenging as it is hard to predict exact changes in the forthcoming versions.
Performance testing assesses the responsiveness, scalability, robustness, and stability of an application as it undergoes different workloads.
Depending on the objective of performance testing a few of its subtypes are differentiated:
Load testing checks whether an app can run adequately as the number of concurrent users is continually growing but does not go beyond the estimated workload limits.
Stress testing is conducted to check app behavior during extreme workloads that go beyond the estimated limits. Its purpose is not only to track and assess app performance but also to reveal breaking points.
Executing volume testing, QA engineers check app behavior as a database is populated with a huge amount of data.
QA teams execute stability testing with the purpose to check whether an AUT runs smoothly while being exposed to expected but long-lasting workloads. During stability testing, QA engineers check an app for possible memory leaks, unexpected failures, crashes, server restarts, and other issues that can have a serious toll on app stability.
The purpose of spike testing is to monitor app performance as the workload suddenly and significantly rises or falls, i.e. during rapid and dramatic changes in user traffic.
Security testing is of primary importance as it reveals security vulnerabilities, threats, and risks and ensures secure protection of an app against cyber attacks. To provide secure protection of the sensitive data that most modern applications contain and detect all security loopholes before hackers find them, it is crucial to execute various types of security testing. Here are they:
This type of security testing suggests the use of special software that scans an AUT against known security vulnerabilities.
Security scanning detects network and system weaknesses for further elimination of these risks.
This type of security testing implies a comprehensive analysis of security risks in networks, servers, applications, etc., assessment of their criticality (low, medium, or high), and advising on preventive measures to be taken to mitigate them.
Security auditing considers every minor security flaw by inspecting every code line and app design. It helps to make sure that a proper security strategy has been applied and that it complies with the common security standards and norms.
This testing suggests the engagement of specialists who think in the same way as malicious hackers do. These specialists are ethical hackers who receive permission from a company to hack its system. Ethical hackers apply the most diverse techniques and tricks to find loopholes and get access to sensible data. On test completion, they report all found vulnerabilities and provide recommendations for improvements.
Penetration testing is a subtype of ethical hacking. It imitates different hacker attacks while app behavior is monitored and analyzed in order to reveal weak spots and understand whether it can withstand such attacks. In comparison with ethical hacking, penetration testing is one-dimensional, i.e. it targets a particular issue or vulnerability, while ethical hacking is more holistic.
Posture assessment is considered to be a combo of security scanning, ethical hacking, and risk assessment. It is conducted to review an overall security posture of a company.
Usability testing is an essential type of software testing executed to ensure that software is indeed easy to use, comfortable, intuitive, and user-friendly, i.e. that real users will be able to grasp on the fly the way an app should be used.
Localization testing is rather specific and suggests checking whether an app has been properly customized in terms of language and culture. It is relevant when an application targets the markets of several countries. Localization testing verifies content accuracy and layout, time and data format, phone number format, currency, etc.
Failover and recovery testing
This non-functional testing ensures that an application can allocate extra resources and recover from critical hardware crashes and failures.
Compliance testing is a must for applications of certain kinds. It is performed to check whether an application meets industry requirements and state norms and regulations. For instance, for healthcare apps targeting the market of the USA, it is essential to comply with the norms of HIPAA.
A developed application undergoes continuous change. A source code, as well as an environment, is constantly modified. So there is a group of tests related to changes.
Smoke testing is a short cycle of tests executed after code modification to make sure that an app can be launched and that its core functionality works as expected. Provided smoke testing is successfully completed, a QA team can get to more comprehensive testing of the app. Sometimes smoke testing is also regarded as Build Verification Testing.
Retesting is executed to assure that a bug caught in a previous build has been indeed fixed in the new one. In this case, only previously failed tests are rerun.
In the course of regression testing previously passed tests are rerun to check whether bug fixing or any other code modifications have not had a toll on the functionality that worked perfectly before.
QA team carries out sanity testing to check whether a particular piece of the app functionality works as expected after changes in an app or environment. It is executed prior to regression testing but after smoke testing. It can be conducted without any test cases just on the ground of domain knowledge.
Levels of software testing
Gone are the days when a QA team got to work only on the completion of the development process. Currently, software testing is simultaneous with software development. On the basis of a development phase at which testing is conducted, a few levels of testing are differentiated - unit testing, integration testing, system testing, and acceptance testing. Let's consider them in detail.
Unit testing comes first and is usually executed by developers. In its course, each isolated unit i.e. minimal testable software component representing a particular function or method is tested one by one. As a rule, unit tests are automated and prepared with the method of white box testing that we will discuss later. Unit testing ensures the proper work of each code unit and checks whether modifications of the root code have not affected app functionality. While being regularly run, unit testing helps detect bugs while the track is still hot. This approach significantly optimizes bug fixing and helps to save a ton of time, effort, and money.
It often happens that code units work perfectly in isolation, but fail to function properly while being combined, There can be problems with interworking and simultaneous operation. That is where integration testing comes. It concentrates mostly on the interfaces and data flow between software units. This time testing is executed by a QA team.
There are a few subtypes of integration testing - big bang testing and incremental testing.
Big bang testing
Big bang testing is a type of integration testing that suggests an integration of all available software units at once and their testing as an entity. This approach is relevant only for small and simple applications.
Increment testing is a type of integration testing in the course of which components are added and tested gradually. The testing process continues until all the related units are added and successfully tested. As long as incremental testing is executed while some software components are not available, stubs and drivers are actively used to substitute them and simulate data communication with them. Stubs are called by tested components, while drivers call tested components.
There are a few ways to execute incremental integration testing:
Top-down incremental testing
In the course of integration testing at first software components of the higher level are added and tested. Testing continues until components of the lowest level are added and tested as well.
Bottom-up incremental testing
Here everything is exactly the other way round - software components are gradually added from the ones of the lowest level to the ones of the highest level.
Hybrid incremental testing
When this approach is used, both low-level and high-level components are added and tested simultaneously.
In this case, a fully integrated system is tested, and either functional or non-functional tests are carried out which means that attention is also paid to such aspects as performance, compatibility, security, recovery, usability, etc. QA specialists are responsible for the execution of system testing.
User acceptance testing
User acceptance testing (UAT) is conducted when testing on all previous levels has been completed. Its objective is to make sure that an application under test corresponds with the needs of end-users and complies with business requirements. For this reason, UAT is executed not by a QA team but by end-users, a target group with corresponding domain knowledge, or a customer.
As a rule, by the time user acceptance testing is perfumed, all critical bugs have been already detected and fixed, therefore the main focus here is to check whether requirements are met and find out whether there is room for improvements. However, it is also worth noting that pretty often serious issues are uncovered at this level as testing is usually conducted in a real-world environment and by real users who can indicate errors that QA engineers could have missed out on due to the lack of domain knowledge.
On completion of UAT, a decision is made whether to give the green light to software release.
Software testing methods
There are three methods of software testing that are differentiated on the ground of how well a QA engineer knows the internal logic and structure of an application.
When black-box testing is applied, QA engineers test an app without having access to the source code and use GUI to verify functionality and check whether outputs and inputs comply with project requirements. Black-box testing is always executed from the end user's perspective and therefore it does not require experience in coding.
On the contrary, white-box testing suggests access to the root code and excellent knowledge of the inner structure. Consequently, specialists leveraging white-box testing must be good at programming as their aim is to analyze an internal structure and logic used behind it. Unit testing is the best example where white box testing is usually used.
Grey-box testing is a balanced blend of two previous methods. It has found an active use in integration testing. In this case, an application is tested by a specialist with limited access to the source code
Positive vs negative testing
Software testing can also be differentiated depending on the type of input values.
Positive testing is applied whenever a tester uses valid input values. In such a way we can check whether an application works as it is supposed to work as a user uses correct input data. Negative testing is applied to test an application against incorrect input values. This approach is equally important as the positive one, as it is crucial to make sure that an application adequately responds to invalid input data and displays the corresponding error notifications and warnings. QA engineers get to negative tests only after the positive ones to detect more bugs and enhance app quality.
Automation testing vs manual testing
And finally, here is our last category where we will consider manual and automation testing that are totally opposite approaches but which are equally important. As long as manual testing stands at the origins of software testing, we offer to start with this approach.
Manual testing is software testing executed by human testers without any automation tools.
Although a great number of automation tools are available on the market today, manual testing is still relevant as any artificial intelligence or machine learning technology cannot replace the human eyes and brain. Still, we must admit that manual testing can be rater time-consuming and labor-intensive when it comes to large-scope projects. One more drawback of manual testing is the high probability of human errors.
Nevertheless, manual testing is a way to go for simple and small projects and a necessity for large projects at certain development phases.
Automation testing is exactly what its name suggests. It is software testing executed with the help of automation tools and without or with minimum human intervention. To say that the advent of automation testing significantly simplified software testing would be an understatement. Automation testing has immensely optimized and speeded up a QA process and opened a lot of opportunities.
With the implementation of automation testing, an extensive suite of regression tests can be completed within a few hours instead of several days or even weeks. Just a pair of automation QA engineers can cope with a work scope that would require an army of manual testers which helps to reduce labor costs. Moreover, a QA team can be sure of testing outcomes as there is no risk of human error compared to manual testing. In addition, automation testing is a great way to reduce time-to-market and ship your software product within a short time frame without compromising on its quality.
However, automation testing also has certain disadvantages. First of all, it suggests significant upfront investment, and therefore it is not cost-efficient for small-scope projects. Secondly, not all test scenarios can be automated, and, frankly speaking, it does not make sense to automate everything. Finally, automated testing cannot replace manual testing, it is still used while the software is not stable enough, when exploratory testing is executed, and when critical functionality requiring special human attention is tested.
It is hardly possible to underestimate the importance of software testing for the success of an application of any kind. Nowadays, software testing is not a one-short activity but a complex of tests that helps to check all the ins and outs of an application. To get the most out of a QA process, the most diverse testing techniques, methods, and practices are used. Consequently, it has led to the existence of a plethora of different testing types that QA engineers actively use to provide superior quality of applications. Which tests to carry out and whether to automate them or to execute manually depends on the peculiarities and requirements of each particular project.