Efficient software testing is rather multifaceted and includes a great variety of tests. Performance testing is one of the high priority. Why is it so? The thing is that software performance not only has an impact on the user experience in general but also defines the business opportunities for further growth. As you can see it makes sense to consider this type of testing in depth. So let’s go into the nitty-gritty.
What is software performance testing
So if you are still unaware of what performance testing is, let’s give a definition to it. Performance testing is non-functional software testing executed to evaluate such characteristics as the speed, scalability, stability, and reliability as an application undergoes a particular level of workload. The prime objective of performance testing is to detect and eliminate performance problems before the launch of an application.
In the course of performance testing, diverse metrics can be collected and analyzed. The most important among them are average load time, bandwidth, response times, throughput, garbage collection, memory/CPU/disk space usage, amount of latency, the maximum number of concurrent users, error rate, thread counts, and others.
What are the types of performance testing
Performance testing can be conducted in different ways depending on the set goals. Consequently, a few types of performance testing can be differentiated.
Load Testing
Load testing is executed with the aim to assess app behavior while the workload, i.e. the number of concurrent users/transactions is gradually increasing but does not exceed the estimated limit. This type of performance testing is mostly carried out to identify response time, resource utilization, and throughput rate.
Stress Testing
Stress testing is similar to load testing but in this case, the workload level goes far beyond the estimated limits. This testing concentrates on the hardware parts, like CPU, memory, hard disk, etc. to define the potential influence of an AUT on these limited device resources.
Endurance Testing
This is one more type of performance testing evaluating the behavior of an application as it undergoes the normal but long-lasting workload. Its objective is to check the long-term capability of the application.
Spike Testing
In the course of spike testing app behavior is examined as workloads are rapidly, substantially, and repeatedly increased. The workloads are short-term but go far beyond normal limits.
Scalability Testing
Scalability testing is just indispensable when it comes to planning capacity growth. The reason is that it defines the effectiveness of an application in ‘scaling up’ or in other words handling an increase in the number of concurrent users.
Volume Testing
Volume testing checks whether an application works as expected as it has to proceed large data volumes.
How to execute performance testing
Performance testing is rather complex and cannot be carried out on the fly. On the contrary, it requires substantial preparation and is carried out in a number of steps, described below.
Test planning
So thorough planning is crucial for performance testing. Before getting down to the active actions, it is necessary to study the architecture of an application under testing, available specifications, the behavior of real users, etc. in order to define goals of performance testing and arrive at the right decisions as for the following points:
the scope of testing, i.e. types of performance testing to be executed
performance metrics to be collected in the course of performance testing
acceptance criteria to be used as benchmarks
appropriate performance testing tools
requirements for a test environment
the most critical business scenarios to script
an appropriate workload model, etc.
Whenever certain questions remain undetermined, it is always a good idea to get clarification from a project manager, business analyst, or other stakeholders who are armed with the information you need.
Test environment setup
As long as requirements for the test environment have been already determined, the team can get to its setting up which suggests configuration of corresponding hardware, software, databases, etc.
Test design
At this stage, the performance tests are designed with account of critical business scenarios, proper user variability, test data identification and generation, and specification of the metrics to be collected in the course of the test run.
The workload model as well as the nature and number of scenarios in which the app performance is going to be challenged must be as much similar to real-world ones, as possible.
Test execution
Now, it is high time to proceed to the execution of prepared performance tests, monitor app performance in real time, collect logs, and required performance metrics.
Analysis of test outcomes, app tuning, and retesting
As soon as the test run is completed, a team investigates the obtained test outcomes and metrics, generate reports, and share insights with the rest of the product team. In case certain bottlenecks have been identified, a development team performs the recommended improvements and performance testing is executed again to make sure that an application has been properly tuned, and current performance characteristics meet acceptance criteria.
Performance Testing Requirements
In this section, we’d like to consider in detail some aspects that we have already mentioned but which definitely require much more attention. So, let’s talk about the things that are important to provide efficient performance testing execution.
Test environment identification
The nature or characteristics of a test environment are more critical for performance testing compared to functional testing.
When it comes to the execution of performance tests it is crucial to set up a test environment that replicates the production environment in the best possible way.
In these terms, it is strongly recommended to determine the number of servers and their model, hardware version, operating systems, types of configuration files, memory characteristics, data level and data volume in a database, etc.
Acceptance criteria identification
To understand whether performance tests are passed or failed it is important to be guided by certain benchmarks or acceptance criteria that outline the way application should behave in terms of performance. So understanding which performance metrics should be checked you should define their appropriate values. You can get this information from the project requirements or by communicating with stakeholders. If the stakeholders are out of the subject as well, it is considered to be a good practice to study and analyze similar applications to understand which values of response time, load time, CPU consumption, etc. must be considered as acceptable.
Key scenario identification
Scenarios can be defined as user paths that usually incorporate numerous application activities. Key scenarios are those that are considered to be high-risk, most commonly used, or with a considerable performance impact.
It is essential to script key scenarios to make performance testing as realistic as possible.
In order to define such scenarios, performance test engineers should closely collaborate with a product owner, business analysts, and other stakeholders. If the software has been already launched, analytics tools can be used to identify users’ behavior and detect the most frequently used scenarios.
Proper performance testing tool selection
A properly selected testing tool or even rather a set of tools significantly contributes to the efficiency and veracity of performance testing. Today there is a wide variety of performance testing tools on the market. Still, none of the available tools is versatile enough to meet all your performance needs. Therefore, you can hardly be satisfied with a single tool. So choose a few tools that are the most suitable for your project requirements and let you execute the whole set of performance tests.
Specifics of the performance testing process
As it was mentioned, performance testing is rather complex. Therefore, to execute it in the right way, get the most out of it, and receive valuable insights it is necessary to know certain peculiarities. So, let’s shed the light on some of them.
Pay attention to the whole spectrum of performance tests
The substitution of performance testing by load testing is a common tendency these days for many teams. Still, it is worth noting that in essence performance testing is not restricted to load testing only. Instead, it also includes the whole host of other tests like stress testing, volume testing, endurance testing, strike testing, and scalability testing. To detect all possible performance issues and be confident in the robustness, speed, and scalability of an application under test it is essential to carry out the entire complex of performance tests.
Design realistic tests reflecting the actual behavior of real users
To make performance testing as efficient as possible and take advantage of accurate test results, it is strongly advised to design testing suites so that they reflect the real-world activity. Particular attention must be paid to the users’ ‘think time’ that can vary greatly. For this reason, it is a good practice to randomize think time. End users react in a different way and performance testing should take into account this reality.
This approach is far from being perfect. In this situation, it's usually too late to fix issues easily and at minimum expense.
To avoid this problem, it is advised to be proactive and take an agile approach by introducing iterative testing throughout the whole software development life cycle. It is always a good idea to execute performance testing as a part of unit testing and then regularly repeat performance testing execution as the application grows. That way you can detect performance problems while the trail is still hot and tackle them quickly and easily.
Conclusion
Performance testing is not what you can ignore if you are eager to hit the market with a high-quality product and have ambitious plans for further growth. To detect all possible performance issues and eliminate all risks in a timely manner performance testing can be executed in a few ways. Consequently, several types of performance testing can be differentiated. To get the desired efficiency of performance testing it is advised to adhere to certain steps of performance testing execution that were described above. Here we cannot help but state that the first step that is test planning is indeed essential. The thing is that it is impossible to carry out this type of testing properly without a prior comprehensive study, analysis, and planning. Moreover, there is also the whole range of peculiarities that must be considered in order to get indeed reliable testing outcomes and let a product team reach the performance goals and provide the capability of an application to scale up.