In the IT industry, the concept of tests and testing comes up extremely frequently – especially in projects based on Agile methodology, there is always a place for a tester. One of the often neglected areas of testing is performance evaluation. Performance testing is a good software development practice and will allow you to gather important information about the health of your application.
This article will give you an overview of what performance testing is, why it's important, and what types of tests there are. On top of that, we are going to introduce situations that will convince you to use performance testing yourself.
What are the most common tests?
There are many types and levels of testing that additionally interact with each other, which may confuse you at first.
Testing can be performed manually or automatically. Types of software testing include: functional testing, non-functional testing, structural testing, and change-related testing (regression testing). With regard to the level of testing, the following tests can be distinguished: unit tests, integration tests, system tests and acceptance tests. Regardless of which type of testing we choose, they have common goals – minimizing product implementation risk, ensuring quality, and meeting the expectations of owners and their customers.
Learn some facts about performance testing
In most projects there is a need for performance testing, so in the following section we would like to focus on the most important characteristics of this type of testing.
What is performance testing and why is it needed?
Performance testing belongs to the category of non-functional test procedures that, in general, determine HOW the system works. It consists in simulating system load based on the developed scripts.
Performance testing allows to determine what load can be handled by the system without a noticeable drop in performance (e.g. response time when processing a large number of queries, receiving by the system a large amount of data) and whether it will continue to perform as normal under the defined workload. The idea is to simulate user behavior in a controlled environment before the effects occur in a production environment. This proves particularly useful for large-scale systems.
Types of performance testing
Depending on the aspect of the application that we want to test under high load conditions, we can identify several types of performance testing.
- Load Testing – How does the system perform under expected load (i.e., the load does not exceed a critical point)? This helps identify bottlenecks that impact an application’s performance.
- Stress Testing – Is the system capable of operating under heavy load?
- Spike Testing – How does the application handle sudden, large spikes in load?
- Endurance Testing – Can the software handle increased load over a long period of time?
- Volume Testing – How does the system that stores and processes large amounts of data respond to a simulated load?
- Scalability Testing – To what loads the application can be scaled up?
Performance testing: Do you really need it?
Performance testing gives us a complete picture of how a system or application is performing. Find out how valuable every second is to the business in terms of resource loading and what you are exposing yourself to by skipping the testing stage.
What do you gain from performance testing?
Performance testing plays an important role in quality assurance and minimizes the risk of problems as it allows you to find and fix bugs that have been generated by increased traffic earlier. Having clearly defined limits of technical capabilities of the application, we are able to estimate the resources necessary for proper operation. As a result of performance tests, the system will become resistant to overloading, and you will be prepared for sudden, unexpected spikes in users (without worrying about system stability and availability). In the end, you can expect increased customer satisfaction.
What are the risks associated with a lack of performance testing?
The system can handle a finite number of requests and performs very well in a limited environment, with weaknesses only becoming apparent as traffic increases. If the website suddenly becomes very popular, the situation gets out of control. The server gets overloaded and the system is suspended – the customers can’t sign in, finalize the order, so they can't expect efficient service. Such negligence means that your business goals will not be achieved, you will suffer great financial losses and experience changes in your company's image. Besides, lack of optimization in terms of performance and page load speed will cause a drop in your position in search results.
Test cases
To discuss the cases, we will use K6, a state-of-the-art performance testing toolkit, which allows you to verify the business and technical assumptions of your project. K6 is an open-source tool designed for engineering teams, including Software Testers, QAs, DevOps. K6 testing can also exist as a cloud-based service.
- Product life scenario #1
We perform optimization enhancements that should improve product performance. We don't anticipate an impact on the increased number of users, but the changes may affect the internal processes of the system.
Example: Current user traffic generates 80% of the server load – Will the server load decrease after implementing the changes?
A real case
One functionality was improved (faster generation of images on the website), and several libraries that were needed for optimization were updated.
After implementation, the effect was opposite to expected: the system performed worse with the same user traffic. Pages were loading very long or not loading at all. It was a mistake to assume that the page display process would work properly for all users when the results for a single user improved. The fact was that the page with the image had a better load time, but that didn't work for more users. The assumption addressed the final outcome, but omitted some of the system processes that must work to achieve that outcome. The processes were not controlled in manual testing because this type of testing could not indicate a small difference in system resource requirements.
Solution
Load testing should be selected as the test type because the assumed user behavior is not changed, only the system verification is performed. In this case, you need to assume what user traffic is constant and compare the results before and after the changes.
Source: https://k6.io/docs/test-types/load-testing/
- Product life scenario #2
We are implementing a new functionality and the marketing department is promoting it and encouraging users to try it out. Traffic of active users is expected, but in addition to this, there may be users who have not been active for a long time. Extra traffic can also be generated by random visitors to the site who have only become interested in the functionality.
Example: Current user traffic generates 80% of server load – What happens when we reach 90%, 95%, 100% load? How will the architecture of our product try to defend itself so that it runs stably? What happens when the number of users increases to 120% or even 150%?
A real case
Within a short period of time (15 minutes after the post was shared on social media), the huge user interest resulted in the entire product being unavailable. The servers went down because they were not able to function properly. This was caused by errors in environment configuration and in database queries that used too many server resources and were not optimized.
As a result, only some users managed to explore the new functionality. The others already had negative feedback on the performance of the product.
Solution
Stress testing should be selected as the test type because the expected user behavior can change in a short period of time. In this case, you need to know the typical user traffic, but also assume that it will increase significantly to different values.
Source: https://k6.io/docs/test-types/stress-testing/
In the cases presented, a certain analogy can be drawn. Neglecting performance testing caused both products to face serious problems.
It should be kept in mind that software development is not limited to writing code according to specific client requirements and then “it'll work out somehow”. Testing is an important stage of product development which, from the business point of view, should be planned in advance. Nowadays we are dealing with more and more complex applications with diverse architecture and complicated logic. The use of multiple technologies entails dependencies between components that are difficult to predict. The selection of tools and the ability to verify the correct operation is critical to the final success of the product. By performing these tests, we are confident that the application will not generate problems, and the stable and fast operation will be appreciated by users.
How do you go from a business plan to performing a test? This and other questions will be answered in our next article.