Unified communications systems today carry a far greater level of complexity that even a few years ago. The level of complexity can vary, depending on the organization and industry. UC environments are also prone to regular change, and in today's workplace, not only are there ongoing upgrades, additions and improvements, but remote working has added a whole other level of intricacy.
So how do you know when these changes affect the overall experience for users and customers, and ensure peak performance from all your systems? The answer is performance testing.
Performance testing will help ensure your software meets the expected levels of service and provide a positive user experience. Performance testing will highlight improvements you should make to your applications relative to speed, stability, and scalability before they go into production. If you release applications without testing, you 'll almost certainly encounter a variety of different types of problems that could lead to a damaged brand reputation. The adoption, success, and productivity of applications depends directly on properly implementing performance testing.
In this guide, we'll look at four key types of performance testing: Load testing, stress testing, capacity testing, and soak testing. We'll describe why performance testing is crucial, focusing on two performance testing subsets in particular: Load testing and stress testing.
Find out how you can test your entire technology ecosystem and prevent your business from lagging behind with our guide.
While load and stress testing are different, they're not independent from each other. Let's examine the differences.
A load test ensures that a network system can handle an expected volume of traffic and how it behaves when bombarded with specific levels of simultaneous requests. This is unlike stress testing which assesses the robustness of a system with requests beyond the upper limit.
The goal of load testing is to prove that a system is capable of handling its load limit, with minimal to acceptable performance degradation. Before carrying out a load test, the threshold of acceptable performance degradation needs to be pre-defined by the testers.
The example in the graph below, shows a load of 20 users, testing to see that the page time does not exceed 3.5 seconds.
Image source: Radview
Load testing should be done when you want to test how many users your system can actually handle. You can configure a load test to simulate various user scenarios focusing on different parts of your system. You can also test how the load behaves when coming from different geo-locations, or how the load might build up, then level out. A load test should be performed regularly to ensure your system is always spot on.
Load testing is best performed in a production environment to understand average response times under expected user load. These average response times become the baseline for an acceptable Service Level Agreement (SLA). From here, testers can determine additional load thresholds that are considered unacceptable in terms of expected performance for optimal customer experience.
Read our blog on cloud load testing vs on-premises load testing
Stress tests are designed to increase the number of simultaneous requests on a system beyond the upper limit of resource usage, and where performance is degraded - even to the point of complete failure. Where a load test will peak out in the number of simultaneous users, a stress test will continue to increase load on the system capacity until the resources are overloaded. This type of performance testing pushes the system to a state of potential failure, to see how the system behaves, and whether it recovers successfully.
While load testing ensures that a given function, program, or system can simply handle its expected load, stress testing is about overloading a system past its upper limits, until it breaks. Both stress testing and load testing play important roles in determining exactly how well a given piece of front-end software, such as a website, or a back-end system can deal with their actual load.
Stress testing deliberately induces a failure scenario under extreme load, so that you can analyze the risk involved at the breaking points. This helps with capacity planning, for example, by creating the opportunity to tweak programs to make them break more gracefully. Stress testing prepares for the unexpected, fine tuning response time, and through spike testing, determines exactly how far a given system can be pushed. This method of performance testing measures the outer limits of performance capacity.
Depending on the application, software, or technology being used in your environment/system, what's measured during a stress test can vary, but some of the metrics include overall performance issues, unexpected traffic load spikes, memory leaks, bottlenecks and more:
The example stress test below shows that up to a load size of 41 users, the system functions, despite the increase in page time. But when the load size reaches 42 users, unexpected traffic spikes cause a deterioration, with page time reaching 15-17 seconds.
Image source: Radview
Stress testing enables IT teams to interpret the performance of system software during its failures, allowing them to identify and analyze the behavior of the software. Stress testing enables test teams to check whether data has been saved by the software before crashing and verify whether any lost data can be restored.
Stress testing also enables test teams to understand the behavior of software before crashing. There are many case scenarios in eCommerce where performance testing proves invaluable. For example holidays or promotions like Black Friday, or Cyber Monday, when a website might witness a sudden influx of visitors, causing an increased load, and severe performance issues, or even a total collapse. The inability to handle such massive spikes could lead to loss of revenue and reputation.
Below is a screenshot of a website crash message from Walmart in November 2020, due to heavy load volumes during a buying frenzy on Playstation 5.
Let's look at 5 basic types of stress testing techniques:
Image source: Try QA
In this type of performance testing, all the clients linked with the website servers are tested. Distribution of a group of stress tests to each one of the clients as well as a follow up on their status, is the server’s role and responsibility.
A transactional stress test is used to test the transactions that occur between the applications. The purpose of this method of performance testing is to fine tune and optimize the system.
Stress testing of applications is typically used to identify bottlenecks in performance, network issues, data blockages and locks .
In exploratory stress testing, performance testing is carried out under abnormal conditions which will not likely to occur in a real time scenario:
Systematic stress testing is used to test many systems that run on website servers. It enables the testing of defects where one software interferes with another.
Memory leak is a type of resource leak that happens when a software incorrectly manages memory allocation. Memory leak can also happen when an object is stored in memory but cannot be accessed by the running code. A dynamic analysis tool that can track memory leaks generally monitors both the allocation and de-allocation of memory.
Capacity testing is sometimes called scalability testing, and helps identify the maximum capacity of users the system can support, while not exceeding a maximum defined page time.
The main objective is to identify the ‘safety zone’ for peak performance of your system, and to what extent can you extend it, without hurting end user experience.
In the results graph below, for a page time of 3.5 seconds, the system supports 28 users. But when the load size reaches 29, page time starts exceeding the threshold of 3.5 seconds.
The data gleaned from capacity testing is crucial to a business. It's a fact that 40% of users will abandon a site if it takes more than 3 seconds to load. Just a one second delay can cause small to medium sized eCommerce websites to bleed over $250,000 annually. As per Marketing Bulldog, 79% of customers who report dissatisfaction with website performance are less likely to return.
Image source: Radview
The purpose of a soak test is to reveal performance problems that surface only over a long period of time. Soak testing highlights issues such as:
Soak testing looks for trends and changes in system behavior. In the use case below, there is a constant load size, but then, after 50 minutes of running the test, there’s an increase in memory, indicated by the red line, which hurts throughput. This is due to a periodical, background process that affects the system when it starts running. A typical soak test can last hours, days or even weeks.
Image source: Radview
Metrics are the key performance indicators (KPIs) that determine the success of each performance test. The most commonly used metrics include:
Average transaction response time - the average time taken to perform transactions during each second of the scenario.
Total transactions per second - the total number of transactions that passed, the total number of transactions that failed and the total number of transactions that stopped during the performance test run.
Transactions per second - for each transaction, the number of times it passed, failed or stopped during each second of the performance test.
Transaction response time (under load) - transaction times relative to the number of users at any given point during performance tests.
Errors per second - Average number of errors that occur during each second of performance tests.
Hits per second - the number of HTTP requests made by users to the web server during each second of the performance test.
In today's business world, Interactive Voice Response (IVR) systems are an important part of an organization's customer service and communication infrastructure. IVRs are used by almost all industries and their respective applications like banking, insurance, utilities, telecom, travel, retail etc. to process a customer's phone call, rather than connecting to an actual human agent.
Performance testing tracks metrics that allow you to confirm your entire environment works as expected, meaning that CPU & memory consumption don't go out of range, response times stay within acceptable ranges and network occupancy levels stay in the safe zone.
Performance testing an organization's IVR system is crucial to maintain a positive customer experience. IR's customer experience testing solutions provide accurate analytics and valuable IVR system information.
Read more about the importance of IVR performance testing and how it can ensure that your customer experience is the best it can be.