Introduction to Performance Testing

Introduction to Performance Testing

What is performance testing?

Performance testing is a non-functional testing technique performed to determine the system parameters in terms of responsiveness and stability under various workloads. Performance testing measures the quality attributes of the system, such as scalability, reliability and resource usage.

Performance testing is the testing, which is performed, to ascertain how the components of a system are performing, given a particular situation.

Goal of performance testing?

Performance testing is done to provide stakeholders with information about their application regarding speed, stability and scalability. More importantly, performance testing uncovers what needs to be improved before the product goes to market. Without performance testing, software is likely to suffer from issues such as: running slow while several users use it simultaneously, inconsistencies across different operating systems and poor usability. Performance testing will determine whether or not their software meets speed, scalability and stability requirements under expected workloads.

Common Performance problems

Most performance problems revolve around speed, response time, load time and poor scalability. Speed is often one of the most important attributes of an application. A slow running application will lose potential users. Performance testing is done to make sure an app runs fast enough to keep a user's attention and interest. Take a look at the following list of common performance problems and notice how speed is a common factor in many of them:

a)      Long Load time - Load time is normally the initial time it takes an application to start. This should generally be kept to a minimum. Load time should be kept under a few seconds if possible.

b)      Poor response time - Response time is the time it takes from when a user inputs data into the application until the application outputs a response to that input. Generally this should be very quick. Again if a user has to wait too long, they lose interest.

c)       Poor scalability - A software product suffers from poor scalability when it cannot handle the expected number of users or when it does not accommodate a wide enough range of users. Load testing should be done to be certain the application can handle the anticipated number of users.

d)      Bottle necking - Bottlenecks are obstructions in system which degrade overall system performance. Bottle necking is when either coding errors or hardware issues cause a decrease of throughput under certain loads. Bottle necking is often caused by one faulty section of code.

 Some common performance bottlenecks are

·         CPU utilization

·         Memory utilization

·         Network utilization

·         Operating System limitations

·         Disk usage

Need for Performance Testing

At the highest level, performance testing is almost always conducted to address one or more risks related to expense, opportunity costs, continuity, and/or corporate reputation. Some more specific reasons for conducting performance testing include:

• Assessing release readiness by:
• Enables you to predict or estimate the performance characteristics of an application in production and evaluate whether or not to application is ready for release in production.
• Providing data to aid in the prediction of revenue losses or damaged brand credibility due to scalability or stability issues, or due to users being dissatisfied with application response time.
• Assessing infrastructure adequacy by:
• Evaluating the adequacy of current capacity.
• Determining the capacity of the application’s infrastructure, as well as determining the future resources required to deliver acceptable application performance.
• Comparing different system configurations to determine which works best for both the application and the business.
• Improving the efficiency of performance tuning by:
• Analyzing the behavior of the application at various load levels.
• Identifying bottlenecks in the application.
• Providing information related to the speed, scalability, and stability of a product prior to production release, thus enabling you to make informed decisions about whether and when to tune the system.

Performance Testing Techniques:

• Load testing - It is the simplest form of testing conducted to understand the behaviour of the system under a specific load. Load testing will result in measuring important business critical transactions and load on the database, application server, etc., are also monitored.
• Stress testing - It is performed to find the upper limit capacity of the system and also to determine how the system performs if the current load goes well above the expected maximum.
• Soak testing - Soak Testing also known as endurance testing, is performed to determine the system parameters under continuous expected load. During soak tests the parameters such as memory utilization is monitored to detect memory leaks or other performance issues. The main aim is to discover the system's performance under sustained use.
• Spike testing - Spike testing is performed by increasing the number of users suddenly by a very large amount and measuring the performance of the system. The main aim is to determine whether the system will be able to sustain the workload.

Performance Testing Process

1. Identify your testing environment - Know your physical test environment, production environment and what testing tools are available. Understand details of the hardware, software and network configurations used during testing before you begin the testing process. It will help testers create more efficient tests.  It will also help identify possible challenges that testers may encounter during the performance testing procedures.
2. Identify the performance acceptance criteria - This includes goals and constraints for throughput, response times and resource allocation.  It is also necessary to identify project success criteria outside of these goals and constraints. Testers should be empowered to set performance criteria and goals because often the project specifications will not include a wide enough variety of performance benchmarks. Sometimes there may be none at all. When possible finding a similar application to compare to is a good way to set performance goals.
3. Plan & design performance tests - Determine how usage is likely to vary amongst end users and identify key scenarios to test for all possible use cases. It is necessary to simulate a variety of end users, plan performance test data and outline what metrics will be gathered.
4. Configuring the test environment - Prepare the testing environment before execution. Also, arrange tools and other resources.
5. Implement test design - Create the performance tests according to your test design.
6. Run the tests - Execute and monitor the tests.
7. Analyse, tune and retest - Consolidate, analyze and share test results. Then fine tune and test again to see if there is an improvement or decrease in performance. Since improvements generally grow smaller with each retest, stop when bottle necking is caused by the CPU. Then you may have the consider option of increasing CPU power.

Challenges in Performance Testing

·         Inadequate non-functional requirements for performance testing

·         Inability to customize the tool output to develop load generation script

·         Scaled Down Performance Test Environment

·         Performance Testing Tool selection

·         Large amount of Test Data

Attributes of Performance Testing:

·         Speed

·         Scalability

·         Stability

·         reliability

Performance Testing Tools

·         Jmeter  (Open Source)

·         Load Runner

·         Web Load

·         Open STA (Open Source)