This is a very useful book for Load Test Professionals and for developers. This guide shows you an end-to-end approach for implementing performance testing. Whether you are new to performance testing, or looking for ways to improve your current performance testing approach, you will find insights that you can tailor for your specific scenarios.

This guide is related to our Performance Testing Guidance Project. – J.D. Meier, Carlos Farre, Prashant Bansode, Scott Barber, Dennis Rea

Download the Guide/Ebook

Start using the guide today, while we continue to make improvements.

• Download the Performance Testing Guidance for Web Applications Guide

Parts

• Part 1, Introduction to Performance Testing
• Part II, Exemplar Performance Testing Approaches
• Part III, Identify the Test Environment
• Part IV, Identify Performance Acceptance Criteria
• Part V, Plan and Design Tests
• Part VI, Execute Tests
• Part VII, Analyze Results and Report
• Part VIII, Performance Testing Techniques

Chapters

• Introduction

Part 1, Introduction to Performance Testing

• Chapter 1 – Fundamentals of Web Application Performance Testing
• Chapter 2 – Types of Performance Testing
• Chapter 3 – Risks Addressed Through Performance Testing

Part II, Exemplar Performance Testing Approaches

• Chapter 4 – Web Application Performance Testing Core Activities
• Chapter 5 – Coordinating Performance Testing with an Iteration-Based Process
• Chapter 6 – Managing an Agile Performance Test Cycle
• Chapter 7 – Managing the Performance Test Cycle in a Regulated (CMMI) Environment

Part III, Identify the Test Environment

• Chapter 8 – Evaluating Systems to Increase Performance-Testing Effectiveness

Part IV, Identify Performance Acceptance Criteria

• Chapter 9 – Determining Performance Testing Objectives
• Chapter 10 – Quantifying End-User Response Time Goals
• Chapter 11 – Consolidating Various Types of Performance Acceptance Criteria

Part V, Plan and Design Tests

• Chapter 12 – Modelling Application Usage
• Chapter 13 – Determining Individual User Data and Variances

Part VI, Execute Tests

• Chapter 14 – Test Execution

Part VII, Analyze Results and Report

• Chapter 15 – Key Mathematic Principles for Performance Testers
• Chapter 16 – Performance Test Reporting Fundamentals

Part VIII, Performance-Testing Techniques

• Chapter 17 – Load-Testing Web Applications
• Chapter 18 – Stress-Testing Web Applications

Download the Guide/Ebook

Start using the guide today, while we continue to make improvements.
• Download the Performance Testing Guidance for Web Applications Guide

Abstract: Agile development methods are becoming more and more common in our projects. The objective is to achieve higher quality and shorter lead times with a minimum of overhead. With frequent deliveries, close cooperation within the agile teams and the customer, continuous integration, short feedback loops and frequent changes of the design creates new challenges but also new opportunities for testing.

A well defined agile test strategy guides you through the common obstacles with a clear view of how to evaluate the system. Testing starts with the exploration of the requirements and what the customer really wants. By elaborating on the User Stories from different perspectives, both efficient and effective test ideas can be created. Testing becomes a continuous and integrated process where all parties in the project are involved.

Click Here to Download the Ebook

Structural and Functional Technique

• Both Structural and Functional Technique is used to ensure adequate testing
• Structural analysis basically test the uncover error occur during the coding of the program.
• Functional analysis basically test he uncover occur during implementing requirements and design specifications.

• Functional testing basically concern about the results but not the processing.
• Structural testing is basically concern both the results and also the process.
• Structural testing is used in all the phases where design , requirements and algorithm is discussed.
• The main objective of the Structural testing to ensure that the functionality is working fine and the product is technically good enough to implement in the real environment.
• Functional testing is some times called as black box testing, no need to know about the coding of the program.
• Structural testing is some times called as white box testing because knowledge of code is very much essential. We need the understand the code written by other users.
• Various Structural Testing are
  1. Stress Testing
  2. Execution Testing
  3. Operations Testing
  4. Recovery Testing
  5. Compliance Testing
  6. Security Testing

• It is – Ability to restart the operations after the integrity of the application is lost.
• Objective of the recovery testing is to ensure that the operations can continue even though the disaster is happening.
• Recovery testing includes:
  1. Storage of data in the preserved location.
  2. Documents need to maintain properly about recovery.
  3. Assign the professional testers involving in Recovery process.
  4. Develop a recovery tool and make it available all the times.
• Recovery testing is normally takes care by professional testers, System analysts you know entire flow of the applications.

• It is advised that the operational staff need not involved in recovery testing and only professionals need to be involved.
• Recovery can be conducted in two modes:

1. Either with the recovery tools and the set of the methods and procedures to resolve it.
2. Once the system is developed and the failure is introduced in to the system and the ability to recover it.

The user should estimate the potential loss inability to recover over the time spans. Ability not to recover may lead to the business loss. The potential loss is calculated in such that the amount of resource used for recovery testing.

Application programmable Interfaces (APIs) are collections of software functions or procedures that can be used by other applications to fulfill their functionality. APIs provide an interface to the software component. These form the critical elements for the developing the applications and are used in varied applications from graph drawing packages, to speech engines, to web-based airline reservation systems, to computer security components.

Each API is supposed to behave the way it is coded, i.e. it is functionality specific. These APIs may offer different results for different type of the input provided. The errors or the exceptions returned may also vary. However once integrated within a product, the common functionality covers a very minimal code path of the

API and the functionality testing / integration testing may cover only those paths. By considering each API as a black box, a generalized approach of testing can be applied. But, there may be some paths which are not tested and lead to bugs in the application. Applications can be viewed and treated as APIs from a testing perspective.

There are some distinctive attributes that make testing of APIs slightly different from testing other common software interfaces like GUI testing:


Testing APIs requires a thorough knowledge of its inner workings – Some APIs may interact with the OS kernel, other APIs, with other software to offer their functionality. Thus an understanding of the inner workings of the interface would help in analyzing the call sequences and detecting the failures caused.

Adequate programming skills – API tests are generally in the form of sequences of calls, namely, programs. Each tester must possess expertise in the programming language(s) that are targeted by the API. This would help the tester to review and scrutinize the interface under test when the source code is available.

Lack of Domain knowledge – Since the testers may not be well trained in using the API, a lot of time might be spent in exploring the interfaces and their usage. This problem can be solved to an extent by involving the testers from the initial stage of development. This would help the testers to have some understanding on the interface and avoid exploring while testing.

No documentation – Experience has shown that it is hard to create precise and readable documentation. The APIs developed will hardly have any proper documentation available. Without the documentation, it is difficult for the test designer to understand the purpose of calls, the parameter types and possible valid/invalid values, their return values, the calls it makes to other functions, and usage scenarios. Hence having proper documentation would help test designer design the tests faster.

Access to source code – The availability of the source code would help tester to understand and analyze the implementation mechanism used; and can identify the loops or vulnerabilities that may cause errors. Thus if the source code is not available then the tester does not have a chance to find anomalies that may exist in the code.

Time constraints – Thorough testing of APIs is time consuming, requires a learning overhead and resources to develop tools and design tests. Keeping up with deadlines and ship dates may become a nightmare.

Testing of API calls can be done in isolation or in Sequence to vary the order in which the functionality is exercised and to make the API produce useful results from these tests. Designing tests is essentially designing sequences of API calls that have a potential of satisfying the test objectives. This in turn boils down to designing each call with specific parameters and to building a mechanism for handling and evaluating return values.

Thus designing of the test cases can depend on some of the general questions like

• Which value should a parameter take?
• What values together make sense?
• What combination of parameters will make APIs work in a desired manner?
• What combination will cause a failure, a bad return value, or an anomaly in the operating environment?
• Which sequences are the best candidates for selection? etc.

Read Part 2: API Testing – Points to take care

Points to take care while planning for API Testing:
1. Ensuring that the test harness varies parameters of the API calls in ways that verify functionality and expose failures. This includes assigning common parameter values as well as exploring boundary conditions.

2. Generating interesting parameter value combinations for calls with two or more parameters.

3. Determining the content under which an API call is made. This might include setting external environment conditions (files, peripheral devices, and so forth) and also internal stored data that affect the API.

4. Sequencing API calls to vary the order in which the functionality is exercised and to make the API produce useful results from successive calls.

By analyzing the problems listed above, a strategy needs to be formulated for testing the API. The API to be tested would require some environment for it to work. Hence it is required that all the conditions and prerequisites understood by the tester. The next step would be to identify and study its points of entry. The GUIs would have items like menus, buttons, check boxes, and combo lists that would trigger the event or action to be taken. Similarly, for APIs, the input parameters, the events that trigger the API would act as the point of entry. Subsequently, a chief task is to analyze the points of entry as well as significant output items. The input parameters should be tested with the valid and invalid values using strategies like the boundary value analysis and equivalence partitioning. The fourth step is to understand the purpose of the routines, the contexts in which they are to be used. Once all this parameter selections and combinations are designed, different call sequences need to be explored.

The steps can be summarized as following:
1. Identify the initial conditions required for testing.
2. Identify the parameters – Choosing the values of individual parameters.
3. Identify the combination of parameters – pick out the possible and applicable parameter combinations with multiple parameters.
4. Identify the order to make the calls – deciding the order in which to make the calls to force the API to exhibit its functionality.
5. Observe the output.