The idea behind the load test is simulating that multiple users execute the logic of the solution simultaneously. The primary goal of the load test is to discover scalability and performance issues in the solution. Testing the solution also gives benchmark statistics on performance as the solution modified or extended.
The tester creates a load test in the VSTS environment using a wizard that steps through the process of defining a load test scenario. A scenario is a certain situation of the use of the application and is defined with a load pattern, test mix, browser mix, and network mix.
The load pattern defines how the load is applied to the application and has two flavors, a constant and a step load definition. The constant load definition applies a continuous load of users on the applications. For example, when the tester sets the maximum user count setting on 50, the load test continuously simulates 50 users stressing out the applications. This option is useful for peak usage testing and stability testing to see how the system performs under constant peak stress. The step load definition starts the load test with a small number of users and adds a predefined amount each step until the maximum is reached, or the system performs badly, or the solution falls over. This option is suitable for testing the performance limitations of the system, like the maximum number of users is before the system falls.
The test mix defines which tests are included in the load test and how they are distributed. The tester can select all the automated test types that are stored in the test projects of the solution. When selecting more than one test, the tester needs to define what percentage of the total load capacity should be executing each test, as shown in the figure below.
The browser mix to define the kinds of browser to use for web tests. The browsers types include various versions of Microsoft Internet Explorer, Netscape, Mozilla, Pocket Internet Explorer, and smartphones profiles.
The network mix defines the kinds of network types are simulated, and ranges from LAN, dial-up through T3 connections for the load test.
After finishing the New Load Test Wizard, VSTS gives the tester an overview of all properties of the load test as depicted in the figure below.
After defining the way to test the load on the system, the tester needs to identify what information is important to collect during the test. Setting these so called counter sets is a difficult task of the tester. Fortunately VSTS provides default counter sets to simplify the creation of the load test.
Finally, the tester has to set the load test’s Run Settings, where it is decided how much time the test needs to warm up, how long the test runs, and what the sample rate is. The warming up time is the time between the start of the test and the first sample to be taken of the counters. For example, the system starts slow the first minute due to the lack of caching files and the tester does not want this warming up to influence the test results.
The sample rate is the time interval how often the counters are sampled. For instance, the tester prescribes VSTS to take test data samples of the counters every ten seconds.
During the execution of the load test, the tester is able to monitor the counters in real time and add or remove counters. The tests results, as shown in the figure below, are presented in tables and graphs, which also can be modified during the test run by adding or removing counters.
The test results are stored in a database or XML-file on the Visual Studio Team Foundation Server to be available to all the team members or other stakeholders. From a failed test, a tester is able to create a bug work item that is linked to these test results to be addressed to a developer. The VSTS project portal website provides the managers and other team members with extensive test and bug-tracking reports to track bug work items.
Test Load Agent
For large – scale applications, one computer might not be enough to simulate the desired load. Visual Studio Team System 2008 Test Load Agent can distribute the work across different machines. It can simulate approximately 1,000 users per processor. This product requires a separate installation and requires one controller and at least one agent. To configure the environment select Administer Test Controller from the Test menu. There you can select a Controller and add Agents. Then from the Test Run Configuration window in the Controller and Agent node you can select to run the tests remotely and select the configured controller.
Recent Comments