Test Plan Preparation Guidelines – Part 1

The software test plan is the primary means by which software testers communicate to the product development team what they intend to do. The purpose of the software test plan is to prescribe the scope, approach, resource, and schedule of the testing activities. To identify the items being tested, the features to be tested, the testing tasks to be preformed, the personnel responsible for each task, and the risks associated with the plan.
The test plan is simply a by-product of the detailed planning process that’s undertaken to create it. It’s the planning that matters, not the resulting documents. The ultimate goal of the test planning process is communicating the software test team’s intent, its expectations, and its understanding of the testing that’s to be performed.
The following are the important topics/guidelines, which helps in preparation of Test plan:

High-Level Expectations
The first topics to address in the planning process are the ones that define the test team’s high-level expectations. They are fundamental topics that must be agreed to, by everyone on the project team, but they are often overlooked. They might be
considered “too obvious” and assumed to be understood by everyone, but a good tester knows never to assume anything.

People, Places and Things
Test plan needs to identify the people working on the project, what they do, and how to contact them. The test team will likely work with all of them and knowing who they are and how to contact them is very important. Similarly, where documents are stored, where the software can be downloaded from, where the test tools are located, and so on need to be identified.

Inter-Group Responsibilities
Inter-Group responsibilities identify tasks and deliverable that potentially affect the test effort. The test team’s work is driven by many other functional groups – programmers, project manages, technical writers, and so on. If the responsibilities aren’t planned out, the project, specifically the testing, can become a worst or resulting in important tasks been forgotten.

Test phases
To plan the test phases, the test team will look at the proposed development model and decide whether unique phases, or stages, of testing should be performed over the course of the project. The test planning process should identify each proposed test phase and make each phase known to the project team. This process often helps the entire team from and understands the overall development model.

Test strategy
The test strategy describes the approach that the test team will use to test the software both overall and in each phase. Deciding on the strategy is a complex task one that needs to be made by very experienced testers because it can determine the successes or failure of the test effort.

Bug Reporting
Exactly what process will be used to manage the bugs needs to be planned so that each and every bug is tracked, from when it’s found to when it’s fixed – and never, ever forgotten.

Metrics and Statistics
Metrics and statistics are the means by which the progress and the success of the project, and the testing, are tracked. The test planning process should identify exactly what information will be gathered, what decisions will be made with them, and who will be responsible for collecting them.

Risks and Issues
A common and very useful part of test planning is to identify potential problem or risky areas of the project – ones that could have an impact on the test effort.

Reference/Source: Software Testing, Second Edition, By: Ron Patton

Read – Guidelines for effective test planning – Part 2

Classic Test Automation Mistakes


There are many good reasons to start automation. But sometimes – Companies start automation testing with enthusiasm but end up with manual testing only. Below are 5 mistakes that management might do while implementing automation:
1. “Test automation is the answer to testing problems”. Management says “We have so many tests and so little time, let’s use test automation. It will solve our problems.”
However it’s not true. Implementing/designing scripts by automation tool may take 3 times 100 times longer than the implementation of manual testing scripts. So if your project is at critical and final stage, so starting testing automation might not be a good idea.2. “Development of automation scripts (or automation testing process) cannot be started, as application is not stable yet. Developers are still in progress of developing the application.”
It is not true. Phases of test automation process and software development process are almost same. Automation itself is a development process. As soon as the development activities start, at same time planning of Automation testing should be started. Development of scripts can also be started earlier. You can get access to software (which is under-construction) from development team and start preparing automation scripts. You can get help from low level design documents.

3. “Now our testing team has automation tool. They will develop more tests efficiently.”
Automated tests development is a code development. Are you sure, your test team is trained in this? At least one programmer should be there in the team. Training must be given to the automation team, so that they design better tests.

4. Management (along with Test manager): “A sales executive demonstrated an automation tool to us and it seems that tool will add value to our testing. We think we should buy it.”
Vendor gives demonstration on which application? Is it yours or is it vendor’s application itself? Is tool support the same technology on which your application is developed? How much programming is required to develop tests? (Note – this is directly related to people skills and ROI). Consider Cost to Automate the Tests.

Involve Automation experts/Test architects while purchasing any automation tool.

5. “It has been 3 months and we didn’t see any progress so far.”
How many testers are involved in automation? Are they 100% allocated to automation?
Sometimes, only one automation expert is allocated to the project. Management thought that for automation fewer resources are required. However it is not always true. Also, the automation team should be 100% dedicated to automation project only. They should not be shared in other projects.
In the last, remember – A fool with a tool is always a fool. ~ by Anne Mette Jonassen Hass (Guide to AST)

References – LogiGear.com
Must Read – http://www.softwaretestingtimes.com/2010/04/selecting-software-testing-tool-testing.html


Note – As far as the title is concerned, Brian Marick (@testing.com) wrote a paper on “Classic Testing Mistakes”. So we put the title as “Classic Test Automation Mistakes” [just for fun 🙂]


QTP Automation Object Model Simplified | QTP AOM

We received couple of queries on QTP AOM. In this post we simplified the QTP AOM, so that a newbie in QTP can also understand what is Automation Object Model and how it works.
Automation is a technology through which we can access software objects inside one application from other applications. This can be done by VB scripting, VC++, Visual Basic, Javascript etc. So automation helps in managing the functionality of an application programmatically.


Object Model – Every software system consists of some objects & classes. System works because of these objects. And Object model is nothing but an object model is a structural representation of objects and classes. An object model defines a set of classes and interfaces, together with their properties, methods and events, and their relationships.


QTP Automation Object Model – Concluding from above two definitions – We can access QTP objects and control/managing its functionality by VB script. For example –

  • We can open QTP with the help of vb script.
  • We can run QTP tests without opening QTP manually. By vbscript, we can invoke QTP and define the testes which need to be executed.
  • We can set iterations for the execution of the test script and can also pass parameters.

All the above statements/conditions are defined in a vbscript. VBscript opens QTP and manages above functionality.


How this can be done? How VBscript controls QTP?
QTP provides an interface (object model), through which we can automate QTP itself by vbscript. By this we can automate tasks like – Run and configure QTP, Run scripts etc.
See below examples –  Run QTP – Copy the below code and pate in notepad, same file as “ABC.vbs”.
Dim qtApp
Set qtApp = CreateObject(“QuickTest.Application”) ‘ Create the application object
qtApp.Launch ‘Start QuickTest
qtApp.Visible = True ‘ Make it visible
Now Run the “ABC.vbs”. It will open QTP.


Simple way to create script for automating QTP:
QTP also provides the option to generate the automation script (which controls the automation and functionalities of QTP). For this, first you need to open QTP manually and define all the appropriate QTP settings like – test settings, results settings, Options, Action settings, iteration settings etc.
Now go to Test Settings -> In General tab and Object Identification tab, there is “Generate Script” button. Clicking on the button will generate a vbs file with the defined settings.

In Next post we will discuss when to use these QTP automation scripts.

Rate Yourself in Software Testing | Skills of a Professional Tester | Lets become Test Professional from Test Dummies.

Today employers want test professionals – not test dummies. There are unique skills of testers that are highly desired by employers to make the testing process more efficient and effective. See the checklist for a cross-section of the desirable skills of a test professional
✚ Know the best places to find bugs and defects and most effective methods for uncovering them
✚ Know how to prioritise testing based on risk exposure
✚ Develop test strategies that balance testing needs against management budget and commitment
✚ Apply testing methodologies, processes and analysis techniques
✚ Possess basic technical skills to aid testing (e.g.scripting in VB/Perl/batch files), fundamentals of SQL, MS Office tools for data collation, and report
generation
✚ Understand test automation tools and frameworks, including the advantages, disadvantages and adoption process
✚ Know inspection and review techniques
✚ Understand non-functional test issues, such as load and performance, availability, failover/recovery, compatibility, usability and security
✚ Manage and coordinate testing projects
✚ Build a test team and foster an effective test team culture
✚ Collect metrics and measurements to assess effectiveness of testing and quality of the product, and reporting mechanisms to inform management
of outcomes
✚ Know approaches to improving test processes.

Not everyone in the team needs to have all these skills – in fact, it would be quite uncommon to be experienced in all areas. However, there should be a strategy to ensure that a team can be assembled with a diverse cross-section of these skills. If you are an employer, consider including these skills in the tester’s position description. If your staff are lacking in some of these areas, ask yourself what it is costing your projects, and plan a strategy for increasing the skills of your existing team and/or seeking additional staff. If you are a tester, do a quick check to see whether you have these skills, as well as experience in these areas. If you are missing some, develop a strategy to increase your skills base – you will reap the rewards.


Other Comments: It is possible that Certified Software Testing Professionals with unique skills in test management, test automation, and load/performance testing, may increase their prospects of employment and income. Certification is required, reason is – “Clients demands that they need only certified people in their project” or “‘Companies convince their clients by saying that we have certified software test professionals, we deliver a high quality product.”

For further information on CSTP certification Contact “Dr Kelvin Ross” who is the coordinator for the CSTP program. E-mail: kelvinr@kjross.com.au
Source – K. J. Ross & Associates provides specialist services in software verification and validation, including test management, risk assessment and test strategy, test process
improvement, test outsourcing, automated testing, load and performance testing, independent test laboratory, and training and mentoring.
Visit: www.kjross.com.au

Effective Practical Test Reporting Tips & Techniques | Software Testing Summary Reports Generation

Effective Practical Test Reporting Tips & Techniques | Software Testing Summary Reports Generation




status-report
Testing is very important phase in the Life Cycle of Software Development so whatever the work testers and test lead are doing in this Phase it should be properly documented and testers should be able to provide the needful information in proper Reports. Here are some important documents, their frequency and the reason for those documents have a look at it:

Test Documentation Frequency Reason
High Level Test Plan Each project A test plan is produced for each project. It defines what functions will be tested and what functions will not be tested. This document will detail any risks and contingencies as well as stating assumptions and defining required resources.
Test Specification Each stage This document details the test conditions and test cases for each stage of testing. This will be used by the testers in the running of their tests.
Test Logs Each stage Test Logs are to be produced by each tester – this will enable progress to be monitored and controlled. It will also provide a suitable test audit at the end of the project.
Test Progress Report Weekly This report will provide managers with a weekly progress on the testing being carried out. ‘S’ curves to show test progress will be used as well as faults found/fixed.
Test Summary Report When required This summary report will be produced when requested by Senior Management at any stage of the development lifecycle. This will be a condensed version of the Test Progress Report specifically aimed at Senior Management.
Post Project Report Each project At the end of each project a ‘Post Project Review’ will be carried. This activity will contain an analysis of what went well and what didn’t go well in terms of the project. A report will be produced detailing the changes for process improvement.

Some or all of the following charts should be used to monitor progress. It is recommended that the graphs be displayed clearly on a wall so everyone is made aware of the current situation. Charts, rather than tables, are also recommended since they are easier to read and are more likely to attract attention and be taken notice of:

Faults Found:
Fault Found in each day of testing
This chart simply logs the number of faults found during each day of the test schedule. The severity of faults could be indicated by using different columns on the chart.
Faults Found vs. Faults Fixed:

Fault Found vs Faults fixedMonitoring the number of faults found together with the number of faults fixed is a useful way to spot potential scheduling problems early. In the example above the increasing gap between the number of faults found and the number fixed suggests that more development effort is needed to fix more of the outstanding faults. Leaving these until the last moment invariably ends in disaster (delayed release, poor quality system, or both).
Tests Run and Tests Passed vs. Tests Planned:

Tests Run and tests Passed vs Planned
Since we will know at the start of a test effort how many tests we are intending to run (defined by the scope) it is possible to plan the number of tests that should be completed each day of the test effort. This will normally look like an S-Curve, as shown on the example graph above (this is a third-order polynomial). Plotting both the number of tests actually run and the number of tests that have passed will quickly highlight any of a number of problems. For example, if the number of tests run falls below the number planned then either more testers or more time will be needed. If the number of tests passed falls much below the number run (i.e. if a large number of tests fail) then more faults than expected are being found and this will impact both the test team and the developers.
Faults Per Owner:

Faults per Owner
It is important to know how many faults are being worked on at any one time and by whom. This chart will show whether there is a balance of faults being assigned to the team. Each “column” on the bar chart can show priority and/or severity for further analysis.
Number of Defects for each Cycle:
This Graph shows the number of bugs (New bugs, Assigned Bugs, Open Bugs, Fixed Bugs, Reviewed-not-ok Bugs, Closed Bugs, Deferred Bugs) in each cycle for every release. The status of the bugs may be different from company to company. So based on the company the testers have to change the Status in Graph.
 Testing Cycle vs Defect Status Number of different severity bugs in each Cycle:
This Graph shows the number of bugs of different severity (Critical bugs, Major Bugs, Minor Bugs, Cosmetic Bugs) in each cycle for every release. This Severity also each and every organization maintains their own standards so it may be different from company to company. Based on their own standards they have to change the Graph.
Test Cycles Vs Severity

Number of different severity bugs in each Test Level:
This Graph shows the number of bugs of different severity (Critical bugs, Major Bugs, Minor Bugs, Cosmetic Bugs) in test level (Unit Testing, Integration Testing, System Testing and User Acceptance Testing) for every release. This Severity also each and every organization maintains their own standards so it may be different from company to company. Based on their own standards they have to change the Graph.

Type of testing Vs Bug Severity
 Number of different severity bugs in each Version:

This Graph shows the number of bugs of different severity (Critical bugs, Major Bugs, Minor Bugs, Cosmetic Bugs) in each Version. This Severity also each and every organization maintains their own standards so it may be different from company to company. Based on their own standards they have to change the Graph.
Version vs Bug Severity

End of the day reports for Testing Engineer:

For each individual Tester he can prepare a separate report for each and every task
End of the day reports for Testing Engineer.doc

Reports generated by Test Lead


For Test Lead he is having so many tasks like Client Meeting, Conference Calls, Test Strategy, Test Plan preparation, Knowledge Transfer, Organization internal meetings and Meeting with Testers.
But here are some important reports:
Reports generated by Test Lead.doc
 ————————————————————————————————————
Author – unknown. [Posted from our old forum and old blog. If you know the author, or if you are the author please contact us, we will update the  post]