Article for newbie who want to know what is agile, how agile works, how agile testing methodology is different from traditional  methodologies and what are the challenges in agile.

 
Today’s trend in Software Development is racing towards achieving Targets, Quality and Customer Satisfaction within a limited time frame. This is mostly because the Business Scenario in today’s world is different from what it used to be, a few years ago. Most of the Product Development companies are now adopting a new age concept called “Agile Methodology” for the Software Development Life Cycle.
Software Testing has been a prime focus ever since the IT Industry has realized the importance of Quality of Deliverables, no matter what methodology we follow with the Software Development. However, since the Software Testing is a part of the Business Model, the Testing process also needs to change accordingly. Needless to say, we have so called “Agile Testing” as a result of such a Business Model.

Introduction to Agile Testing
While the given application under test is still evolving depending upon the customer needs, the mindset of the end user and the current market condition, it is highly impractical to go for the usual standard SDLC Models like Water Fall, V&V Model etc. Such models are most suitable for the Applications that are stable and non-volatile. The concept of “Time-To-Market” is the key word in today’s IT Business that compels the Software vendors to come up with new strategies to save the time, resources, cut down the cost involved and at the same time, deliver a reliable product that meets the user requirements. In this case, a reasonably good amount of end-to-end testing is carried out and the product could be acceptable with known issues/defects at the end of an intermediate release. These defects are harmless for the Application usability.
To adopt such a process in a systematic way, we have a new concept called Agile Methodology. This methodology continuously strives to overcome the issues of dynamically changing requirements while still trying to maintain a well-defined process.

The process is as follows:
1. The Customer prepares the Business Requirements and the Business Analyst or the Engineering team reviews it. Ideally, the Quality Assurance/Testing team is also involved in reviewing these requirements in order to be able to plan further stages accordingly.

2. During the Design and Implementation stages, the Engineering team writes User Stories and the analysis of issues at various stages. The Customer reviews these on regular basis and updates the Requirement specifications accordingly. The Testing team would follow up on regular basis at every stage until a consolidated documentation is prepared. This is to ensure that the Customer, the Engineering team and the Testing team are at the same page always and thus ensuring complete test coverage.

3. While the Engineering team starts the implementation, the Testing team starts with test planning, test strategies and test cases preparation. These would be properly documented and handed over to the Customer and the Engineering team for review. This is to ensure the complete test coverage and avoid unnecessary or redundant test cases.

4. As and when the Developer implements the code, the Testing team identifies if the application can be built using this code for a quick testing. This is to identify the defects at the early stage so that the developer can fix them in the next round on priority basis and continue with further development. This iteration continues until the end of the code implementation. Once the testing cycle starts, the Test team can now focus more on major test items such as Integration, Usability Testing and System Testing etc.

Scope of testing an application
The Testing team knows the complexity involved and it is accepted by the customer that the software development and/or the enhancements and hence the testing is a continuous process. Testing of the application at a black box level would suffice in order to identify the issues and raise the defects by the Testing team. The application continues to evolve until it reaches the stage of final acceptance. Hence the scope of testing would continue to evolve as per the Customer needs.

Process followed at various stages in the product life cycle
Every intermediate release of the product would be divided into two short cycles, usually of the duration of 40 days each. Each cycle would be executed in the following stages. The roles and responsibilities of every individual and the team are clearly defined for each stage.

– Design Specifications: The Testing team’s efforts would focus on performing any tool or process improvements and reviewing, understanding, and contributing to the nascent specifications.
– Implementation: While the Engineering/Development team is implementing the code, Testers would develop complete Testing Plan and Test Sets (set of test cases) for each of the features included in the cycle. Engineering features must be included; they would likely require some level of collaboration with the engineering feature developer. All Test Sets should be ready to execute by the end of implementation period of the respective cycle. After Test Set preparation, calculate the time estimation and prioritization for the Test Set execution based on the complexity and expected execution time for each test suite.
– While the test execution time estimation is notoriously difficult, this number should provide the Customer with a starting point for benchmarking.
– Testing/QA: Test Set execution, raising defects and follow up with the Engineering Team. End-to-end validation of the defects. Focus simultaneously on improving the quality of test cases. Watching out for and adding new cases as testing proceeds. Testing the software end-to-end to discover regressions and subtle systemic issues. Learning to focus more on using the time available to uncover the largest number of and most important bugs. Any deviation from the estimated time should be communicated across well in advance, so that the schedule can be worked upon depending upon the priority of the pending tasks. If there are certain issues or test cases blocking due to unknown errors, they would be differed until the beginning of next Testing/QA Cycle.
– Before acceptance: Follow up on ad-hoc requests/ changes in requirements on a regular basis, besides trying to complete the defined tasks.

Looking at various broad areas of testing of a complex application, the system structure, and the depth of functionality implemented and the level of complexity, the complete end-to-end Test Execution within a limited time frame would be next to impossible with any standard SDLC Model available. The Product/application involves a good deal of learning of how and when to use the application and requires that the end user know the functionality of various modules prior to constructing a User defined model (UDM) for his Business purpose or even for testing.

Reasons for Agile Testing methodology to test an Application

The testing wouldn’t be 100% complete in this case before the finished product reaches the hands of the end user. This is true especially when the target audience and its system structure are unknown. Different users have their own set of ideas and unique problems. Given this fact, it is hard to say that it is 100% bug free software when it reaches the customer. Hence, taking into account the constraints involved in using the standard SDLC Models, it is worth adopting an approach such as Agile Testing, which is more suitable for the dynamically changing requirements.

Context Driven Testing and Rapid Testing:
This type of testing is what usually the Agile Testers incorporate. Context Driven testing is the one where the test scenarios are not known before hand. It mostly comes from the context in which the application is being executed. In our case, it is constructing the UDMs based on a given use case from the targeted end user and test it for the scenarios that are explained by his system configuration. It also includes constructing the UDMs based on any trouble-shooting discussion arising out of defects pointed out during the Customer acceptance and testing.

Rapid Testing is a process of defining our own strategies to test the Software, not necessarily following any specific Process or a Model. It focuses mainly on identifying the defects as quickly as possible rather than focusing on the end user requirements.

Heuristic approach is used in such cases. The tester uses his common sense and previous work experience to test the application at various levels in order to figure out where the application stands.

References: http://www.testing.com/agile

Online Documentation for Agile Testing and Rapid Testing:
Agile Testing Challenges
Rapid Software Testing

Error Guessing
Why can one Tester find more errors than another Tester in the same piece of software?

More often than not this is down to a technique called ‘Error Guessing’. To be successful at Error Guessing, a certain level of knowledge and experience is required. A Tester can then make an educated guess at where potential problems may arise. This could be based on the Testers experience with a previous iteration of the software, or just a level of knowledge in that area of technology. This test case design technique can be very effective at pin-pointing potential problem areas in software. It is often be used by creating a list of potential problem areas/scenarios, then producing a set of test cases from it. This approach can often find errors that would otherwise be missed by a more structured testing approach.

An example of how to use the ‘Error Guessing’ method would be to imagine you had a software program that accepted a ten digit customer code. The software was
designed to only accept numerical data.

Here are some example test case ideas that could be considered as Error Guessing:
1. Input of a blank entry
2. Input of greater than ten digits
3. Input of mixture of numbers and letters
4. Input of identical customer codes

What we are effectively trying to do when designing Error Guessing test cases, is to think about what could have been missed during the software design.

This testing approach should only be used to compliment an existing formal test method, and should not be used on its own, as it cannot be considered a complete form of testing software.

Exploratory Testing
This type of testing is normally governed by time. It consists of using tests based on a test chapter that contains test objectives. It is most effective when there are little or no specifications available. It should only really be used to assist with, or compliment a more formal approach. It can basically ensure that major functionality is working as expected without fully testing it.

Ad-hoc Testing

This type of testing is considered to be the most informal, and by many it is considered to be the least effective. Ad-hoc testing is simply making up the tests as you go along. Often, it is used when there is only a very small amount of time to test something. A common mistake to make with Ad-hoc testing is not documenting the tests performed and the test results. Even if this information is included, more often than not additional information is not logged such as, software versions, dates, test environment details etc.
Ad-hoc testing should only be used as a last resort, but if careful consideration is given to its usage then it can prove to be beneficial. If you have a very small window in which to test something, then the following are points to consider:

1. Take some time to think about what you want to achieve
2. Prioritize functional areas to test if under a strict amount of testing time
3. Allocate time to each functional area when you want to test the whole item
4. Log as much detail as possible about the item under test and its environment
5. Log as much as possible about the tests and the results

Random Testing

A Tester normally selects test input data from what is termed an ‘input domain’ in a structured manner. Random Testing is simply when the Tester selects data from the input domain ‘randomly’. In order for random testing to be effective, there are some important open questions to be considered:
1. Is random data sufficient to prove the module meets its specification when tested?
2. Should random data only come from within the ‘input domain’?
3. How many values should be tested?

As you can tell, there is little structure involved in ‘Random Testing’. In order to avoid dealing with the above questions, a more structured Black-box Test Design could be implemented instead. However, using a random approach could save valuable time and resources if used in the right circumstances. There has been much debate over the effectiveness of using random testing techniques over some of the more structured techniques. Most experts agree that using random test data provides little chance of producing an effective test. There are many tools available today that are capable of selecting random test data from a specified data value range. This approach is especially useful when it comes to tests associated at the system level. You often find in the real world that ‘Random Testing’ is used in association with other structured techniques to provide a compromise between targeted testing and testing everything.