Each API is supposed to behave the way it is coded, i.e. it is functionality specific. These APIs may offer different results for different type of the input provided. The errors or the exceptions returned may also vary. However once integrated within a product, the common functionality covers a very minimal code path of the
API and the functionality testing / integration testing may cover only those paths. By considering each API as a black box, a generalized approach of testing can be applied. But, there may be some paths which are not tested and lead to bugs in the application. Applications can be viewed and treated as APIs from a testing perspective.
There are some distinctive attributes that make testing of APIs slightly different from testing other common software interfaces like GUI testing:
Testing APIs requires a thorough knowledge of its inner workings – Some APIs may interact with the OS kernel, other APIs, with other software to offer their functionality. Thus an understanding of the inner workings of the interface would help in analyzing the call sequences and detecting the failures caused.
Adequate programming skills – API tests are generally in the form of sequences of calls, namely, programs. Each tester must possess expertise in the programming language(s) that are targeted by the API. This would help the tester to review and scrutinize the interface under test when the source code is available.
Lack of Domain knowledge – Since the testers may not be well trained in using the API, a lot of time might be spent in exploring the interfaces and their usage. This problem can be solved to an extent by involving the testers from the initial stage of development. This would help the testers to have some understanding on the interface and avoid exploring while testing.
No documentation – Experience has shown that it is hard to create precise and readable documentation. The APIs developed will hardly have any proper documentation available. Without the documentation, it is difficult for the test designer to understand the purpose of calls, the parameter types and possible valid/invalid values, their return values, the calls it makes to other functions, and usage scenarios. Hence having proper documentation would help test designer design the tests faster.
Access to source code – The availability of the source code would help tester to understand and analyze the implementation mechanism used; and can identify the loops or vulnerabilities that may cause errors. Thus if the source code is not available then the tester does not have a chance to find anomalies that may exist in the code.
Time constraints – Thorough testing of APIs is time consuming, requires a learning overhead and resources to develop tools and design tests. Keeping up with deadlines and ship dates may become a nightmare.
Testing of API calls can be done in isolation or in Sequence to vary the order in which the functionality is exercised and to make the API produce useful results from these tests. Designing tests is essentially designing sequences of API calls that have a potential of satisfying the test objectives. This in turn boils down to designing each call with specific parameters and to building a mechanism for handling and evaluating return values.
Thus designing of the test cases can depend on some of the general questions like
- Which value should a parameter take?
- What values together make sense?
- What combination of parameters will make APIs work in a desired manner?
- What combination will cause a failure, a bad return value, or an anomaly in the operating environment?
- Which sequences are the best candidates for selection? etc.
Read Part 2: API Testing – Points to take care
Recent Comments