5. Populating the template: development of parameterized tests

This is probably the most important step when the source code of the test is written (in T2C format).

Overall structure of a file in T2C format is shown below.

Overall structure of a file in T2C format

Typically, a file in T2C format consists of the following parts:

The source code in <GLOBAL>, <STARTUP> and <CLEANUP> sections is written in the target programming language. It will be inserted verbatim in the appropriate places of the resulting test source file by T2C file generator.

The typical structure of a single <BLOCK> section is as follows.

The typical structure of a single block in a .t2c file

Each <BLOCK> defines a parameterized test scenario. If different scenarios are nesessary to check the function(s) under test (the targets of this group of tests), several <BLOCK> sections can be provided for this set of test targets. Generally, it is not recommended to implement significantly different test scenarios in a single <BLOCK>.

The logic of the group of tests to be created from this <BLOCK> is provided mostly in <CODE> and <FINALLY> sections. These sections are written in the target programming language with one exception: special placeholders can be used in them to refer to the test parameters. For example, T2C file generator will use the value of the parameter #4 in the resulting test source instead of <%4%> placeholder (note that the numbers of parameters start from 0).

However, the placeholders like <%4%> are rarely used directly in <CODE> or <FINALLY> sections. Usually, a symbolic name should be provided for each parameter in the <DEFINE> section (like INDEX and LENGTH instead of <%0%> and <%1%> on the figure above) and then the symbolic names are used in the source of a parameterized test. It often makes the test scenario clearer if the names of the parameters are chosen properly.

For each requirement to be checked in this test scenario, REQ or REQva macro should be called. These macros are provided by T2C API and are used to check requirements and report failures. Among their arguments are the ID of the requirement being checked and the expression that evaluates to 0 if and only if the requirement is violated. If the expression evaluates to 0, the macros output appropriate message about the failure to the test execution log (journal) probably along with the textual representation of the requirement. The latter is loaded from the catalogue of requirements (see Section 3, “Creating catalogue of requirements”).

In case of C or C++ programming language, T2C file generator will usually create a single test function for each set of parameter values defined by <VALUES> and <PURPOSE> sections of the <BLOCK>. The core part of that function will be prepared by substituting the placeholders in <CODE> and <FINALLY> sections with appropriate values. The rest is defined by the templates T2C file generator uses to create the resulting files. The blocks of source code needed to support a particular target testing framework are usually also encapsulated in these templates.

An example of how the actual test functions are created by T2C from the parameterized test scenarios is shown below. Note that in real file generation targets, the resulting test functions are usually defined in a more complex way and contain additional instructions, for instance, to support a particular testing framework, etc.

Param

The values of the parameters of a test scenario should be chosen carefully. Ideally, they should allow to check the system under test in each situation where it is necessary and reasonable.

Among the parameters of a test are often the arguments of the function(s) checked there. Sometimes its expected return value or the expected error code is also a parameter of the test. In more complex cases, even the parts of the test scenario can be made its parameters, for instance, the instructions to properly prepare the data for the function under test, etc. Anything that can be a part of the source code of a test can be made its parameter.

Note

If there are many sets of parameter values provided in a .t2c file, the generated sources of the tests can be quite large in size. It is because of the way T2C operates: for each set of parameter values a test function is generated based on the corresponding parameterized test scenario and placed in the resulting file. It may take a while for a compiler to process such a big file. Some compilers may even refuse to process it.

So, it should be a rule of thumb, to choose the parameters and their values reasonably. Choose them so as to check the system under test in the necessary conditions and at the same time try to keep the number of sets of parameter values not very large. If in fact it starts getting large, you can try to refactor the tests and revisit their parameters to get away with the smaller number of the sets of values.