If a fragment of the documentation contains a requirement for many functions, it can be difficult to specify all the respective IDs in the list correctly. For instance, the following list of identifiers could be used for a fragment of the description of printf():
“asprintf.13.02; fprintf.13.02; printf.13.02; snprintf.13.02; sprintf.13.02; vasprintf.13.02; vdprintf.13.02; vfprintf.13.02; vprintf.13.02; vsnprintf.13.02; vsprintf.13.02”
Imagine you need to write such a list for each requirement trying not to forget to change the indexes appropriately. It can be a really hard time.
There is a way to handle this. We can create a so called define-block; (or substitution block): denote the whole list of functions with a single identifier, say, @print_funcs. It begins with '@' to distinguish it from function names (this is not mandatory though). Its definition should be placed in the HTML file we are marking up and it looks like this:
<a class="define" name="@print_funcs"
title="asprintf; sprintf; fprintf; printf; snprintf; vasprintf;
vdprintf; vfprintf; vprintf; vsnprintf; vsprintf"></a>
Now instead of that long list we can write just this: “@print_funcs.13.02”.
If a define-block is placed in the HTML file manually, it is recommended that it is done at the beginning of the body of this HTML. However, ReqMarkup provides probably more convenient means to create, remove and modify such blocks (see the example in the case study). Using ReqMarkup to manage define-blocks has another advantage: creation, removal and modification of define-blocks can be undone and redone like almost any other editing operation.
The prefixes (“app” and “ext”) work properly with the subsitutions. That is, the followng lists of identifiers are allowed:
A substitution identifier may contain letters, digits, '_' and '@'. Other symbols are not allowed. The identifier is case sensitive.
The substitution ID may be an element of another ID list like any ordinary ID. Moreover, it is possible to use substitutions in a definition of another substitution.
Example:
<a class="define" name="ALL_PRINT_FUNCS"
title="f_PRINT_FUNCS; @_PRINT_FUNCS; s_PRINT_FUNCS"></a>
f_PRINT_FUNCS, @_PRINT_FUNCS, s_PRINT_FUNCS are defined somewhere else in this HTML file (no matter before or later).
Please make sure you do not create recursion here.
Sometimes it can be convenient not to use a define-block but rather specify the substitution directly. Let us consider the following list of IDs: “do_get_long.01; do_get_ulong.01; do_get_uint.01”.
We could create a define-block named, say, @VALUE_TYPE which value is “long; ulong; uint” and then use do_get_@VALUE_TYPE.01 instead of the list of IDs. But it could be clearer if we used do_get_[long; ulong; uint].01 instead. Fortunately, unnamed substitutions like the one in the latter case are allowed by ReqMarkup. The list of values to be placed in the ID is specified in square brackets, the values being separated by semicolons. Spaces right before and after a semicolon are ignored.
When testing a library that uses GLib Object System (GObject), sometimes it can be convenient not to link the requirements for properties and signals of GObject-style objects to the functions which behaviour they affect. In fact, the problem is that it is often difficult to determine which functions are affected by the requirements for a given property or a signal.
For simplicity, we can assign IDs for these requirements using almost the same rules as for the functions. Name of the property or the signal should be used in the ID instead of the function name and dashes ('-') should be replaced with underscores ('_').
It is recommended to prefix the name of the property (or signal) with a name of the class this property (or signal) belongs to. This can help to avoid ID conflicts if there are properties or signals with the same names in different classes. The name of the class should be written in lowercase with underscores separating the words.
Examples:
Typical directory structure of a test suite prepared using “c_standalone” T2C target is shown below.
<some_name>-t2c/ // The root directory of the test suite | +-- <some_name>.cfg // Parameters of the test suite +-- include/ // Common include files for the tests +-- reqs/ // Catalogues of requirements | | | +-- <group1>.xml | +-- <group2>.xml | +-- <group3>.xml | | | ... +-- src/ // This directory contains .t2c files. | | // Apart from that, its layout can be arbitrary. | +-- <group1>.t2c | +-- <group2>.t2c | | | +-- <s3>/ // This directory contains a group of .t2c files | | | | | +-- <group31>.t2c | | +-- <group32>.t2c | | | +-- <s4>/ | | | | ... ... | +-- tests/ // This directory will be created by T2C file generator | | // Generated files will be placed here | +-- Makefile // Makefile to build all the tests | +-- <group1>/ | | | | | +-- <group1>.c | | +-- Makefile // Makefile for this particular test | | | +-- <group2>/ | | | | | +-- <group2>.c | | +-- Makefile // Makefile for this particular test | | | ... +-- testdata/ // Data used by the tests should be stored here | | | +-- testdata_src/ // Contains a makefile and source code | | // of the test data that needs to be built (modules, etc.) | +-- <group1>/ | | | | | ... | +-- <group2>/ | | | | | ... | ... ...
Common parameters of the tests can be specified in a .cfg file located in the root directory of the suite. The file should have the same name as this directory but without '–t2c' suffix (if it was present). For example, the configuration parameters for example01-t2c test suite should be specified in example01-t2c/example01.cfg. A .cfg file should be present in each test suite prepared with T2C even if it is left empty.
Almost any parameter that occurs in T2C template files can be given a value via the .cfg file. T2C itself knows nothing about the configuration of the test suite, it just substitutes appropriate placeholders in the templates with the values given in the .cfg file.
An example of a .cfg file defining some of the commonly used parameters is shown below. The lines beginning with '#' are comments. A short description of each parameter is provided in the comments above it.
# Custom compiler options
COMPILER_FLAGS=`pkg-config --cflags glib-2.0`
# Custom linker options
LINKER_FLAGS=`pkg-config --libs glib-2.0`
# Maximum time in seconds each test is allowed to run
WAIT_TIME=60
# Copyright holder
#
# [NB] This is an example of how a long single-line value can be written
# in several lines using '\'.
COPYRIGHT_HOLDER = Institute for System Programming \
of the Russian Academy of Sciences (ISPRAS)
# A boilerplate notice to appear at the top of each test source file
# generated by T2C for this test suite
#
# [NB] This is an example of a multi-line value of a parameter.
# The lines between "=>>" and "<<" will be inserted to appropriate place
# as they are, without any changes.
LICENSE =>>
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
<<
It is sometimes necessary to use external data to check some of the requirements. For example, the test may need to load data from a file.
It is recommended to store the data necessary for the tests in the subdirectories of the testdata directory for the test suite. These subdirectories should have the same names as the corresponding .t2c files. So the tests from a .t2c file will look for their data in a separate directory.
To obtain the path to these data from the test code, we can use T2C_GET_DATA_PATH(rel_path) macro.
Suppose we need to get the path to the file myfile.txt from the test which sources are in glib_arrays.t2c. In “c_standalone” target, T2C_GET_DATA_PATH("myfile.txt") will return the absolute path to testdata/glib_arrays/myfile.txt.
The pointer returned by T2C_GET_DATA_PATH() should be freed when it is no longer needed.
Suppose we need to write some header files for the tests we develop. Sometimes it could be convenient to place these headers to <root_dir_of_test_suite>/include/ directory or its subdirectories.
The templates for “c_standalone” T2C target are organized so that the path to <root_dir_of_test_suite>/include/ will be specified automatically in -I compiler option in the generated makefiles for the tests.
The key functionality of version 2.x is different from 1.x, hence the differences in how T2C file generator operates, etc. However T2C 2.x is mostly backward compatible with T2C 1.x with respect to the format of .t2c files and T2C API.
The most important differences are listed below.
In T2C 2.x, the file generator (T2C command line tool) processes one .t2c file at at time rather than a whole bunch of them at once. Such behaviour is more convenient, for example, for using T2C in makefiles or with other build systems.
Almost no restriction is imposed on the directory structure of the test suite: the structure is now defined by the templates used rather than by T2C itself. As the templates can be added and overridden by the user, the directory structure of the test suite as well as the set of files to generate can be rather arbitrary.
The set of files to be generated is no longer limited to C/C++ sources and makefiles. Again, the templates, not T2C, dictate which files to generate and where to place them. Suite-specific files can also be created. To do this, it is enough to provide custom templates for these additional files with the suite - and the files will be generated by T2C along with the default ones (created from the default or overridden templates).
Not just templates of makefiles, but rather any template can now be overridden for a test suite or even for a particular .t2c file.
Different file generation targets (or “profiles”) are now supported.
It is no longer necessary to place each T2C tag in a separate line. That is the following is acceptable in a .t2c file:
<BLOCK><TARGETS>foo</TARGETS><CODE>printf("Hello World!\n");</CODE></BLOCK>
Still abusing this feature can make .t2c files less clear and readable. It it better to lay out the contents of that file like this:
<BLOCK>
<TARGETS>foo</TARGETS>
<CODE>
printf("Hello World!\n");
</CODE>
</BLOCK>
…or like this:
<BLOCK>
<TARGETS>
foo
</TARGETS>
<CODE>
printf("Hello World!\n");
</CODE>
</BLOCK>
T2C tags may occupy more than one line in the file. Apart from the code fragments in the target programming language, you can insert a newline in any place where a whitespace is acceptable. The goal here is, again, to make the .t2c file clearer. For example, consider the following opening tag of a <BLOCK> section containing attributes (the support for attributes of the BLOCKs is described here):
<BLOCK name = "My Super Test" depth = "medium" min_lib_ver="2.0.1" max_lib_ver="3.9.99">
It could be easier to read and understand if written like this:
<BLOCK name = "My Super Test"
depth = "medium"
min_lib_ver="2.0.1"
max_lib_ver="3.9.99"
>
T2C 2.x is not so permissive with respect to the violations of .t2c file format as T2C 1.x which silently ignores many kinds of them. A stricter error-checking policy is now employed. So, even if you do not use any other features of T2C 2.x for some reason, you may still benefit from this enhanced syntax checking of .t2c files.
It is recommended to use “##” instead of a single “#” character to begin T2C-style comments. This helps avoid confusion with directives used by T2C or a preprocessor for the target language. Still, for compatibility reasons, #-comments are supported in T2C 2.x and T2C tries to do its best to detect whether a string is a directive or not if it begins with of a single “#”.
A T2C-style comment should now occupy the whole line. Particularly, comments at the same line as a value in a <PURPOSE> section are no longer allowed.
A <BLOCK> section can now have user-defined attributes (see the details here). Special attributes like “parentControlFunction” supported by T2C 1.x are no longer treated separately. They were almost never used with T2C 1.x anyway. If an attribute like this is needed, it can be specified for a <BLOCK> like any other attribute and the templates (“test_case” group) can be adjusted to use it.
#t2c_basename T2C directive is no longer supported. When overriding the templates of makefiles was not supported by T2C 1.x, this directive was used to implement a temporary solution to this problem. Since then, this directive was deprecated even in T2C 1.x.
Both <VALUES> and <PURPOSE> sections can now be used in a <BLOCK> to specify the values of test parameters. The meaning and the syntax are the same. <PURPOSE> sections are deprecated. It is recommended to use <VALUES> in all new tests developed with T2C 2.x.
The lines containing only whitespace characters or no characters at all are now ignored in <PURPOSE> (or <VALUES>) sections.
You can specify a long value in more than one line in <PURPOSE> (or <VALUES>) section breaking the line using '\' character, for example,
<PURPOSE>
SET(TRUE; \
FALSE;\
DONT_KNOW)
call_func1(); call_func2(); \
call_func3();
0
</PURPOSE>
The fragment above is equivalent to the following one:
<PURPOSE>
SET(TRUE; FALSE; DONT_KNOW)
call_func1(); call_func2(); call_func3();
0
</PURPOSE>
The rules for escaping special characters with '\' character in SET and RES constructs have changed. In short, now escaping is necessary if otherwise it would be ambiguous how to interpret the construct. See Section 2.7.7.4, “Escaping special characters” for details.
ABORT_TEST macro is now available which has the same meaning as ABORT_TEST_PURPOSE (just a shorter name). A number of other ABORT_xxx macros are also available that allows to set test result to UNSUPPORTED, FIP, etc., if these test verdicts are supported in the particular T2C target.
In addition to REQ, REQva macro is available. The syntax of REQva is as follows:
REQva(<list_of_IDs>, <expression>, <format_string>, ...);
That is, unlike REQ, REQva allows formatted message output in case of a failure. The syntax of the format strings is the same as for printf() function.
Any parameter specified in a .cfg file for the test suite (or even for a particular .t2c file) can be used in the templates of the files to be generated. That is, if you add the following to a .cfg file of the test suite
MYPARAM = some value
you can then use <$MYPARAM$> in the templates of the test sources, makefiles and other files to refer to MYPARAM parameter. When processing the template, T2C file generator will substitute the value of this parameter (some value) in place of <$MYPARAM$>.
You can now break long lines in a .cfg file using '\' character like it can be done in C/C++ sources, Makefiles, etc.
Example:
ADD_SOURCES = myfile.c \
yourfile.cxx \
other.h
The fragment above is equivalent to the following:
ADD_SOURCES = myfile.c yourfile.cxx other.h
You can now specify multiline values of the parameters - see LICENSE parameter in the example of .cfg file of the test suite.