Go to the first, previous, next, last section, table of contents.

Test Scripts

The term test script is commonly used for a document that describes how to perform a test of something. In the context of tg the word has a more specific meaning: it means a complete, machine-processable description of a test driver program which performs the test. In such a script only the essentials of the driver have to be specified. The tg command is nothing but a compiler for test scripts; it translates them into correct Ada source code.

Writing test scripts instead of coding the driver by hand not only saves a lot of work (the generated program is typically ten times larger than the script), but also helps to construct tests in a uniform, standardized way.

There are two main sections in a test script: the global section and the test case section. As you can tell from the words, parameters for the whole test driver are set in the global section, while the individual test cases form the test case section.

Test Script Basics

Test Scripts should have filenames ending with `.ts'.

A test script mainly contains chunks of Ada code, marked by keywords which tell tg at what place in the driver it should put that code. The basic idea of the format is to have the keywords begin in column 1, followed by the corresponding chunk of code, which can stretch over an arbitrary number of lines, all but the first of which must be indented. The first line which begins with a non-blank character marks the end of the chunk. (Empty lines, or lines containing only whitespace, do not end a chunk.) Example:

prepare Result := 0;
        Done   := False;

        if not Initialized then
          Initialize;
        end if;
test ...

The chunk of code which should become the `prepare part' of a test case begins with `Result :=...' and ends with `end if;' in the sixth line, then comes the next section, the `test part'. (More on the meaning of these `parts' later.)

tg is not case-sensitive, just like the Ada language itself. You may write the tg keywords in upper, lower, or mixed case, just as your taste and coding conventions suggest. Comments take the usual Ada form (-- ...). Note that comments which apply to the script should begin in column 1, otherwise tg might consider them part of a code chunk and copy them into the driver program (which could be confusing if someone ever examines the driver code).

The Global Section

The possible subparts of the global section are fail_handling, error_handling, context, exceptions, and define. They are explained subsequently, in that order. (Note that also in the script file the order of these parts may not be different. This might change in future versions of tg.) Apart from the context subpart, all of these are optional.

Remember also that all the keywords which start a subpart must begin in column 1.

fail_handling ( stop | continue ) Specify what the driver should do if a test case fails. The alternatives are to stop execution after the first failing test case, or to continue execution regardless of the test case results. Default is continue.
error_handling ( stop | continue ) Similar to fail_handling above. Specify what to do if a test case results in an error. Default is stop.
context clauses clauses is an arbitrary number of with-/use-clauses which will be used as context clauses for the driver. clauses may span several lines. If it does, all but the first line must be indented. There must be a context clause here which makes visible subprograms Put_Line and New_Line, since the test driver performs its output through these. They will generally be those defined in Ada.Text_IO, thus you should normally write
```
context with Ada.Text_IO; use Ada.Text_IO;
```
in the context subpart. But you may also provide alternate versions for Put_Line and New_Line.
exceptions list list must be a comma-separated, semicolon-terminated list of Ada exception identifiers. These exceptions will be `monitored' by the driver, which means that a specific handler for each of these will be generated in each place where exceptions are checked for. This feature was necessary in the old, Ada 83 version of tg to find the names of exceptions that were raised. It is no longer needed in the current, Ada 95 version. tg creates others-handlers wherever they are necessary, and uses the predefined package Ada.Exceptions to get the exception names.
define lines Subpart for global definitions. lines will be placed into the declarative part of the generated driver. You will typically define objects and subprograms here which are needed by the test cases. Each of the test cases can also have its own define part for data or subprograms needed only by that individual test case. Example:
```
define Exit_Status : Integer;  -- used by all the test cases

       function Result_Is_Correct (R : Result_Type) : Boolean is
       begin
         ...
       end Result_Is_Correct;
```

The Test Case Section

The test case section of the script begins after the last part of the global section. It may contain an arbitrary number of test case descriptions and code parts.

Test Case Descriptions

A test case description represents a single test case. A tg test case is characterized as follows:

It forms an Ada block of its own, thus it is possible to declare any data objects or subprograms needed for that particular test case. You can do this in the define part (which resembles the define part of the global section).
Before the actual test call, you might want to make some preparations. You can specify a prepare part, which is a chunk of code that the driver executes before the test call.
The actual test part consists of a single Ada statement, typically a subprogram call to the software item under test. Hence, another word for the test part is test call.
After the test call has been executed, the driver checks whether it produced the expected result. There are two elements of that result:
1. The program path that was taken after the test call. Execution might have continued normally, or the test call might have raised an exception. The test driver automatically determines the taken path and stores it in the form of a string for future use. You don't need to create any exception handlers yourself.
2. The second element of the result is the value of an arbitrary predicate. You may specify any boolean Ada expression which the driver checks after the test call. It may be a simple check for the value of a variable, or a call to a complex function of type Boolean.
The driver reports the result of the test case (pass/fail) to the standard output stream. You may specify the verboseness of that report. It is, for example, possible to suppress the report of passing test cases completely, while getting a full description of what happened in the event of a failing test case. You set the verboseness through command line options at translation time (see section The tg Command).
Finally, you might have to clean up things. You might, for example, want to delete any files created during the test, etc. You can do this in the optional cleanup part of each test case.

The pattern explained above is precisely reflected in the format of test case descriptions. It looks like this:

*****    test-case-title
define   definitions
prepare  preparations
test     test-statement
pass     [ path ] [ ,  predicate ]
cleanup  cleanup-code

The meaning of the subparts define, prepare, and cleanup is clear from what has been said so far. All of them are optional. The title line, the test part and the pass-clauses, which are mandatory for every test case, are explained below.

For complete example test cases, See section A Complete Example.

Test Case Titles

The start of a test case description is marked by the "keyword" *****, which also serves as an optical marker in the script file. The rest of the line is the test case title. It should explain briefly what is tested in that test case, allowing to decide quickly where the error lies, should this test case fail. Example:

***** function List_Length: List of length zero

tg numbers the test cases from 1 to n. It is recommended that you insert the number of the test case into the title as well, because this makes it easier to find a particular test case later. A parenthesized number at the beginning of the title is recognized by tg as the test case number. Example:

***** (17) function List_Length: List of length zero

It is, of course, easy for these numbers to become inconsistent during the writing of a script file. Therefore, there is a special Test Script Mode for Emacs, which provides a command to insert and correct the numbers automatically. If everything is installed correctly, and if you use the suffix .ts for your script files, you only need to type C-c C-n in Emacs to get the correct numbers.

Should the test case numbers in the script be inconsistent, tg warns about it at translation time.

The Test Call

tg allows an arbitrary chunk of Ada code in the test part. But in general it should only contain a single statement, because this makes it much easier to find out what happened during the test (especially if the test call raises an exception).

The results of the test call should be stored in variables local to that test case, such that they can later be checked in the pass-clause(s).

Example:

test  Result := Test_Item (Some_Parameter);

Pass Clauses

There may be an arbitrary number of pass clauses after the test part. The result of the test case is "pass" if any one of these clauses is met. A pass-clause may have any of the following three forms:

pass  path
pass  predicate
pass  path, predicate

path indicates which path the program should take after the test call. Its value may be either `=>', which stands for the normal path (no exception was raised), or exception exception-name, to indicate that the test call should raise the named exception. exception-name must be an exception identifier which is visible in the test case description. You may use expanded (dot) notation here. If there is no path indication, it defaults to `=>'.

predicate must be an Ada expression yielding a boolean value. It may span several lines (continuation lines indented, as always), and tg uses it as the condition of an if statement. If no predicate is given, it defaults (effectively) to True.

A pass clause is said to be met if both the actual program path is correct and the predicate yields True.

Examples:

pass Number_Of_Elements = 5

pass exception Constraint_Error

pass exception IO_Exceptions.Name_Error, Analyze_Result

pass Status = True
     and then Is_Empty (List)

pass =>, Max = 10.23  -- `=>' is not required here

pass =>               -- the simplest pass-clause

Code Parts

You may insert code parts between the test cases to do additional work. For example, you might want to initialize a package before doing the actual testing. The syntax is fairly simple:

code lines

tg wraps Ada blocks around code parts and adds an exception handler at the end of each. Should an exception propagate out of a code part, the driver reports it as an "error". The global parameter error_handling (also see section The Global Section) determines whether execution of the driver continues after this event or not.

Example:

code Init;
     Put_Line ("Package initialized.");

     if Tasking_Status /= Running then
       Put_Line ("Tasking is off.");
     end if;

     Put_Line ("Now continuing/starting with the test cases.");

Go to the first, previous, next, last section, table of contents.