Introduction

A test driver is a program which tests a piece of software.

If you want to test something, for example an Ada package, you normally need to design a huge number of separate, elementary tests, which must be performed one after the other. Each elementary test typically consists of a subprogram call to some routine of the package, passing certain input data to it, plus a subsequent analysis of the results of that call. We call such an elementary test a test case.

It follows that, normally, test drivers are very simple but large programs. Mostly, they consist of endless repetitions of a single common pattern: that of a test case. Thus, it would be nice to have a tool that generates such driver programs automatically.

tg does just that. You feed it with a description of a test, given in some special, convenient format in which you only need to specify the essentials of each test case. tg takes this description -- we call it the test script -- and translates it into the source code of a corresponding test driver, which you can then compile, link, and execute.

tg's native language is Ada, but you might also use it to test software written in other programming languages. (This would require `interfacing' that software to Ada, since the test driver is always an Ada program.) tg itself is also written in Ada.

(Throughout this document, Ada refers to "Ada 95", not to the now-obsolete version of the language called "Ada 83". You cannot use tg in an Ada 83 environment. Though you might contact me, the author, for an old version of tg written in Ada 83 -- I still have it on disc.)

A Small Example

Suppose you want to test an Ada function which counts the number of `&' characters within a given string.

function Count_Ampersand (Str : in String) return Natural;

One of your test cases might be to call it with parameter "abc&&&abc" and verify that the result is three. The code for this could look as follows.

Put ("Testing three ampersands in the middle... ");
begin
  Count := Count_Ampersand ("abc&&&abc");
  if Count = 3 then
    Put_Line ("pass.");
  else
    Put_Line ("fail.");
  end if;
exception
  when others =>
    Put_Line ("fail.");
end;

That is a lot of code for a single test case. In tg notation, to specify this test case, you would just write:

***** Testing three ampersands in the middle...
test Count := Count_Ampersand ("abc&&&abc");
pass Count = 3

The tg command translates this into code similar to that shown above and puts it into a complete driver program.

Testing Terminology

As it was explained above, tg views a test as a sequence of test cases. The test is described, or specified, in a test script file. You write this script in a special macro-type language which is described in the main part of this manual. The piece of software which you test is called the test item. It may be a single subprogram, or a package, or a complete software system.

We say that a test is performed by executing the test driver program. Likewise, the individual test cases are performed when the code into which their description was translated is executed.

The crucial point of a test case is a subprogram call to the test item. We call this the test call. The driver might do some preparatory work before the test call, and after the call has returned, the driver analyzes its results.

That means, there are three types of results: the result of the test call, of the test case, and of the complete test as a whole.

• The result of the test call is what it returns to the test driver. This can be the return value of a function, or the values of any out parameters. But it also includes everything that can be determined as an effect of the test call, like the state of a global flag, or output data written to some file, etc. The result of the test call further includes the program path taken after it, because after an Ada subprogram call execution may either continue normally or it may be aborted by the raising of an exception.
• The driver finds the result of the test case by comparing the result of the test call to the expected result. If they are equal, the test item is said to have passed the test case, otherwise it failed it. We also say the result of the test case is either pass or fail. There is a third possibility, which occurs when something unexpected happens in a test case outside the test call. For example, you might want to test a function that returns the length of a list, and you first have to build an example list to pass to that function. If during building that list something goes wrong, the result of the test case is not `fail', but error. The result `error' is more precisely defined as an exception propagating out of any part of a test case, apart from the test call.
• Finally, there is a total test result which is defined as pass if the test item passed all the test cases, and fail if it didn't. The total test result is undefined if the result of any test case was `error'.

Go to the first, previous, next, last section, table of contents.