A.1 Test Functions

test scans the named script file looking for lines which start with %!. The prefix is stripped off and the rest of the line is processed through the octave interpreter. If the code generates an error, then the test is said to fail.

Since eval() will stop at the first error it encounters, you must divide your tests up into blocks, with anything in a separate block evaluated separately. Blocks are introduced by the keyword test immediately following the %!. For example,

        %!test error("this test fails!");
        %!test "this test doesn't fail since it doesn't generate an error";

When a test fails, you will see something like:

          ***** test error('this test fails!')
        !!!!! test failed
        this test fails!

Generally, to test if something works, you want to assert that it produces a correct value. A real test might look something like

        %!test
        %! a = [1, 2, 3; 4, 5, 6]; B = [1; 2];
        %! expect = [ a ; 2*a ];
        %! get = kron (b, a);
        %! if (any(size(expect) != size(get)))
        %!    error ("wrong size: expected %d,%d but got %d,%d",
        %!           size(expect), size(get));
        %! elseif (any(any(expect!=get)))
        %!    error ("didn't get what was expected.");
        %! endif

To make the process easier, use the assert function. For example, with assert the previous test is reduced to:

        %!test
        %! a = [1, 2, 3; 4, 5, 6]; b = [1; 2];
        %! assert (kron (b, a), [ a; 2*a ]);

assert can accept a tolerance so that you can compare results absolutely or relatively. For example, the following all succeed:

        %!test assert (1+eps, 1, 2*eps)          # absolute error
        %!test assert (100+100*eps, 100, -2*eps) # relative error

You can also do the comparison yourself, but still have assert generate the error:

        %!test assert (isempty([]))
        %!test assert ([ 1,2; 3,4 ] > 0)

Because assert is so frequently used alone in a test block, there is a shorthand form:

        %!assert (...)

which is equivalent to:

        %!test assert (...)

Each block is evaluated in its own function environment, which means that variables defined in one block are not automatically shared with other blocks. If you do want to share variables, then you must declare them as shared before you use them. For example, the following declares the variable a, gives it an initial value (default is empty), then uses it in several subsequent tests.

        %!shared a
        %! a = [1, 2, 3; 4, 5, 6];
        %!assert (kron ([1; 2], a), [ a; 2*a ]);
        %!assert (kron ([1, 2], a), [ a, 2*a ]);
        %!assert (kron ([1,2; 3,4], a), [ a,2*a; 3*a,4*a ]);

You can share several variables at the same time:

        %!shared a, b

You can also share test functions:

        %!function a = fn(b)
        %!  a = 2*b;
        %!assert (a(2),4);

Note that all previous variables and values are lost when a new shared block is declared.

Error and warning blocks are like test blocks, but they only succeed if the code generates an error. You can check the text of the error is correct using an optional regular expression <pattern>. For example:

        %!error <passes!> error('this test passes!');

If the code doesn't generate an error, the test fails. For example,

        %!error "this is an error because it succeeds.";

produces

        ***** error "this is an error because it succeeds.";
        !!!!! test failed: no error

It is important to automate the tests as much as possible, however some tests require user interaction. These can be isolated into demo blocks, which if you are in batch mode, are only run when called with demo or verbose. The code is displayed before it is executed. For example,

        %!demo
        %! t=[0:0.01:2*pi]; x=sin(t);
        %! plot(t,x);
        %! you should now see a sine wave in your figure window

produces

        > t=[0:0.01:2*pi]; x=sin(t);
        > plot(t,x);
        > you should now see a sine wave in your figure window
        Press <enter> to continue:

Note that demo blocks cannot use any shared variables. This is so that they can be executed by themselves, ignoring all other tests.

If you want to temporarily disable a test block, put # in place of the block type. This creates a comment block which is echoed in the log file, but is not executed. For example:

        %!#demo
        %! t=[0:0.01:2*pi]; x=sin(t);
        %! plot(t,x);
        %! you should now see a sine wave in your figure window

Block type summary:

%!test

check that entire block is correct

%!error

check for correct error message

%!warning

check for correct warning message

%!demo

demo only executes in interactive mode

%!#

comment: ignore everything within the block

%!shared x,y,z

declares variables for use in multiple tests

%!function

defines a function value for a shared variable

%!assert (x, y, tol)

shorthand for %!test assert (x, y, tol)

You can also create test scripts for builtins and your own C++ functions. Just put a file of the function name on your path without any extension and it will be picked up by the test procedure. You can even embed tests directly in your C++ code:

        #if 0
        %!test disp('this is a test')
        #endif

or

        /*
        %!test disp('this is a test')
        */

but then the code will have to be on the load path and the user will have to remember to type test('name.cc'). Conversely, you can separate the tests from normal octave script files by putting them in plain files with no extension rather than in script files.