Regression Tests

We offer a platform to perform regression testing on Fesslix. The source code of the regression tests is hosted on Launchpad (https://code.launchpad.net/~wbetz/fesslix/flx_testing).

Overview

The regression tests consit of the following sets of tests:

sets of regression tests
  • test-set: set of tests that aims to cover all features of Fesslix with tests
  • examples-set: a set of tests that aims to be slightly more practical orientated than the test-set.

Directory structure of a regression set

A directory containing a set of regression tests typically has the following structure:

  • output
    Scripts can use this directory to create files (temporary).
  • refout
    The reference output of the scripts is stored in this directory.
  • refout/other
    Reference versions of the files created by scripts is stored in this directory.

Perform the regression tests

First of all you have to change into the test directory of flx_testing.

Thereafter you can execute the script fesslix_tests.sh:

Terminal

bash fesslix_tests.sh PATH_TO_THE_FESSLIX_EXECUTABLE PATH_TO_EXAMPLE_DIRECTORY YES_OR_NO_FOR_LEAK_CHECK


e.g.:

Terminal

bash fesslix_tests.sh ../build/src/fesslix ../examples yes


Create a test

In order to create a new test you have to perform the steps listed in the following:

1) Change into the right directory

You need to go to the directory of the set of regression tests to which you would like to add the test.

2) Write the parameter file of the test

3) Check if it works correct

4) Give a short description of the test

5) Clear the output directory (if it is not already empty):


Terminal

rm output/*


6) Execute the command:


Terminal

FESSLIX_EXECUTABLE NAME_OF_TEST.flx > refout/NAME_OF_TEST.out 2>&1


7) Move the files to the output directory (if there are some files to move):


Terminal

mv output/* refout/other/


fesslix.org – Home  |  Contact  |  Impressum  |  © 2015-2016 Wolfgang Betz