# flibs/ftnunit(n) 1.1 "flibs"

## NAME

flibs/ftnunit - Unit testing

## TABLE OF CONTENTS

TABLE OF CONTENTS
SYNOPSIS
DESCRIPTION
ROUTINES
GENERATING TESTS FROM A TABLE
TODO
RELATED WORK
COPYRIGHT

## SYNOPSIS

runtests.bat
runtests.sh
runtests.tcl
call runtests( testproc )
call runtests_init
call runtests_final
call test( proc, text )
call assert_true( cond, text )
call assert_false( cond, text )
call assert_equal( value1, value2, text )
call assert_comparable( value1, value2, margin, text )
exists = ftnunit_file_exists( filename )
call ftnunit_get_lun( lun )
call ftnunit_remove_file( filename )
call ftnunit_make_empty_file( filename )

## DESCRIPTION

JUnit is a well-known facility for defining and running unit tests in Java programs. The ftnunit framework was inspired by that facility. It is not as good-looking as JUnit, by no means:

- It has no graphical user-interface
- As Fortran does not allow introspection, the test routines can not be detected automatically, instead as a programmer you need to set up a high-level routine yourself that collects all the unit tests.
- A runtime error, like division by zero, may lead to a termination of the program. There is no (portable) way to catch these. Instead, the framework relies on a batch file or shell script to repeatedly start the program until all tests are run.

Despite these limitations, ftnunit can be a great help:

- The Tcl program gentabletest.tcl generates a complete test program based on a simple input file (see GENERATING TESTS FROM A TABLE).
- The code to test the various components (subroutines, functions, tasks consisting of several program units) can be combined with the program itself, without interfering with the ordinary code.
  This is achieved by defining a single routine (test_all, say) that runs all the unit tests and that is called via the provided routine runtests:

      program myprog
          ...
      !
      ! The routine runtests will check if unit tests are requested
      ! If not, it will return immediately. This way we make sure
      ! the unit tests remain part of the program.
      !
      ! The routine test_all runs all unit tests
      ! (see the dataproc_testing module)
      !
          call runtests( test_all )
      !
      ! Ordinary processing
      !
          ...

      end program

  The routine runtests checks if there is a file "ftnunit.run". If there is such a file, it will run the given subroutine. Otherwise it will return and the rest of the program is executed.
- Because the test code is incorporated in the program itself, it is less likely that they evolve independently: changes in the argument lists of the subroutines and functions may lead to compile errors in the test code.
- There is no need to set up a whole new program for testing portions of the program.

The source file "test_ftnunit.f90" illustrates how to use the ftnunit framework:

- The main program calls the routine "runtests" and passes it the argument "test_all", a routine defined in a module called "dataproc_testing".
- The routine "test_all" consists of nothing but calls to the generic routine "test":

      subroutine test_all

          call test( test_no_file, "Read non-existent file" )
          call test( test_empty_file, "Read an empty file" )
          call test( test_invalid_file, "Read an invalid file" )
          call test( test_ordinary_file, "Read an ordinary file" )

      end subroutine test_all

- The module includes a source file "ftnunit_test.f90". This is a remnant of a previous version. Please ignore this.
- The generic routine "test" checks whether a particular unit test needs to be run (via the test number) and then runs the subroutine that was passed as one of its arguments. One such routine looks like this:

      subroutine test_no_file

          integer :: nodata
          real    :: vmean, vmin, vmax

          call ftnunit_remove_file( 'no_such_file' )
          call write_name( 'no_such_file' )

          call open_files
          call process_data( nodata, vmean, vmax, vmin )

          call assert_true( nodata == 0, "No data read" )

      end subroutine test_no_file

  The assertion is used to check that the result is as expected.
- The program contains some deliberate errors and the resulting log file looks like this":

      Test: Read non-existent file
      Test: Read an empty file
      Test: Read an invalid file
      forrtl: severe (59): list-directed I/O syntax error, unit 11, file c:\arjen\flibs\tests\ftnunit\invalid_file

      Image              PC        Routine            Line        Source
      test_ftnunit.exe     004151B9  Unknown               Unknown  Unknown
      test_ftnunit.exe     00415017  Unknown               Unknown  Unknown
      test_ftnunit.exe     004141F4  Unknown               Unknown  Unknown
      test_ftnunit.exe     00414629  Unknown               Unknown  Unknown
      test_ftnunit.exe     00409C05  Unknown               Unknown  Unknown
      test_ftnunit.exe     004095FB  Unknown               Unknown  Unknown
      test_ftnunit.exe     0040144B  Unknown               Unknown  Unknown
      test_ftnunit.exe     00401FE9  Unknown               Unknown  Unknown
      test_ftnunit.exe     00401A2C  Unknown               Unknown  Unknown
      test_ftnunit.exe     00401BB3  Unknown               Unknown  Unknown
      test_ftnunit.exe     0040294A  Unknown               Unknown  Unknown
      test_ftnunit.exe     0040232E  Unknown               Unknown  Unknown
      test_ftnunit.exe     0044A1E9  Unknown               Unknown  Unknown
      test_ftnunit.exe     00433519  Unknown               Unknown  Unknown
      kernel32.dll       7C816D4F  Unknown               Unknown  Unknown

      Incrementally linked image--PC correlation disabled.
      Test: Read an ordinary file
      Number of failed assertions:                    0
      Number of runs needed to complete the tests:    3

The program is run via one of the following files:

runtests.bat
  A batch file for use under MS Windows
runtests.sh
  A Bourne shell script for use under UNIX/Linux or similar systems, like Cygwin or Mingw.
runtests.tcl
  A Tcl program that presents a simple graphical user-interface

## ROUTINES

The module ftnunit contains the following subroutines and functions:

call runtests( testproc )
  Routine to start the unit tests. It checks if the file "ftnunit.run" exists. If so, it will call the subroutine testproc that was passed. Otherwise it will simply return, so that the ordinary program execution may continue.
  If the subroutine testproc returns, the program stops, unless you have called the subroutine runtests_init before runtests.
call runtests_init
  Routine to initialise the ftnunit system, so that you call runtests more than once. To complete the tests, call runtests_final, as this will print the final statistics and stop the program.
call runtests_final
  Routine to finalise the ftnunit system: it will print the final statistics and stop the program, but only if the file "ftnunit.run" is present.
  subroutine testproc
    Subroutine that calls the individual test routines. It takes no arguments. It wil generally exist of a series of calls to the routine test - see below.
call test( proc, text )
  Routine to run the individual unit test routine (emph proc). It decides if the test has not run yet and if so, the test routine is called. Otherwise it is skipped.
  test takes care of all administrative details.
  Note: to make it possible to use private unit test routines, the source code of this subroutine is kept in a separate file, ftnunit_test.f90 that should be included in an appropriate place in the program's sources. This way, you can make it a private routine in each module. The only public access to the unit testing routines is then via the subroutine testproc that is passed to runtests.
  subroutine proc
    Subroutine that implements an individual unit test. It takes no arguments. Within each such subroutine the complete unit test is run.
  character(len=*), intent(in) text
    Text describing the particular unit test. It is printed in the log file.
call assert_true( cond, text )
  Routine to check that a condition is true. If not, a message is printed in the log file and the number of failures is increased.
  logical cond
    The condition to be checked
  character(len=*), intent(in) text
    Text describing the condition
call assert_false( cond, text )
  Routine to check that a condition is false. If not, a message is printed in the log file and the number of failures is increased.
  logical cond
    The condition to be checked
  character(len=*), intent(in) text
    Text describing the condition
call assert_equal( value1, value2, text )
  Routine to check that two integers are equal or if two one-dimensional integer arrays are equal. If not, a message is printed, along with the values that were different.
  integer [, dimension(:)] value1
    The first integer value or array
  integer [, dimension(:)] value2
    The second integer value or array
  character(len=*), intent(in) text
    Text describing the condition
call assert_comparable( value1, value2, margin, text )
  Routine to check that two reals are almost equal or if two one-dimensional real arrays are almost equal. If not, a message is printed, along with the values that were different.
  The margin is taken as a relative tolerance. Two values are considered almost equal if:

          abs( v1 - v2 ) < margin * (abs(v1)+abs(v2)) / 2

  real [, dimension(:)] value1
    The first real value or array
  real [, dimension(:)] value2
    The second real value or array
  character(len=*), intent(in) text
    Text describing the condition
exists = ftnunit_file_exists( filename )
  Logical function to check that a particular file exists
  character(len=*), intent(in) filename
    Name of the file to be checked
call ftnunit_get_lun( lun )
  Subroutine to get a free LU-number
  integer, intent(out) lun
    Next free LU-number
call ftnunit_remove_file( filename )
  Subroutine to remove (delete) a file
  character(len=*), intent(in) filename
    Name of the file to be removed
call ftnunit_make_empty_file( filename )
  Subroutine to make a new, empty file
  character(len=*), intent(in) filename
    Name of the file to be created

## GENERATING TESTS FROM A TABLE

The Tcl program "gentabletest.tcl" reads the test specifications from an input file and generates a complete Fortran program. The ideas from Bil Kleb's "Toward Scientific Numerical Modeling" ftp://ftp.rta.nato.int/PubFullText/RTO/MP/RTO-MP-AVT-147/RTO-MP-AVT-147-P-17-Kleb.pdf were used for the set-up.

To do: provide a detailed description. For the moment: see example.tbl, including below.

    ! Example of generating test code via a table
    ! -------------------------------------------
    ! The routine to be tested determines the minimum oxygen concentration
    ! in a river, based on the Streeter-Phelps model:
    !
    !    dBOD/dt = -k * BOD
    !
    !    dO2/dt = -k * BOD + ka * (O2sat-O2) / H
    !
    ! where
    !    BOD   - biological oxygen demand (mg O2/l)
    !    O2    - oxygen concentration (mg O2/l)
    !    O2sat - saturation concentration of oxygen (mg O2/l)
    !    k     - decay rate of BOD (1/day)
    !    ka    - reareation rate of oxygen (m/day)
    !    H     - depth of the river
    !
    ! We need boundary (initial) conditions for BOD and oxygen and
    ! the equations describe the concentrations of BOD and oxygen in a
    ! packet of water as it flows along the river.
    !
    ! Note:
    ! It is a very simple model, it is not meant as a realistic
    ! representation.
    !
    ! The routine simply continues the solution until a minimum is found.
    ! The results are: oxymin and time
    !
    !
    ! The keyword DECLARATIONS introduces the declarations we need for the
    ! complete generated code
    !
    DECLARATIONS
        use streeter_phelps
        real :: bod, oxy
        real :: k, ka, h, oxysat, dt, oxymin, time
    !
    ! The keyword CODE introduces the code fragment required to run the
    ! routine or routines. The results and possible checking of error
    ! conditions are separated.
    !
    CODE
        call compute_min_oxygen( bod, oxy, k, ka, h, oxysat, dt, oxymin, time )
    !
    ! The keyword RESULT indicates which arguments/variables hold the
    ! interesting results. Specify one name per line (you can not currently
    ! use array elements) and the allowed margin (taken as absolute, if
    ! followed by "%" as a percentage)
    !
    RESULT
        oxymin  0.001        ! Minimum oxygen concentration
        time    0.01%        ! Time the minimum is reached

    !
    ! The keyword ERROR is used for a code fragment that checks if the
    ! routine has correctly found an error in the input (that is, some
    ! parameter value is out of range). The code is invoked when any of
    ! result variables in a table entry has the keyword ERROR instead of
    ! a proper value.
    ! Use the subroutine "error" to indicate the correctly reported error
    ! condition.
    !
    ERROR
        if ( time == -999.0 ) then
            call error
        endif
    !
    ! The keyword RANGES specifies that the variables are to be taken
    ! from a uniform or a normal distribution. The generated program will
    ! simply select values at random and run the code with them. The report
    ! consists of the detailed output as well as a summary.
    !
    RANGES
        oxy   10.0   2.0  Uniform ! Name of the variable, the mean and the margin (uniform)
                                  ! Normal: mean and standard deviation followed by Normal
                                  ! Note: all parameters must be given!
    !
    ! The keyword TABLE indicates the beginning of a table of input data and
    ! expected values. The first (non-comment) line contains the names of
    ! the variables as used in the code fragments and all others are the
    ! values expected.
    !
    ! There are two special values:
    ! ? -     indicating an unknown value for result variables and a "do not
    !         care" value for input variables
    !         It is useful to generate a table that does contain the (computed)
    !         results (see the file table.out) or to indicate situations
    !         where one or more input variables are out of range and this
    !         should lead to an error
    ! ERROR - indicating that the entry should cause the routine to be
    !         tested to flag an error condition.
    !
    TABLE
    dt   oxy       bod       oxysat    h         k         ka        oxymin    time
    0.1  10        1         10        10        0.1       1.0       10.0      2.0
    1.0  10        1         10        10        0.1       1.0       ?         ?
    !
    ! This case is unacceptable: time step must be positive
    0.0  ?         ?         ?         ?         ?         ?         ?         ERROR
    1.0  0.        10        10        10        0.1       1.0       ?         ?

## TODO

The following things are still left to do:

- Proper inclusion of the routine prolog and epilog
- Extension of the set of assertion routines

## RELATED WORK

There are at least two similar initiatives with regard to a unit testing framework for Fortran:

- Funit (implemented in Fortran and Ruby) by Bil Kleb and others
- A framework implemented in Fortran by Brice Womack and Tom Clune
- FRUIT (implemented in Fortran and Ruby) by Andrew Chen

(Note: To avoid confusion, I have renamed my original module "funit" to ftnunit)

## COPYRIGHT

Copyright © 2006 Arjen Markus <arjenmarkus@sourceforge.net>