490 lines
15 KiB
Groff
490 lines
15 KiB
Groff
[comment {-*- flibs -*- doctools manpage}]
|
|
[manpage_begin flibs/ftnunit n 1.1]
|
|
[copyright {2006 Arjen Markus <arjenmarkus@sourceforge.net>}]
|
|
[moddesc flibs]
|
|
[titledesc {Unit testing}]
|
|
|
|
[description]
|
|
|
|
[emph JUnit] is a well-known facility for defining and running unit
|
|
tests in Java programs. The [emph ftnunit] framework was inspired by
|
|
that facility. It is not as good-looking as JUnit, by no means:
|
|
[list_begin bullet]
|
|
[bullet]
|
|
It has no graphical user-interface
|
|
|
|
[bullet]
|
|
As Fortran does not allow introspection, the test routines can not
|
|
be detected automatically, instead as a programmer you need to set up a
|
|
high-level routine yourself that collects all the unit tests.
|
|
|
|
[bullet]
|
|
A runtime error, like division by zero, may lead to a termination of
|
|
the program. There is no (portable) way to catch these. Instead, the
|
|
framework relies on a batch file or shell script to repeatedly start the
|
|
program until all tests are run.
|
|
|
|
[list_end]
|
|
|
|
Despite these limitations, [emph ftnunit] can be a great help:
|
|
|
|
[list_begin bullet]
|
|
[bullet]
|
|
The Tcl program [emph gentabletest.tcl] generates a complete test program based
|
|
on a simple input file (see [sectref "GENERATING TESTS FROM A TABLE"]).
|
|
|
|
[bullet]
|
|
The code to test the various components (subroutines, functions, tasks
|
|
consisting of several program units) can be combined with the program
|
|
itself, without interfering with the ordinary code.
|
|
[nl]
|
|
This is achieved by defining a single routine (test_all, say) that runs
|
|
all the unit tests and that is called via the provided routine
|
|
[emph runtests]:
|
|
|
|
[example {
|
|
program myprog
|
|
...
|
|
!
|
|
! The routine runtests will check if unit tests are requested
|
|
! If not, it will return immediately. This way we make sure
|
|
! the unit tests remain part of the program.
|
|
!
|
|
! The routine test_all runs all unit tests
|
|
! (see the dataproc_testing module)
|
|
!
|
|
call runtests( test_all )
|
|
!
|
|
! Ordinary processing
|
|
!
|
|
...
|
|
|
|
end program
|
|
}]
|
|
|
|
The routine runtests checks if there is a file "ftnunit.run". If there is
|
|
such a file, it will run the given subroutine. Otherwise it will return
|
|
and the rest of the program is executed.
|
|
|
|
[bullet]
|
|
Because the test code is incorporated in the program itself, it is less
|
|
likely that they evolve independently: changes in the argument lists of
|
|
the subroutines and functions may lead to compile errors in the test
|
|
code.
|
|
|
|
[bullet]
|
|
There is no need to set up a whole new program for testing portions of
|
|
the program.
|
|
|
|
[list_end]
|
|
|
|
The source file "test_ftnunit.f90" illustrates how to use the ftnunit
|
|
framework:
|
|
|
|
[list_begin bullet]
|
|
[bullet]
|
|
The main program calls the routine "runtests" and passes it the argument
|
|
"test_all", a routine defined in a module called "dataproc_testing".
|
|
|
|
[bullet]
|
|
The routine "test_all" consists of nothing but calls to the generic
|
|
routine "test":
|
|
|
|
[example {
|
|
subroutine test_all
|
|
|
|
call test( test_no_file, "Read non-existent file" )
|
|
call test( test_empty_file, "Read an empty file" )
|
|
call test( test_invalid_file, "Read an invalid file" )
|
|
call test( test_ordinary_file, "Read an ordinary file" )
|
|
|
|
end subroutine test_all
|
|
}]
|
|
|
|
[bullet]
|
|
The module includes a source file "ftnunit_test.f90". This is a remnant
|
|
of a previous version. Please ignore this.
|
|
|
|
[bullet]
|
|
The generic routine "test" checks whether a particular unit
|
|
test needs to be run (via the test number) and then runs the subroutine
|
|
that was passed as one of its arguments. One such routine looks like
|
|
this:
|
|
|
|
[example {
|
|
subroutine test_no_file
|
|
|
|
integer :: nodata
|
|
real :: vmean, vmin, vmax
|
|
|
|
call ftnunit_remove_file( 'no_such_file' )
|
|
call write_name( 'no_such_file' )
|
|
|
|
call open_files
|
|
call process_data( nodata, vmean, vmax, vmin )
|
|
|
|
call assert_true( nodata == 0, "No data read" )
|
|
|
|
end subroutine test_no_file
|
|
}]
|
|
|
|
The assertion is used to check that the result is as expected.
|
|
|
|
[bullet]
|
|
The program contains some deliberate errors and the resulting
|
|
log file looks like this":
|
|
|
|
[example {
|
|
Test: Read non-existent file
|
|
Test: Read an empty file
|
|
Test: Read an invalid file
|
|
forrtl: severe (59): list-directed I/O syntax error, unit 11, file c:\arjen\flibs\tests\ftnunit\invalid_file
|
|
|
|
Image PC Routine Line Source
|
|
test_ftnunit.exe 004151B9 Unknown Unknown Unknown
|
|
test_ftnunit.exe 00415017 Unknown Unknown Unknown
|
|
test_ftnunit.exe 004141F4 Unknown Unknown Unknown
|
|
test_ftnunit.exe 00414629 Unknown Unknown Unknown
|
|
test_ftnunit.exe 00409C05 Unknown Unknown Unknown
|
|
test_ftnunit.exe 004095FB Unknown Unknown Unknown
|
|
test_ftnunit.exe 0040144B Unknown Unknown Unknown
|
|
test_ftnunit.exe 00401FE9 Unknown Unknown Unknown
|
|
test_ftnunit.exe 00401A2C Unknown Unknown Unknown
|
|
test_ftnunit.exe 00401BB3 Unknown Unknown Unknown
|
|
test_ftnunit.exe 0040294A Unknown Unknown Unknown
|
|
test_ftnunit.exe 0040232E Unknown Unknown Unknown
|
|
test_ftnunit.exe 0044A1E9 Unknown Unknown Unknown
|
|
test_ftnunit.exe 00433519 Unknown Unknown Unknown
|
|
kernel32.dll 7C816D4F Unknown Unknown Unknown
|
|
|
|
Incrementally linked image--PC correlation disabled.
|
|
Test: Read an ordinary file
|
|
Number of failed assertions: 0
|
|
Number of runs needed to complete the tests: 3
|
|
}]
|
|
|
|
[list_end]
|
|
|
|
The program is run via one of the following files:
|
|
|
|
[list_begin definitions]
|
|
|
|
[call runtests.bat]
|
|
A batch file for use under MS Windows
|
|
[call runtests.sh]
|
|
A Bourne shell script for use under UNIX/Linux or similar systems, like
|
|
Cygwin or Mingw.
|
|
[call runtests.tcl]
|
|
A Tcl program that presents a simple graphical user-interface
|
|
|
|
[list_end]
|
|
|
|
[section ROUTINES]
|
|
The module ftnunit contains the following subroutines and functions:
|
|
|
|
[list_begin definitions]
|
|
|
|
[call [cmd "call runtests( testproc )"]]
|
|
Routine to start the unit tests. It checks if the file "ftnunit.run"
|
|
exists. If so, it will call the subroutine [emph testproc] that was
|
|
passed. Otherwise it will simply return, so that the ordinary program
|
|
execution may continue.
|
|
[nl]
|
|
If the subroutine testproc returns, the program stops, unless you have
|
|
called the subroutine [emph runtests_init] before [emph runtests].
|
|
|
|
[call [cmd "call runtests_init"]]
|
|
Routine to initialise the ftnunit system, so that you call [emph runtests]
|
|
more than once. To complete the tests, call [emph runtests_final], as
|
|
this will print the final statistics and stop the program.
|
|
|
|
[call [cmd "call runtests_final"]]
|
|
Routine to finalise the ftnunit system: it will print the final statistics
|
|
and stop the program, but only if the file "ftnunit.run" is present.
|
|
|
|
[list_begin arg]
|
|
[arg_def "subroutine" testproc]
|
|
Subroutine that calls the individual test routines. It takes no
|
|
arguments. It wil generally exist of a series of calls to the
|
|
routine [emph test] - see below.
|
|
[list_end]
|
|
|
|
|
|
[call [cmd "call test( proc, text )"]]
|
|
Routine to run the individual unit test routine (emph proc). It decides
|
|
if the test has not run yet and if so, the test routine is called.
|
|
Otherwise it is skipped.
|
|
[nl]
|
|
[emph test] takes care of all administrative details.
|
|
[nl]
|
|
Note: to make it possible to use [emph private] unit test routines,
|
|
the source code of this subroutine is kept in a separate file,
|
|
[emph ftnunit_test.f90] that should be included in an appropriate
|
|
place in the program's sources. This way, you can make it a private
|
|
routine in each module. The only public access to the unit testing
|
|
routines is then via the subroutine [emph testproc] that is passed to
|
|
[emph runtests].
|
|
|
|
[list_begin arg]
|
|
[arg_def "subroutine" proc]
|
|
Subroutine that implements an individual unit test. It takes no
|
|
arguments. Within each such subroutine the complete unit test is run.
|
|
|
|
[arg_def "character(len=*), intent(in)" text]
|
|
Text describing the particular unit test. It is printed in the log
|
|
file.
|
|
[list_end]
|
|
|
|
|
|
[call [cmd "call assert_true( cond, text )"]]
|
|
Routine to check that a condition is true. If not, a message is printed
|
|
in the log file and the number of failures is increased.
|
|
|
|
[list_begin arg]
|
|
[arg_def "logical" cond]
|
|
The condition to be checked
|
|
|
|
[arg_def "character(len=*), intent(in)" text]
|
|
Text describing the condition
|
|
[list_end]
|
|
|
|
|
|
[call [cmd "call assert_false( cond, text )"]]
|
|
Routine to check that a condition is false. If not, a message is
|
|
printed in the log file and the number of failures is increased.
|
|
|
|
[list_begin arg]
|
|
[arg_def "logical" cond]
|
|
The condition to be checked
|
|
|
|
[arg_def "character(len=*), intent(in)" text]
|
|
Text describing the condition
|
|
[list_end]
|
|
|
|
|
|
[call [cmd "call assert_equal( value1, value2, text )"]]
|
|
Routine to check that two integers are equal or if two one-dimensional
|
|
integer arrays are equal. If not, a message is printed, along with the
|
|
values that were different.
|
|
|
|
[list_begin arg]
|
|
[arg_def "integer \[, dimension(:)\]" value1]
|
|
The first integer value or array
|
|
|
|
[arg_def "integer \[, dimension(:)\]" value2]
|
|
The second integer value or array
|
|
|
|
[arg_def "character(len=*), intent(in)" text]
|
|
Text describing the condition
|
|
[list_end]
|
|
|
|
|
|
[call [cmd "call assert_comparable( value1, value2, margin, text )"]]
|
|
Routine to check that two reals are almost equal or if two one-dimensional
|
|
real arrays are almost equal. If not, a message is printed, along with
|
|
the values that were different.
|
|
[nl]
|
|
The margin is taken as a relative tolerance. Two values are
|
|
considered almost equal if:
|
|
[example {
|
|
abs( v1 - v2 ) < margin * (abs(v1)+abs(v2)) / 2
|
|
}]
|
|
|
|
[list_begin arg]
|
|
[arg_def "real \[, dimension(:)\]" value1]
|
|
The first real value or array
|
|
|
|
[arg_def "real \[, dimension(:)\]" value2]
|
|
The second real value or array
|
|
|
|
[arg_def "character(len=*), intent(in)" text]
|
|
Text describing the condition
|
|
[list_end]
|
|
|
|
|
|
[call [cmd "exists = ftnunit_file_exists( filename )"]]
|
|
Logical function to check that a particular file exists
|
|
|
|
[list_begin arg]
|
|
[arg_def "character(len=*), intent(in)" filename]
|
|
Name of the file to be checked
|
|
[list_end]
|
|
|
|
|
|
[call [cmd "call ftnunit_get_lun( lun )"]]
|
|
Subroutine to get a free LU-number
|
|
|
|
[list_begin arg]
|
|
[arg_def "integer, intent(out)" lun]
|
|
Next free LU-number
|
|
[list_end]
|
|
|
|
|
|
[call [cmd "call ftnunit_remove_file( filename )"]]
|
|
Subroutine to remove (delete) a file
|
|
|
|
[list_begin arg]
|
|
[arg_def "character(len=*), intent(in)" filename]
|
|
Name of the file to be removed
|
|
[list_end]
|
|
|
|
|
|
[call [cmd "call ftnunit_make_empty_file( filename )"]]
|
|
Subroutine to make a new, empty file
|
|
|
|
[list_begin arg]
|
|
[arg_def "character(len=*), intent(in)" filename]
|
|
Name of the file to be created
|
|
[list_end]
|
|
|
|
[list_end]
|
|
|
|
|
|
[section "GENERATING TESTS FROM A TABLE"]
|
|
The Tcl program "gentabletest.tcl" reads the test specifications from an
|
|
input file and generates a complete Fortran program. The ideas from
|
|
Bil Kleb's "Toward Scientific Numerical Modeling"
|
|
[uri ftp://ftp.rta.nato.int/PubFullText/RTO/MP/RTO-MP-AVT-147/RTO-MP-AVT-147-P-17-Kleb.pdf]
|
|
were used for the set-up.
|
|
|
|
[para]
|
|
To do: provide a detailed description. For the moment: see [emph example.tbl], including below.
|
|
|
|
[example {
|
|
! Example of generating test code via a table
|
|
! -------------------------------------------
|
|
! The routine to be tested determines the minimum oxygen concentration
|
|
! in a river, based on the Streeter-Phelps model:
|
|
!
|
|
! dBOD/dt = -k * BOD
|
|
!
|
|
! dO2/dt = -k * BOD + ka * (O2sat-O2) / H
|
|
!
|
|
! where
|
|
! BOD - biological oxygen demand (mg O2/l)
|
|
! O2 - oxygen concentration (mg O2/l)
|
|
! O2sat - saturation concentration of oxygen (mg O2/l)
|
|
! k - decay rate of BOD (1/day)
|
|
! ka - reareation rate of oxygen (m/day)
|
|
! H - depth of the river
|
|
!
|
|
! We need boundary (initial) conditions for BOD and oxygen and
|
|
! the equations describe the concentrations of BOD and oxygen in a
|
|
! packet of water as it flows along the river.
|
|
!
|
|
! Note:
|
|
! It is a very simple model, it is not meant as a realistic
|
|
! representation.
|
|
!
|
|
! The routine simply continues the solution until a minimum is found.
|
|
! The results are: oxymin and time
|
|
!
|
|
!
|
|
! The keyword DECLARATIONS introduces the declarations we need for the
|
|
! complete generated code
|
|
!
|
|
DECLARATIONS
|
|
use streeter_phelps
|
|
real :: bod, oxy
|
|
real :: k, ka, h, oxysat, dt, oxymin, time
|
|
!
|
|
! The keyword CODE introduces the code fragment required to run the
|
|
! routine or routines. The results and possible checking of error
|
|
! conditions are separated.
|
|
!
|
|
CODE
|
|
call compute_min_oxygen( bod, oxy, k, ka, h, oxysat, dt, oxymin, time )
|
|
!
|
|
! The keyword RESULT indicates which arguments/variables hold the
|
|
! interesting results. Specify one name per line (you can not currently
|
|
! use array elements) and the allowed margin (taken as absolute, if
|
|
! followed by "%" as a percentage)
|
|
!
|
|
RESULT
|
|
oxymin 0.001 ! Minimum oxygen concentration
|
|
time 0.01% ! Time the minimum is reached
|
|
|
|
!
|
|
! The keyword ERROR is used for a code fragment that checks if the
|
|
! routine has correctly found an error in the input (that is, some
|
|
! parameter value is out of range). The code is invoked when any of
|
|
! result variables in a table entry has the keyword ERROR instead of
|
|
! a proper value.
|
|
! Use the subroutine "error" to indicate the correctly reported error
|
|
! condition.
|
|
!
|
|
ERROR
|
|
if ( time == -999.0 ) then
|
|
call error
|
|
endif
|
|
!
|
|
! The keyword RANGES specifies that the variables are to be taken
|
|
! from a uniform or a normal distribution. The generated program will
|
|
! simply select values at random and run the code with them. The report
|
|
! consists of the detailed output as well as a summary.
|
|
!
|
|
RANGES
|
|
oxy 10.0 2.0 Uniform ! Name of the variable, the mean and the margin (uniform)
|
|
! Normal: mean and standard deviation followed by Normal
|
|
! Note: all parameters must be given!
|
|
!
|
|
! The keyword TABLE indicates the beginning of a table of input data and
|
|
! expected values. The first (non-comment) line contains the names of
|
|
! the variables as used in the code fragments and all others are the
|
|
! values expected.
|
|
!
|
|
! There are two special values:
|
|
! ? - indicating an unknown value for result variables and a "do not
|
|
! care" value for input variables
|
|
! It is useful to generate a table that does contain the (computed)
|
|
! results (see the file table.out) or to indicate situations
|
|
! where one or more input variables are out of range and this
|
|
! should lead to an error
|
|
! ERROR - indicating that the entry should cause the routine to be
|
|
! tested to flag an error condition.
|
|
!
|
|
TABLE
|
|
dt oxy bod oxysat h k ka oxymin time
|
|
0.1 10 1 10 10 0.1 1.0 10.0 2.0
|
|
1.0 10 1 10 10 0.1 1.0 ? ?
|
|
!
|
|
! This case is unacceptable: time step must be positive
|
|
0.0 ? ? ? ? ? ? ? ERROR
|
|
1.0 0. 10 10 10 0.1 1.0 ? ?
|
|
}]
|
|
|
|
[section TODO]
|
|
The following things are still left to do:
|
|
[list_begin bullet]
|
|
[bullet]
|
|
Proper inclusion of the routine [emph prolog] and [emph epilog]
|
|
|
|
[bullet]
|
|
Extension of the set of assertion routines
|
|
|
|
[list_end]
|
|
|
|
[section "RELATED WORK"]
|
|
There are at least two similar initiatives with regard to a unit testing
|
|
framework for Fortran:
|
|
|
|
[list_begin bullet]
|
|
[bullet]
|
|
[uri {http://nasarb.rubyforge.org} {Funit (implemented in Fortran and
|
|
Ruby)}] by Bil Kleb and others
|
|
|
|
[bullet]
|
|
[uri {http://www.sourceforge.net/projects/pfunit} {A framework
|
|
implemented in Fortran}] by Brice Womack and Tom Clune
|
|
|
|
[bullet]
|
|
[uri {http://www.sourceforge.net/projects/fortranxunit} {FRUIT
|
|
(implemented in Fortran and Ruby)}] by Andrew Chen
|
|
|
|
[list_end]
|
|
|
|
(Note: To avoid confusion, I have renamed my original module "funit" to
|
|
[emph ftnunit])
|
|
|
|
[manpage_end]
|