Anatomy of test.test
This document will go through the pieces of test.test that are available to use, describing what each does and when it could be used.
job backreference to the job this test instance is part of
outputdir eg. results/<job>/<testname.tag>
resultsdir eg. results/<job>/<testname.tag>/results
profdir eg. results/<job>/<testname.tag>/profiling
debugdir eg. results/<job>/<testname.tag>/debug
bindir eg. tests/<test>
src eg. tests/<test>/src
tmpdir eg. tmp/<tempname>_<testname.tag>
Everytime a test runs this function runs. For example, if you have iterations=100 set this will run 100 times. Use this when you need to do something like ensure a package is there.
setup() runs the first time the test is set up. This is often used for compiling source code one should not need to recompile the source for the test for every iteration or install the same
This is called by job.run_test N times, where N is controlled by the interations parameter to run_test (Defaulting to one). it also gets called an additional time with profilers turn on, if any profilers are enabled.
This processes any results generated by the test iteration and writes them out into a keyval. It's generally not called for the profiling iteration, as that may have difference performance implications.