Generate the exact list of test data files needed to run a test executable using automated means across multiple OSes.
The Chromium build system, basically building Chromium, is (almost) deterministic. The build is declared in the GYP language, which lists all the source files and supports conditions to work with different OSes, different targets, like with Aura or with Valgrind. As such, it is possible to build a single source file and knowing exactly what input files need to be present to compile it.
Running tests have always assumed they are run directly in the directory they were built in, that is, all the source tree is available. This has huge cost, as only a tiny fraction of these files are needed, and an even a smaller fraction from the files that were built.
Generating the exact list of needed files to run a test would require an insane amount of manual work so it must be automated and work across multiple OSes.
For now, no infrastructure is needed as this is all python code.
There are 2 importants classes; ApiBase and Results.
ApiBase declares the common interface to trace a child process, independent of the implementation or the OS. It has a limited interface and separates the action of tracing from the action of analyzing the traces. ApiBase.get_tracer() returns an ApiBase.Tracer derived instance that manages the lifetime of the tracing infrastructure. Once tracing is done, ApiBase.parse_log() generates a list of Results instance.
The implementation is to use dtrace on OSX, "NT Kernel Logger" on Windows and strace on linux. Much of the particularities between each implementation is described in the docstring of each implementation class. As a starter, strace is a pure user-mode tracer, while dtrace and the NT Kernel Logger are kernel-only tracers, which have no effect on the traced processes. The API hides this fact and have a consistent API across user mode and kernel mode tracers.
It holds the results of the trace. It is immutable so any mutation with one of its member function effectively create a new instance. The results contains a tree of Results.Process instances, each representing the root process and each child process started by the root process. Each process has a .files attribute that list each of the files accessed by this process. It also contains meta data like the command line, the executable name, etc.
Chromium uses GTest to drive its unit tests. To improve efficiency, a trace can just work on a subset. The script
It's source lives inside the chromium tree at http://src.chromium.org/viewvc/chrome/trunk/tools/swarm_client along the Isolate scripts. It was mainly coded by maruel@ with the assistance of csharp@.
Wall time duration is reduced by tracing multiple test cases concurrently. This significantly help on 8+ cores systems. No distributed tracing infrastructure is planned yet.
Only system-local scalability is considered, as no unattended "Tracing a unit test on Swarm" has been implemented yet (tracing on swarm probably won't be implemented since the swarm bots would have to download the full chromium checkout). The scalability effort is to make sure all the CPU cores are used by tracing multiple test cases simultaneously, reducing wall-time to get the complete results for an unit test executable.
N/A as the tool require root access on OSX and Windows.
The testing is in two parts;
So the automated smoke test must be run on each of the supported platform. This is currently done manually due to the limited number of contributors.