OverviewThis page explains how to convert a googletest executable into a fully isolated test. This is done by describing runtime dependencies for a test.
For information about the infrastructure itself and the roadmap, see Isolated Testing Infrastructure.
Note: If you're a Googler who needs to perform a manual test on a platform for which you cannot readily build (e.g. Windows Developer with no development environment on your Mac book), follow these instructions to get the build from a try run.
What's "Isolate"?Goal: describe the exact list of files needed to run a executable.The isolate project is all the file formats, tools and server code to make this work fast and efficiently. Every test executable list the data files it depends on, which allows the testers bots to only download the files they need instead of the whole Chromium source tree.
They are described in BUILD.gn files.
What's "Swarming"?Goal: distribute tasks fast and efficiently in an heterogeneous fleet of bots.
The swarming project is all the tools, server and bot code to run a step (like a unit test) on a remote bot and get results back. It has native google-test sharding capability.
What are the advantages?By reducing the amount of data that we need to transfer to the tester machines, it becomes much easier to increase the number of tester bots that we have using Swarming (part of LUCI). Adding a new test
Expectations of the tests:
HowTosDownload binaries from a try job
Run a test isolated locallyecho gn > out/Release/mb_type # Must be done once to avoid mb.py from performing a clobber tools/mb/mb.py run //out/Release base_unittests # Builds, generates .isolate (via gn desc runtime_deps), and runs (via "isolate.py run") Run a test built locally on Swarming1. Eligibility and loginRight now, only users Chromium team members can use the infrastructure directly. Note that the whole Swarming infrastructure is open source so if any other company would help to recreate the same infrastructure internally, send us a note at swarming-eng@googlegroups.com By login first, you have access tokens so that the following commands do not have to constantly prompt for your identity. You only need to do this once: python tools/swarming_client/auth.py login --service=https://isolateserver.appspot.com python tools/swarming_client/auth.py login --service=https://chromium-swarm.appspot.com If you are running through a text only session on a remote machine, append argument --auth-no-local-webserver ninja -C out/Release base_unittests 3. Compute .isolated file and upload itThe isolated file contains the SHA-1 of each input files. It is archives along all the inputs to the Isolate server. Since the isolate server is a content-addressed cache, only the ones missing from the cache are uploaded. Depending on your connection speed and the size of the executable, it may take up to a minute: python tools/swarming_client/isolate.py archive \ -I https://isolateserver.appspot.com \ -i out/Debug/base_unittests.isolate \ -s out/Debug/base_unittests.isolated 4. Trigger the taskThat's where a task is requested, specifying the isolated (tree of SHA-1 of files to map in):
python tools/swarming_client/swarming.py trigger \
-S chromium-swarm.appspot.com \ -I isolateserver.appspot.com \ -d os Android \ -d pool Chrome \ -d gpu none \ -s <hash printed at step 3> Wait for results. OS currently available:
For other available --dimension values, look at Swarming bots e.g.: https://chromium-swarm.appspot.com/botlist. Notes:
FAQI run a task on Swarming and it hangs thereIt is possible that all the bots are currently fully utilized. In this case, the task is in PENDING state.
It seems tedious to list each test data file individually, can I just list src/ ?In theory yes, in practice please don't and keep the list to the strict minimum. The goal is not to run the tests more slowly and having the bots download 20 gb of data. Reasons includes:
Where can I find the .isolated file?The .isolated files are generated when you build the isolation specific version of a target, e.g. out/Debug or out/Release. The isolation target name is just the normal target name with a _run added to the end.
I have an error, is it related to .isolate files?If you have a test that passes locally and fails on some trybots, then it could be related.
This error can be seen when a browser test js file is not found:
TypeError: Cannot read property 'testCaseBodies' of undefined
Where should I file bugs?Swarming specific bugs can be filed on at http://crbug.com in component Infra>Platform>Swarming.
|