The Chromium Projects

Search this site

Except as otherwise noted, the content of this page is licensed under a Creative Commons Attribution 2.5 license, and examples are licensed under the BSD License.
Chromium OS‎ > ‎Testing Home‎ > ‎

Writing Autotest Tests

Deprecated in favor of this doc.

Writing a simple test

Autotest tests are checked into several locations under third_party/autotest.  They are not upstreamed to the general autotest repository as most are Chromium OS specific.  They are checked out as part of the regular sync command you used from the Chromium OS repository.

There are two flavors of tests: client and server.  All tests are managed by an autotest server machine that is typically the machine where run_remote_tests is invoked.  Client tests execute entirely on the Chromium OS device.  Server tests execute on both server and client or only the server.  Client tests are usually the simplest to write and run.  Server tests are needed, for example, if a test needs to reboot the device or interact with external devices (e.g. to cut off power to the Chromium OS device).

Tests are located in 4 locations in the 
third_party/autotest/files/ 
tree:
  • client/site_tests - These are where most tests live.  These are client tests that are Chromium OS specific.
  • client/tests - These are client tests that are general Linux tests (not Chromium OS specific).
  • server/site_tests - These are server tests that are Chromium OS specific.
  • server/tests - These are server tests that are general Linux tests (not Chromium OS specific).
Decide if your test is a client or server test and choose the appropriate directory from the above.  In the sections below we refer to this directory as ${TEST_ROOT} below.

Next decide which area your test falls within based on the tracker (http://code.google.com/p/chromium-os/issues/list).  It should be something like "desktopui", "platform", or "network" for instance.  This name is used to create the test name; e.g. "network_UnplugCable".  Create a directory for your test at $(TEST_ROOT)/$(LOWERCASE_AREA)_$(TEST_NAME).

Try to find an example test that does something similar and copy it.  You will create at least 2 files:

${TEST_ROOT}/${LOWERCASE_AREA}_${TEST_NAME}/control
${TEST_ROOT}/${LOWERCASE_AREA}_${TEST_NAME}/${LOWERCASE_AREA}_${TEST_NAME}.py

Your control file runs the test and sets up default parameters.  The .py file is the actual implementation of the test.  
Inside the control the TEST_CLASS variable should be set to ${LOWERCASE_AREA}.  The naming convention simply exists to make it easier to find other similar tests and measure the coverage in different areas of Chromium OS.  
You may also want to add this test to one or more existing test suites to have it run automatically.

After this, you should modify the autotest-tests-9999.ebuild under src/third_party/chromiumos-overlay/chromeos-base/autotest-tests and add your test to IUSE_TESTS or it won't be picked up by autotest when you ask it to build specific tests.

For more information on writing your test, see the frequently asked questions.

Adding binaries for your tests to call as part of the test (e.g. test suites, etc.)

In order to cross-compile, your test's compilation step should be implemented inside the setup() method of your python code. A couple of simple examples:
  • Sources inside the autotest repo as part of the test: gl_Bench
  • Sources checked in as a tarball from upstream: hardware_SAT
  • Sources checked in other Chromium OS source repo: firmware_VbootCrypto, desktopui_DoLogin
It is your responsibility to make sure the test will build for all supported platforms as it will cause a build break if it does not.

The setup method of all python scripts are built as part of build_autotest.sh (which is called when you build_packages --withautotest).  This script calls all the setup functions of every python test and runs them.  These setup steps are compiled on the host for execution on the target.  The cross-compiler flags are already set so make sure if you have your own Makefile to take in the CC, CFLAGS, etc from the environment rather than hardcode them.

Note that if you have a setup() method, it should create a src directory, even if empty, to avoid running the setup method on the target device.

Writing tests that require that a user is logged in

If you write a test that requires a user is logged in, have your test subclass site_ui_test.UITest.  Any test that is a subclass of this class will login with the default test account as part
of test initialization and logout as part of test cleanup.