Try Server usage

Quick links

Other sites

Except as otherwise noted, the content of this page is licensed under a Creative Commons Attribution 2.5 license, and examples are licensed under the BSD License.

For Developers‎ > ‎Testing and infrastructure‎ > ‎

Overview

The try server lets committers try uncommitted patches on multiple platform in an automated way.

Contents

1 Overview
2 Workflow
3 How to specify a subset of tests to run
1. 3.1 Examples
4 Specialized chromium-specific slaves
1. 4.1 Layout tests
2. 4.2 Valgrind
5 LKGR
6 How does LKGR relate to Tip of Tree or HEAD?
7 Troubleshooting failed try jobs
8 FAQ

The try server is a standard buildbot plus a set of slaves, currently Linux (Ubuntu), Mac, and Windows.
The try server can be used with gcl try and git-try, both are front ends for trychange.py.
It can also be used via the web interface by clicking on the "Try more bots" link in Rietveld.
Any committer can use the try server.

Non-committers with read-only SVN accounts may also run try jobs. Request a read-only account at Chromium access (Google chrome svn) by filling in the request form.
External contributors can ask a committer to try the job for them and have the result email directly into their inbox.
If you do not have sufficient permissions, attempts to use try servers (such as via the web interface) will result in immediate failure.

File limitations are:

Patches must be less than 20MB.
Binary files are not supported.
Text files with CRLF will have issues too.

Try server waterfall

You can limit the builds shown by e.g. time or committer; see waterfall help.

See the Design doc: Try server.

Workflow

The developer explicitly requests a job to be run with git try, git cl try, or gcl try.
The try server will schedule one job per platform that you can follow live on the waterfall.
The job revision will default to http://chromium-status.appspot.com/lkgr. This can be overridden with the tools with --revision.
On each job end

An email will be sent to the email specified in the job. It is usually specified implicitly.
The corresponding review on http://codereview.chromium.org/ will have its try job status updated live.

On gcl commit / git cl dcommit (Not always), a PRESUBMIT.py will enforce that the job succeeded.

How to specify a subset of tests to run

-b, --bot to specify the bot

-b, --bot is additive, except that if you don't specify it at all you'll get default bots

Allowed uses:

Specify multiple builders using the default tests:

-b buildername,buildername,buildername

or specify multiple different test steps:

-b buildername:some_unittest,some_unittest,some_unittest

or specify a --gtest_filter:

-b buildername:some_unittest:GTestFixture.GTestTest

-t, --testfilter

-t, --testfilter is multiplicative, but if you don't specify it at all you'll get all tests

Allowed uses:

Specify different test steps to run :

-t some_unittest,some_unittest

Specify a --gtest_filter

-t some_unittest:GTestFixture.GTestTest

Examples

Runs the default tests on 'win_rel' but only compile on 'win':

git cl try -b win_rel -b win:compile

Compiles on both 'win_rel' and 'win' without running tests:

git cl try -b win_rel,win -t compile

Runs base_unittests on win_rel but no other test:

git cl try -b win_rel:base_unittests

Runs a single specific test from base_unittests on win_rel:

git cl try -b win_rel:base_unittests:SpecificTestName

Runs base_unittests and in addition default tests on win_rel:

git cl try -b win_rel -b win_rel:base_unittests

What are the "defaulttests", you ask? Easiest answer is just to run a try job and look what is skipped. You can also search for the non_default factory property.

Specialized chromium-specific slaves

Layout tests

Layout tests are now optionally run on normal try builders. Since the tests are not run automatically anymore, you need to specify the exact tests to run:

gcl try foo -t webkit_tests,webkit_lint

Valgrind

We have linux_valgrind and linux_tsan slaves to run popular valgrind tests.

git cl try foo --bot linux_valgrind

ui_tests are not executed on linux_valgrind trybot but you could execute selected tests manually:

git cl try your_cl -b linux_valgrind -t ui_tests:YourTest.Foo

e.g.

git cl try test -b linux_valgrind -t ui_tests:HostRulesTest.*

Please note that you can run tests locally on Linux and Mac with your usual Debug built binaries, so you can save some time by running Valgrind locally on these platforms.
See Using Valgrind for the details.

LKGR

The url for the current LKGR is: http://chromium-status.appspot.com/lkgr.

The last known good revision is determined by the last revision that passes green at: http://chromium-status.appspot.com/revisions

The list of tests and builders that need to pass are listed at masters/master.chromium.lkgr/lkgr_finder.py

How does LKGR relate to Tip of Tree or HEAD?

LKGR is by definition behind ToT because it needs to be built and tested, and most likely another commit occured before a revision was vetted for LKGR.

Troubleshooting failed try jobs

Please, before contacting maruel, please look at these items:

Did it fail at the Update step?

Did your patch contain CRLF files? You can look at the 'patch' link in the 'update' step.

Too bad for you. Create a separate commit to fix the file in the tree and try again with --revision 123

Did you get GYP failures in the update 'stdio'?

Maybe you screwed the GYP file.

Did you get "HUNK", "fuzzing" or "offset" messages in the update 'stdio'?

The files you are modifying have been modified on the tree. Maybe LKGR is more recent or older than your checkout. You can look at the build link in the email in the Build Properties section, search for the number beside revision. Maybe try specifying --revision on your try.

Did it fail at the Compile step?

Maybe your code is broken?

Yes, that is a possibility. Sorry 'bout that.

Maybe someone else broke the slave?

That happens. Especially on Windows. Please try again and alert me with the build status url. You can use the --bot win flag to not try needlessly on other platforms.

Maybe a clobber is required. There are command-line options to request clobber (most likely just -c).

Did it fail an unrelated test?

Maybe it was broken at that revision on the main waterfall. Please look up the revision and take a look on the main waterfall.
Maybe the tree slave is broken?

That happens. Please try again and alert me with the build status url.

I get no email!

Did you get a warning message when sending your job? By default, it sends it to your checkout credential.

'gcl try' fails

Send an email to <tryserver at chromium dot 0rg>.

The try server is soo slow

When a clobber is needed, a full rebuild takes time.
Maybe someone changed a GYP and forgot to update the svn:ignore?

If you feel like getting a peer bonus, please fix the property accordingly, otherwise ping <tryserver at chromium dot 0rg>.

The try server needs a clobber

That happens, usually ask to someone knowledgeable on irc to fix it for you or ping maruel if no answer.
You can also request a clobber using command-line flags (usually -c).

error message: 'hostname nor servname provided, or not known'

If you're using the try server from outside of Google, it will fail to look up the hostname and should then fall back to using svn to enqueue the try job. If svn fails, unfortunately, you'll only see the error message about the hostname, so you won't realize that svn is the problem.
To fix, try this first: svn ls svn://svn.chromium.org/chrome-try
If that last step didn't work, maybe you're using git try on Windows? In that case, you may have a mismatch where the tools are using a mix of depot_tools svn and cygwin svn. In that case, also type /usr/bin/svn ls svn://svn.chromium.org/chrome-try

Anything else?

Please update this guide.

FAQ

I'm modifying a DEPS, will it work?

Yes.

I'm modifying a GYP file, will it work?

Yes.

I'm modifying gyp scripts under src/tools/gyp, will it work?

Yes, but you have to use --no_search, otherwise patch will fail saying it cannot find files to patch.

Get detailed command-line help

Use: git try -h (git will redirect --help to man which fails for git-try)

Or: trychange.py --help

Disable automatic try on gcl upload

To do this for 'gcl try': create a file at '~/.gcl_upload_no_try'

For 'git try' : Edit src/codereview.settings thusly: "TRY_ON_UPLOAD: False"

I have an awesome patch for depot_tools!

Use the same way to contribute code that for chromium in general, e.g. gcl or git-cl.

Skip running some/all of the unit tests?

gcl try & git try accept --testfilter testname:filter-string [-t for short] parameter, this option can be repeated to specify additional tests. For instance:

git try -t unit_tests:SafeBrowsingDatabaseTest.*

If you specify a test filter then only the tests binaries you filter on will run (the rest will be skipped). This means you could send a compile only, no test, try job by passing -t none. You can also use:

git cl try --bot win_rel:unit_tests

Want to compile something not compiled by default

Add the pseudo test 'compile' as a test filer, e.g.

git cl try -b win_rel:compile

How can I see what will happen before I submit the job?

Use the --dry_run flag (which implies --verbose) to (a) see what your patch is and (b) not actually submit the job when using git try or gcl try.

apply_issue failed

By default the trybot will patch your change against the LKGR, so there might be differences in the files you are modifying between the revision you've been working on and the LKGR. The easiest way to fix this is check which revision you were using, and then pass it to the try job with --revision.

For example, if your change depends on some other changes newer than LKGR, you may want to try the patch against HEAD, i.e., git cl try -rHEAD.

If this still doesn't work and:

you are using git
master is not upstream of the branch you are trying

then its possible there are uncommitted changes between your branch and master, causing the apply_issue to fail. In this case, use git try instead of git cl try as follows:

git try --upstream_branch=origin/master

Keep in mind that this means your branch cannot be committed until all upstream branches are committed first, even if the trybots succeed.

compiling failed

Possible issues:

LKGR was broken for a specific build configuration that is not taken in account on LKGR, often the case. Try with -r, --revision with one that is working on the main waterfall.
Incremental build failure, slightly less probable but still happens occasionally. Try with -c, --clobber
The slave has a broken checkout and needs a complete checkout removal, rarely happens but still does. Reply to the try job email.
You patch is broken. Please don't completely rule this possibility out yet. :)

My patch includes updated webkit baselines (binary files) and it's not working

Correct! The try servers do not currently support binary patches, but this is being worked on. Star https://crbug.com/23608.

I want to cancel my job, should I press the 'Stop' button?

No! DON'T EVER DO THAT. This button is for maintainers and is quite tricky to use correctly. Just let it go.

Why! you ask. You may stop it during the update step and leave the svn checkout locked, breaking the following tries. You may stop it during compilation, corrupting the PDB database, breaking the following jobs. You may stop it during ui_tests, leaving zombies around. You may stop it during unit_tests, leaving temp files around.

Run the try job only on one platform

Use --bot win or whatever 'builder' name you want to use from the waterfall.

My patch is in Blink only

We highly recommend you to use git. See http://code.google.com/p/chromium/wiki/UsingWebKitGit.

GIT

Go in src/third_party/WebKit and use git-try. If you want to diff against a specific branch, use git-try --upstream_branch=<branch>.

cd src/third_party/WebKit ; git-try --bot mac_layout,win_layout,linux_layout

GCL

Since gcl try will fail if you don't specify a chromium change, you can use trychange.py directly. Use it from the src directory of your Chromium checkout:

trychange.py --sub_rep=third_party/WebKit --bot mac_layout,win_layout,linux_layout

See trychange.py --help for more details.

Simultaneously patch code in the WebKit and Chromium repositories

GIT

Use --sub_rep. Don't know what --sub_rep is? Then try git-try --help.

cd src ; git-try --sub_rep third_party/WebKit/ --bot mac_layout,win_layout,linux_layout

GCL

Use --sub_rep. Don't know what --sub_rep is? Then try gcl help try.

Submit a try job for another person (e.g. non-committer)

Use Rietvield WebUI with the "Try more bots" link. If you need to use specific flags, use:

trychange.py -R codereview.chromium.org/12345 --email patchauthor@domain.com --name issue_12345 --root src -p 0

Where 12345 and patchauthor@domain.com should be replaced accordingly. The argument -p X is equivalent to an argument to "patch -pX", so you may need to change "0" to different values depending on how the uploaded patch was created. Note that you need to be in the Chromium src/ directory for this command to work.

I need to hack on a try slave for a few hours?

Sorry, only Googlers can do this for now. You can get more information at the internal link.

What is the difference between 'git try' and 'git cl try'?

git try runs the try job on your local changes. This causes a diff file to be committed in a special repository. This file is read by the Try Server and the slave applies it during the update step.
git cl try runs the try job on your changes already uploaded to the codereview tool. The Try Server instructs the slave to download the patch by itself in the apply_issue step.

Subpages (1): Design doc: Try server

Comments