1. Configuring/setting up the buildbot instances

Foreword

  • You can run both a master and a slave on the same computer. On Windows and Mac this may be problematic as many tests pops up Chromium windows while testing. This is not an issue with Linux since it uses xvfb. So testing on linux is simpler.
  • I know this is dense but please reads the relevant instructions carefully. Missing only one step will break the whole thing.
  • Please update the doc for missing/unclear parts.
  • Additional docs for the buildbot are available at buildbot.net.  A good starting point is here: http://buildbot.net/buildbot/docs/current/tutorial/.

Buildbot Master (server)

  1. Create build/site_config/.bot_password text file with a password on one line for your slave to serve to be able to connect to the master.
    echo Bleh > build/site_config/.bot_password
  2. There are configurations for various types of masters each in a well-named directory.  For example if you're just learning (aka experimenting) you should probably use master.experimental.
    • master.cfg contains all the master-specific settings. In theory you shouldn't have to change anything. If it fails or throw exceptions, send an email to chromium-dev@chromium.org.
    • slaves.cfg lists the slaves you will connect.  Again, you probably won't need to change anything.
  3. To run tests on a slave in the experimental configuration, you'll need to add them to the m.ChromiumFactory() calls in the FACTORIES section of master.cfg. Alternatively, to speed up testing on other configurations, you can comment out some of the tests listed there. See _AddTests() in scripts/master/factory/chromium_factory.py for test identifiers.
  4. For production masters only:
    1. Create scripts/common/.status_password with the relevant password only if you want to update LKGR.

Buildbot Slave

Quick HowTo

  1. Note: slave/buildbot.tac nows contains 3 environment variables to override which master and how it's connecting so no need to edit it anymore.
  2. Find the master name you are using. The name "config.Master.*" must match one of the classes in site_config/config_default.py. See the corresponding ActiveMaster in master.cfg
  3. Find the slavename you want to emulate for testing. Do one of:
    1. Open master.*/slaves.cfg and pick any slave name that is on the right builder you want to test on.
    2. Start the master, browse to the internal webpage, click on the desired builder page you want your slave to connect to. (see instructions on the next page, run your own buildbot, for getting to the internal web page.)
  4. If you haven't done a gclient checkout on the slave then you'll need to checkout depot_tools inside your root checkout under build/.
    1. "Update scripts" will fail if you don't run the slave from a gclient checkout. "Update" will fail if ./depot_tools is not present at least on Linux.
    2. svn co http://src.chromium.org/chrome/trunk/tools/depot_tools

In the end, the command you'll run in the run your own buildbot step will look like this:

TESTING_MASTER_HOST=localhost TESTING_MASTER=TryServer TESTING_SLAVENAME=vm384-m4 make restart

This starts a linux try slave connecting to a master running on the same machine, using the master from the master.TryServer directory, and the slave from the master.cfg file in the master directory you have chosen. Note that if you edit the buildbot.tac file, you can leave off some of these environment variables and just do "make restart".

General

  1. You may want to fix slave/info/host and slave/info/admin too; these are shown in the build slave properties.

Windows

  1. Install everything listed here.
    1. Make sure you can build.
  2. For try slaves only, copy patch.exe in %WINDIR%system32  (%WINDIR%\SysNative and %WINDIR%\SysWow32 on x64)
    1. Because UAC triggers on patch.exe, you need to disable UAC for patch.exe to work on Vista. We may simply rewrite it in python instead.
  3. Install the b/slave/ssl/ssl_ui_test_root_ca.crt certificate.
  4. Install a printer and set it as default. XPS printer is not OK.
  5. For layout tests;
    1. Install Safari and QuickTime
    2. Install Java
    3. Set the theme to Luna (the default)
  6. Do not RDP for slave running interactive_ui_tests

Linux

  1. Install everything listed here.
  2. Set '[auth]\npassword-stores = \n' in ~/.subversion/config.  (To keep SVN from using the Gnome Keyring to store your SVN credentials.)
  3. Make sure you can build.
  4. For try slaves
    1. "sudo visudo to add %admin ALL = NOPASSWD: /sbin/shutdown" - this lets the tests shut down or restart the try slave
  5. If you want to use valgrind on the slave, you need to do some extra configurations steps on the master side. See building valgrind docs.
  6. Chromium OS requires git >= 1.7.0
    1. Use the official git-core PPA at https://launchpad.net/~git-core/+archive/ppa
  7. sudo apt-get install xvfb icewm

Linux ARM variant

 The ARM toolchain requires a different gcc, and a near tip-of-tree version of gold to work correctly. Alternate instructions can be found on the wiki.
  1. Do everything for the Linux setup above.
  2. On each try slave:
    1. Install The CodeSourcery toolchain into /usr/local/crosstool-trusted
      1. wget http://www.codesourcery.com/sgpp/lite/arm/portal/package5383/public/arm-none-linux-gnueabi/arm-2009q3-67-arm-none-linux-gnueabi-i686-pc-linux-gnu.tar.bz2
      2. sudo mkdir /usr/local/crosstool-trusted
      3. sudo tar -C /usr/local/crosstool-trusted -xvjf arm-2009q3-67-arm-none-linux-gnueabi-i686-pc-linux-gnu.tar.bz2
    2. Install the arm rootfs
      1. <how to get the latest rootfs and set the crosstool directory??>
      2. sudo apt-get install rootstock
    3. Install gold for tip-of-tree binutils
      1. Grab a cvs snapshot of binutils (probably needs to be done off the buildbot).
        1. cvs -z 9 -d :pserver:anoncvs@sourceware.org:/cvs/src login [password is anoncvs]
        2. cvs -z 9 -d :pserver:anoncvs@sourceware.org:/cvs/src co binutils
      2. Copy that onto the buildbot.
        1. tar cjf src binutils-cvs-MMM-DD-YYYY.tar.bz2 src
        2. scp binutils-cvs-MMM-DD-YYYY.tar.bz2 user@buildbot.dns.name:
        3. tar -C binutils-cvs-MMM-DD-YYYY -xjf binutils-cvs-MMM-DD-YYYY.tar.bz2
      3. Configure and build gold.
        1. cd binutils-cvs-MMM-DD-YYYY/src
        2. ./configure --host=i686-unknown-linux-gnu --build=i686-unknown-linux-gnu --target=arm-none-linux-gnueabi --enable-shared --enable-multilib --with-sysroot=/usr/local/arm-rootfs --prefix=/usr/local/tot-binutils --enable-gold --disable-gdb
        3. make maybe-all-gold
        4. sudo make maybe-install-gold
    4. Modify compile.py to use our version of gold.
      1. open /b/scripts/slave/compile.py in an editor
      2. Change the lines os.environ['CC'] = crosstool + '-gcc' and os.environ['CXX'] = crosstool + '-g++' to 
        1. os.environ['CC'] = crosstool + '-gcc -B/usr/local/tot-binutils/arm-none-linux-gnueabi/bin'
        2. os.environ['CXX'] = crosstool + '-g++ -B/usr/local/tot-binutils/arm-none-linux-gnueabi/bin'
    5. <insert how to set the crosstool environment variable in the first place. >

ChromiumOS

In addition to Linux or Linux ARM, you need to:
  1. sudo visudo -- Add $USER<tab>ALL:NOPASSWD to the end
  2. Copy thirdpartymirrors to ~

Mac

  1. Install everything listed here under "Prerequisites".
  2. Necessary for some tests. Install when you see tests failing:
    1. Install Safari and QuickTime
  3. For try slaves
    1. sudo visudo to add %admin ALL = NOPASSWD: /sbin/shutdown
  4. If you want to use valgrind on the slave, you need to do some extra configurations steps on the master side. See building valgrind docs.

Running a buildbot instance

It's now time to run your own buildbot.
Comments