4. Maintain your Buildbot infrastructure

Mass execution

When you need to revert something on all slaves or something similar, you can use the slaves.py script. From a buildbot checkout, you can run:
./scripts/tools/slaves.py --config c -L "cd /path/to/buildbot && ./depot_tools/gclient sync && sudo shutdown -r now"

See ./scripts/tools/slaves.py --help for more infos on command execution, file transfert and slave filtering, e.g. only run on the x64 slaves or only for one builder, etc.


Manhole permits to ssh into the buildbot master process and examine the global variables. 2 default variables are defined: status and master. You can use dir(status) and master.services to move around the variables to see what's loaded and modify the buildbot state dynamically. To enable it, add a file named .manhole beside master.cfg. For more detail, search for manhole in the code at master_utils.py.

Another example:
import pprint

Also see

Tips and tricks unrelated to maintenance.