For Developers‎ > ‎

Scripted Bisection of Bugs


Summary

A Python script which automates the process of syncing, building, and testing commits for manually reproducible bugs. Use this if bisecting over prebuilt binaries isn't granular enough.

The script works by performing a binary search on the range of commits between the known good and bad revisions specified on the command line.  For each commit in this search space it syncs to that revision, builds and starts the browser and then waits for the user to input whether this revision is good or bad (e.g., by manually visiting a website which triggers the bug). If the pinpointed revision turns out to be a change in Blink, V8, or Skia, the script will attempt to gather a range of revisions for that repository and continue bisecting.

The script is a small wrapper around the performance bisecting script co-opted to enable manual user interaction instead of running a performance test.

Supported Platforms

  • Linux
  • Windows
  • Mac
  • Android
  • ChromeOS (if / when supported by bisect-perf-regressions.py)

Running

  • To run the script:

  • tools/run-bisect-manual-test.py --browser=debug -g 210750 -b 210823 -w ../../bisect_working_dir
    • --browser The browser to build and run (--browser=list will display all the current options)
    • -g The last known good revision
    • -b The first known bad revision
    • -w Working directory to store private copy of depot
The first time this command is run, the full chromium repo will be synced to the  specified working directory which can take some time, but if you reuse the same working directory the script will just use gclient sync and so be much faster.

Gotchas

  • The script uses git under the hood, which means that if you suspect a Blink/V8/Skia roll, be sure that the range you specify includes when the .DEPS.git file was submitted (not just the revision which rolled the .DEPS file).

Interpreting Output

Assuming the bisect succeeds, the culprit revision will be the First Bad Revision (see below), identified by its SHA-1 hash.
Remember that you can get the SVN revision number by running git log with this hash, as in:
git log -1 1e73bb01af4255521f957d3f2a603d179001226b

Sample Output

...
Revision is [(g)ood/(b)ad]: b
RESULT manual_test: manual_test= 0
...
Revision is [(g)ood/(b)ad]: g
RESULT manual_test: manual_test= 1
...
Full results of bisection:
  chromium  1e73bb01af4255521f957d3f2a603d179001226b  0
  chromium  d8bc3ecdc9f5f95de468d3b24ec59b124694ae05  1
  chromium  78d298ce9a459a9f2b48b4d216b54ba3223d3e9c  1
  chromium  b80a2872e75c47b641b6a396ec1290707c1a416d  ?
  chromium  aa38a7c91d7322464a568d19a5ae27d7546702ad  ?
  chromium  257eafb91d1453baef7d929d5ea2fbd61b827d0a  1
  chromium  7f1f63fdadb4383fdaad7cab63e46bcc5210322e  ?
  chromium  663fe04ad401c63e5ae2b93b625b54be666b8002  ?
  chromium  9105a1e099a228599685ee769e82384128464e0e  ?
  chromium  e57644c05e1ba4ceab5dfece6fb12566a10302aa  ?
  chromium  edf48d44a71f3c49a0d0d5e7574c340e843a7791  ?
  chromium  a5a4e99ea504c2ee4f85b7d9cda746fcfcd3bf84  1
Tested commits:
  chromium  1e73bb01af4255521f957d3f2a603d179001226b  {'total: t': 0.0}
  chromium  d8bc3ecdc9f5f95de468d3b24ec59b124694ae05  {'total: t': 1.0}
  chromium  78d298ce9a459a9f2b48b4d216b54ba3223d3e9c  {'total: t': 1.0}
  chromium  257eafb91d1453baef7d929d5ea2fbd61b827d0a  {'total: t': 1.0}
  chromium  a5a4e99ea504c2ee4f85b7d9cda746fcfcd3bf84  {'total: t': 1.0}

Results: Regression may have occurred in range:
  -> First Bad Revision: [1e73bb01af4255521f957d3f2a603d179001226b] [chromium]
  -> Last Good Revision: [d8bc3ecdc9f5f95de468d3b24ec59b124694ae05] [chromium]
  • ? - Build was skipped
  • F - Build failed
  • 0 - Build was deemed "bad"
  • 1 - Build was deemed "good"
  • 'chromium' is the repository of the revision (could also be v8, skia or blink if the script bisected a roll from one of those repositories).

Comments