WebGL Conformance Tests

An effort to port Khronos's conformance tests to the GPU build bots. This allows developers to immediately find the revision that broke WebGL conformance.

How to Check Conformance

All bots on the GPU waterfall run the WebGL conformance suite under the step named, creatively enough, webgl_conformance_test. The bots are expected to generally stay green, so it should be easy to spot failures.

The goal is to keep webgl_conformance_test green. If the test is red, you can click through on individual tests for more information.

Run the Tests Locally

After building the chrome run: 

content/test/gpu/run_gpu_test webgl_conformance

Currently this defaults to running only tests in the 1.0.1 suite. You can specify a different version using the --webgl-conformance-version flag:

content/test/gpu/run_gpu_test webgl_conformance --webgl-conformance-version=1.0.3

If you only want to run a single test or subset of tests, pass a partial test name into --page-filter flag. For example, if you only wanted to run the tests related to index validation you would call:

content/test/gpu/run_gpu_test webgl_conformance --page-filter=index-validation

Test Expectations

content/test/gpu/gpu_tests/webgl_conformance_expectations.py contains the list of all known conformance test failures. See the comments at the top of the expectations file for a list of valid conditions. 

Format

self.{Fail|Skip}('test/url.html', ['array', 'of', 'conditions'], bug='optional crbug number')

Example

self.Fail('conformance/textures/texture-size.html', ['win', 'intel'], bug=121139)

Updating the Conformance Tests

Since the tests rely on an external repository, the local version must be updated on any change upstream. Right now, this is manual work (and blocked by Khronos's poor servers).

See the GPU Pixel Wrangling instructions, section "Updating the WebGL conformance tests", for detailed instructions on updating the snapshot of the WebGL conformance tests in the Chromium repository.
Comments