the Chromium logo

The Chromium Projects

Autotest Graphics Documentation

Keywords: This testing document describes how to run GLBench, TearTest, Piglit, WebGLConformance and other tests and benchmarks on ChromeOS and ChromiumOS devices.

Source code location

Python sources are under src/third_party/autotest/files/client/site_tests

drwxr-xr-x - graphics_GLAPICheck tree | history
drwxr-xr-x - graphics_GLBench tree | history
drwxr-xr-x - graphics_O3DSelenium tree | history
drwxr-xr-x - graphics_SanAngeles tree | history
drwxr-xr-x - graphics_TearTest tree | history
drwxr-xr-x - graphics_WebGLConformance tree | history
drwxr-xr-x - graphics_WindowManagerGraphicsCapture tree | history

Binary dependencies are under src/third_party/autotest/files/client/deps

drwxr-xr-x - glbench-images tree | history
drwxr-xr-x - glbench tree | history
drwxr-xr-x - piglit tree | history

A note on building binaries

The ebuild system knows if a target knows either OpenGL or OpenGL ES, or even both. If only pre-compiled binaries are used there should not be much of a problem with running tests. When editing tests after using the cros_workon script the default system behavior is to recompile the binaries once more without invoking the ebuild system. Unfortunately this means USE="opengles" flags are not honored. Instead the easy way is to specify the proper switch via am environment variable GRAPHICS_BACKEND=OPENGLES./ that gets directly passed to the Makefile. This is somewhat hacky. To use the ebuild system to generate the binaries one has to do a painfully slow USE=opengles emerge-tegra2_seaboard chromeos-base/autotest followed by ./ --use_emerged [...]. Don't forget the --use_emerged flag as it is not default.

Autotest graphics_GLAPICheck


GL_VERSION = 3.2.0 NVIDIA 195.36.24

GL_EXTENSIONS = GL_ARB_color_buffer_float GL_ARB_compatibility GL_ARB_copy_buffer GL_ARB_depth_buffer_float GL_ARB_depth_clamp GL_ARB_depth_texture GL_ARB_draw_buffers GL_ARB_draw_elements_base_vertex [...] NV-GLX NV-CONTROL Generic


SUCCEED: run to the end

Sample output

(cros-chroot) ihf@ql ~/trunk/src/scripts $ GRAPHICS_BACKEND=OPENGL ./ --board=${BOARD} --remote= graphics_GLAPICheck --use_emerged

Initiating first contact with remote host


INFO : Test results:

graphics_GLAPICheck PASS

graphics_GLAPICheck/graphics_GLAPICheck PASS

Total PASS: 2/2 (100%)

No crashes detected during testing.

Elapsed time: 0m25s

Autotest graphics_GLBench

This benchmark executes glbench, a graphics benchmark designed to time how long

various graphic intensive activities take, which includes measuring:

-Z reject rate

-triangle rate

-- deps/glbench/src/checksums

-- can be re-generated with -save option

Sample output

(cros-chroot) ihf@ql ~/trunk/src/scripts $ GRAPHICS_BACKEND=OPENGLES ./ --board=${BOARD} --remote= graphics_GLBench --use_emerged


INFO : Test results:

graphics_GLBench PASS

graphics_GLBench/graphics_GLBench PASS

1280x768_fps_compositing 11.18

1280x768_fps_no_fill_compositing 41.96

mpixels_sec_clear_color 490.74

mpixels_sec_clear_colordepth 323.88

mpixels_sec_clear_colordepthstencil 281.45

mpixels_sec_clear_depth 969.71

mpixels_sec_clear_depthstencil 647.34

mpixels_sec_fill_solid 285.1

mpixels_sec_fill_solid_blended 128.55

mpixels_sec_fill_solid_depth_neq 973.2

mpixels_sec_fill_solid_depth_never 1189.08

mpixels_sec_fill_tex_bilinear 149.56

mpixels_sec_fill_tex_nearest 203.7

mpixels_sec_fill_tex_trilinear_linear_01 164.63

mpixels_sec_fill_tex_trilinear_linear_04 nan

mpixels_sec_fill_tex_trilinear_linear_05 nan

mpixels_sec_fill_tex_trilinear_nearest_05 nan

mpixels_sec_pixel_read 6.72

mpixels_sec_pixel_read_2 6.72

mpixels_sec_pixel_read_3 6.74

mpixels_sec_varyings_shader_1 270.68

mpixels_sec_varyings_shader_2 144.33

mpixels_sec_varyings_shader_4 73.57

mpixels_sec_varyings_shader_8 24.29

mpixels_sec_yuv_shader_1 nan

mpixels_sec_yuv_shader_2 47.82

mpixels_sec_yuv_shader_3 93.36

mpixels_sec_yuv_shader_4 136.15

mtexel_sec_texture_update_teximage2d_1024 149.85

mtexel_sec_texture_update_teximage2d_128 80.92

mtexel_sec_texture_update_teximage2d_1536 146.05

mtexel_sec_texture_update_teximage2d_2048 149.47

mtexel_sec_texture_update_teximage2d_256 132.51

mtexel_sec_texture_update_teximage2d_32 10.29

mtexel_sec_texture_update_teximage2d_512 142.05

mtexel_sec_texture_update_teximage2d_768 147.56

mtexel_sec_texture_update_texsubimage2d_1024 150.94

mtexel_sec_texture_update_texsubimage2d_128 81.59

mtexel_sec_texture_update_texsubimage2d_1536 147.2

mtexel_sec_texture_update_texsubimage2d_2048 149.03

mtexel_sec_texture_update_texsubimage2d_256 130.59

mtexel_sec_texture_update_texsubimage2d_32 10.09

mtexel_sec_texture_update_texsubimage2d_512 113.54

mtexel_sec_texture_update_texsubimage2d_768 150.0

mtri_sec_triangle_setup 30.34

mtri_sec_triangle_setup_all_culled 49.89

mtri_sec_triangle_setup_half_culled 44.91

mvtx_sec_attribute_fetch_shader 149.51

mvtx_sec_attribute_fetch_shader_2_attr 149.34

mvtx_sec_attribute_fetch_shader_4_attr 141.36

mvtx_sec_attribute_fetch_shader_8_attr 106.71

us_swap_swap 19464.19

Total PASS: 2/2 (100%)

No crashes detected during testing.

Elapsed time: 8m58s

>>> Details stored under /tmp/run_remote_tests.purz

The "nan" in this example are passed because of knownbad output images. For more more details and failures look at

(cros-chroot) ihf@ql ~/trunk/src/scripts $ more /tmp/run_remote_tests.purz/graphics_GLBench/graphics_GLBench/summary.txt

Note on HW Qual failures

Sometimes when site_tests/suite_HWQual/ is run a constraint is violated and failure mtri_sec_triangle_setup >= 10 is thrown. This is due to some x86 boards doing vertex shading in the CPU. This exception is considered harmless until a waiver system is developed.

Autotest graphics_O3DSelenium

Autotest graphics_Piglit


./ summary/compare results/baseline.results results/current.results

2011-05-04T11:06:02.995237-07:00 localhost kernel: [ 75.994363] fbo-depth-sampl[4250]: segfault at 0 ip (null) sp 7fc7abac error 14 in card0[75f97000+4000] 2011-05-04T11:06:03.040387-07:00 localhost crash_reporter[4251]: Received crash notification for fbo-depth-sample-compare[4250] sig 11 (developer build - not testing - always dumping) 2011-05-04T11:06:03.234936-07:00 localhost crash_reporter[4251]: Stored minidump to /var/spool/crash/fbo_depth_sample_compare.20110504.110603.4250.dmp 2011-05-04T11:06:03.235566-07:00 localhost crash_reporter[4251]: Leaving core file at /var/spool/crash/fbo_depth_sample_compare.20110504.110603.4250.core due to developer image 2011-05-04T11:06:17.742377-07:00 localhost kernel: [ 90.742080] [drm:i915_hangcheck_elapsed] *ERROR* Hangcheck timer elapsed... GPU hung 2011-05-04T11:06:17.742439-07:00 localhost kernel: [ 90.742100] render error detected, EIR: 0x00000000 2011-05-04T11:06:17.742471-07:00 localhost kernel: [ 90.742168] [drm:i915_do_wait_request] *ERROR* i915_do_wait_request returns -5 (awaiting 29946 at 29941) 2011-05-04T11:06:18.330692-07:00 localhost crash_reporter[4328]: Received crash notification for Xorg[3498] sig 6 (developer build - not testing - always dumping)

Sample output

(cros-chroot) ihf@ql ~/trunk/src/scripts $ ./ --board=${BOARD} --remote= graphics_Piglit --use_emerged


INFO : Test results:

graphics_Piglit PASS

graphics_Piglit/graphics_Piglit PASS

count_subtests_fail 239

count_subtests_pass 1040

count_subtests_skip 687

count_subtests_warn 14

Total PASS: 2/2 (100%)

No crashes detected during testing.

Elapsed time: 13m31s

>>> Details stored under /tmp/run_remote_tests.8gjT

Point browser to


Autotest graphics_SanAngeles

This test runs the San Angeles Observation GPU benchmark. This benchmark uses

a minimal and portable framework to generate a small demo program. It exercises

basic features of OpenGL like vertex arrays, color arrays, and lighting. It

also uses objects generated using procedural algorithms.

This test is a benchmark. It will fail if it fails to complete.

Sample output

(cros-chroot) ihf@ql ~/trunk/src/scripts $ GRAPHICS_BACKEND=OPENGL ./ --board=${BOARD} --remote= graphics_SanAngeles --use_emerged


INFO : Test results:

graphics_SanAngeles PASS

graphics_SanAngeles/graphics_SanAngeles PASS

frames_per_sec_rate_san_angeles 2.1

Total PASS: 2/2 (100%)

No crashes detected during testing.

Elapsed time: 2m17s

>>> Details stored under /tmp/run_remote_tests.vDwN

Autotest graphics_TearTest

scrolling horizontally.

This is a semi-automated test that displays vertical lines scrolling

horizontally and asks the user if tearing was observed. Three variants are


* using uniform update. This tests that glSwapInterval function performs as


* using full texture update. This tests that CPU-GPU interaction is properly

synchronized in the driver.

* using pixmap to texture extension. This tests that pixmap to texture

extension is properly synchronized.

Sample output

(cros-chroot) ihf@ql ~/trunk/src/scripts $ GRAPHICS_BACKEND=OPENGL ./ --board=${BOARD} --remote= graphics_TearTest --use_emerged

[...] - some user interaction

INFO : Test results:

graphics_TearTest PASS

graphics_TearTest/graphics_TearTest PASS

Total PASS: 2/2 (100%)

No crashes detected during testing.

Elapsed time: 1m26s

>>> Details stored under /tmp/run_remote_tests.SHxS

Autotest graphics_WebGLConformance

This test runs the WebGL conformance tests:

Sample output

Doesn't compile any code, so no GRAPHICS_BACKEND needed (but no harm in using it). When the browser doesn't crash/hang it should look like this.

(cros-chroot) ihf@ql ~/trunk/src/scripts $ ./ --board=${BOARD} --remote= graphics_WebGLConformance/control


INFO : Test results:

graphics_WebGLConformance PASS

graphics_WebGLConformance/graphics_WebGLConformance PASS

count_tests_fail 352

count_tests_pass 5064

count_tests_timeout 2

waived_url_000 conformance/tex-image-and-sub-image-2d-with-array-buffer-view.html : 192 failures (192 waived)

waived_url_001 conformance/tex-image-with-format-and-type.html : 12 failures (12 waived)

waived_url_002 conformance/texture-npot.html : 12 failures (12 waived)

waived_url_003 conformance/glsl-conformance.html : 1 failures (1 waived)

waived_url_004 conformance/tex-image-and-sub-image-2d-with-image.html : 8 failures (8 waived)

waived_url_005 conformance/copy-tex-image-and-sub-image-2d.html : 34 failures (34 waived)

waived_url_006 conformance/gl-clear.html : 4 failures (4 waived)

waived_url_007 conformance/more/functions/readPixelsBadArgs.html : 1 failures (1 waived)

waived_url_008 conformance/more/conformance/webGLArrays.html : 1 failures (1 waived)

waived_url_009 conformance/gl-teximage.html : 46 failures (46 waived)

waived_url_010 conformance/texture-active-bind.html : 4 failures (4 waived)

waived_url_011 conformance/read-pixels-test.html : 3 failures (3 waived)

waived_url_012 conformance/gl-object-get-calls.html : 2 failures (2 waived)

waived_url_013 conformance/point-size.html : 1 failures (1 waived)

waived_url_014 conformance/texture-formats-test.html : 4 failures (4 waived)

waived_url_015 conformance/texture-complete.html : 1 failures (1 waived)

waived_url_016 conformance/tex-image-and-sub-image-2d-with-video.html : 8 failures (8 waived)

waived_url_017 conformance/context-lost-restored.html : 2 failures (2 waived)

waived_url_018 conformance/tex-image-and-sub-image-2d-with-image-data.html : 16 failures (16 waived)

Total PASS: 2/2 (100%)

Crashes detected during testing:

chrome sig 6


Total unique crashes: 1

>>> Details stored under /tmp/run_remote_tests.iumX

Autotest graphics_WindowManagerGraphicsCapture

Sample Output

ihf@ql ~/trunk/src/scripts $ GRAPHICS_BACKEND=OPENGL ./ --board=${BOARD} --remote= graphics_WindowManagerGraphicsCapture --use_emerged


INFO : Test results:

graphics_WindowManagerGraphicsCapture PASS

graphics_WindowManagerGraphicsCapture/graphics_WindowManagerGraphicsCapture PASS

Total PASS: 2/2 (100%)

No crashes detected during testing.

Elapsed time: 0m36s

>>> Details stored under /tmp/run_remote_tests.Lb4p