TL;DR Writing an accessibility (a11y) test with axe-core will create 1 test for each rule and test created. This will prevent timeouts. A11y tests DO NOT run in DEBUG builds (also to prevent timeouts), this shouldn't be an issue since we're only testing web UI with axe-core.
Simple Test // Copyright 2019 The Chromium Authors. All rights reserved. // Use of this source code is governed by a BSD-style license that can be // found in the LICENSE file. // Polymer BrowserTest fixture and aXe-core accessibility audit. GEN_INCLUDE([ '//chrome/test/data/webui/a11y/accessibility_test.js', '//chrome/test/data/webui/polymer_browser_test_base.js', ]); TestFixture = class extends PolymerTest {
/** @override */
get browsePreload() { return 'chrome://settings/'; // URL of page to test. }
};
AccessibilityTest.define('TestFixture', { // Must be unique within the test fixture and cannot have spaces.
name: 'TestSuiteName',
// Optional. Configuration for axe-core. Can be used to disable a test.
axeOptions: {},
// Optional. Filter on failures. Use this for individual false positives.
violationFilter: {},
// Optional. Any setup required for all tests. This will run before each one.
setup: function() {
},
tests: {
'Test case description': function() {
// Set up an individual test case. You can open up a dialog or click
// buttons to get the test to the state that should be tested for a11y.
// This will run once per axe-core a11y rule.
// Can return a promise for async setup.
}
},
}); Accessibility audits can take a while to run, so we broke each audit out into its own test. This would be tedious to do by hand so the AccessibilityTest.define function will create them based on some params. This calls TEST_F for each test that's being created and uses a test fixture like all web UI tests. In the past, a11y audit was tacked onto the end of all web UI tests but we want to avoid timeouts since the number of tests in axe-core is much larger than ADT, so a11y tests now have to be created manually-ish. Each test inside the tests data member will be run as its own Mocha test. Example: AccessibilityTest.define('TestFixture', { name: 'TestSuiteName', tests: { 'Test1': function() { // No special setup }, 'Test2': function() { // Show the dialog before testing for a11y. let dialog = document.querySelector('#dialog'); dialog.showModal(); }, }, }); That code snippet will create the following tests:
Each of these tests will have a mocha body that will run both Test1 and Test2, then run the a11y audit on the state of the page after each test setup. Googler only link to design doc. (Doc has a lot of overlap with this page, but goes more into design considerations)
Debugging Axe-Core has an extension that can be used to debug by running the same library that we run the tests with. You must pass in the --extensions-on-chrome-urls flag in order to run these extensions on chrome:// urls. This is a security risk, so only run extensions you trust when doing this. Running Locally The axe-core a11y audit is part of browser_tests. Instructions for building are the same as that target. Note: axe-core tests are NOT part of DEBUG builds to prevent timeouts on test bots. |
|
|