Contributing

Writing a New Test

To write a new Chrome Endure test, you can just copy the general format of the existing Chrome Endure tests in perf_endure.py, but keep the following important issues in mind while writing a new test:

  1. Before you begin writing a new test, make sure you know how to run an existing Chrome Endure test locally by following the setup instructions.
  2. Decide where you want to put your new test.  The easiest thing would be to create a new class inside of perf_endure.py, but it's also possible to create a new file to contain your test.
  3. Ensure that the class containing your new test inherits from class ChromeEndureBaseTest defined in perf_endure.py.  That base class implements common functionality used by all of the Chrome Endure tests.
  4. Start writing the function that implements your test.
    • In general, your test will need to use pyauto function calls to automate the functionality of the browser, allowing you to do things like open tabs and navigate them to particular URLs (pyauto guarantees that at the start of your test, Chromium will already be launched under a clean profile, and there will be a single tab in the browser navigated to about:blank).  Pyauto function calls can be made using the self object, since all Chrome Endure tests (and all other pyauto tests) ultimately inherit from class PyUITest in pyauto.py, which contains most of the pyauto functions available to pyauto tests.  For example, to navigate the default tab from about:blank to gmail.com in your test, you do:
      self.NavigateToURL('http://www.gmail.com')
    • To automate the webapps themselves, it's easiest to use chromedriver, which is available to pyauto tests and therefore also to Chrome Endure tests.  You can construct a chromedriver object with this statement:
      self.NewWebDriver()
      See the existing tests in perf_endure.py for examples of how to use chromedriver to interact with webapps.
  5. Ensure your test measures performance statistics when necessary, and outputs results in a way suitable for your needs.
    • The Chrome Endure tests all use function _GetPerformanceStats, defined in class ChromeEndureBaseTest, to measure and output a set of performance/memory values.  This function is invoked periodically by the Chrome Endure tests to take a current "snapshot" of the performance/memory characteristics while a test is running.  The function measures a particular set of values, then outputs the measured results in a format that can be graphed using the Chrome Endure graphs.
    • If you want to customize the set of measurements taken and/or the way in which the results are outputted, you can implement the custom functionality within your test, instead of calling function _GetPerformanceStats.  If you want to customize the measurements or output, yet still see Chrome Endure performance graphs, make your test invokes function _OutputPerfGraphValue, defined in perf.py.  That function takes care of outputting results in the appropriate format to be graphed.
  6. Run your new test.  To do so, follow the example shown in Step 4 of the setup instructions, except replace the name of the test with that of your new test.  The test name should be of the form file_name.class_name.function_name.  For example, if you created a new class called MyChromeEndureTest inside of file perf_endure.py, and implemented your test inside of that class in a function called testMyWebapp, then the full name of your test would be: perf_endure.MyChromeEndureTest.testMyWebapp.
  7. Analyze the results, depending on how you implemented your test.  If you followed the same format as the other Chrome Endure tests, and are periodically invoking _GetPerformanceStats and are using the Chrome Endure graphs, then you can actually view the graphs on-the-fly while the test is still running.  Every time _GetPerformanceStats is invoked, it immediately outputs the current set of known results for graphing purposes.  This is very useful for tests that may take many hours to complete.  You probably don't want to have to wait until the test is completely finished before seeing any results.
Comments