For Developers‎ > ‎

Telemetry: Introduction

Telemetry: Introduction to the Performance Testing Framework


Telemetry 101

Telemetry has become the framework for cross-(chrome)-platform performance testing.  Telemetry is a python wrapper around Chrome's DevTools Remote Debugging Protocol.  Telemetry abstracts launching, connecting, and opening a page on whatever underlying platform Chrome is running, fetching data via the Inspector timeline and traces, loading Chrome with arbitrary flags, and using Web Page Replay to cache real-world websites so they don’t change when used in benchmarks.


Core Principles

  • Write one performance test that can trivially run on all platforms [Mac/Win/Linux, CrOS, Android] for both Chrome and ContentShell

  • Run on browser binaries, without having to build the browser yourself

  • Use WebPageReplay to get repeatable test results

  • Run on non-chrome browsers for comparative studies (not yet implemented)

  • Clean architecture for writing benchmarks that keeps measurements and use cases separate


Should my test use Telemetry?

It depends, what are you trying to measure?
Are you trying to verify correctness?  If yes, Telemetry is not for you but
browser tests are your friend.

Telemetry is designed for performance tests that:

  • can be written with pure JS/HTML/CSS

  • use real world content (rather than synthetic)

  • could be run on other browsers, not just Chrome


The Code

The Telemetry framework lives in src/tools/telemetry and performance benchmarks live in src/tools/perf


Telemetry Benchmarks

Telemetry is opinionated in what a benchmark is.  A benchmark is a measurement, a page set, and a set of optional browser options.  Measurements should work with any and all page sets.  We do this for consistency and to prevent benchmark rot.


Measurement: given a page, a measurement should evaluate the page’s performance.


Page set: a collection of pages (either links to live sites, or a WebPageReplay archive)

  • A product level idea

  • Examples: top 25 websites, international sites with right to left text, tough scrolling sites


My Next Step With Telemetry


Contact Us or Follow Along

If you have questions, please email telemetry@chromium.org

You can keep up with Telemetry related discussions and code reviews by joining the telemetry group


Frequently Asked Questions

How do I navigate Chrome to a certain URL?

To visit a URL, all you need to do is add the URL to a page set.  For more information on maintaining page sets, see managing page sets.


I get an error when I try to use recorded page sets?

Currently, you need to have src-internal checked out to access recorded page sets.  We are in the process of moving these files out of src-internal so anybody can run them.  For now, you can run on live sites or record your own copies.