Timechart how-to

Developer builds contain a performance analysis command called perf that can be used to create an SVG output file similar to bootchart; the chart shows how CPU cycles and I/O wait times are distributed across processes in the system over time.

Below is a short primer in four lessons describing how to generate and view output from perf timechart.

Lesson 1 - a simple example

  1. Boot Chromium OS, and open a terminal.
  2. Run this command:
    sudo perf timechart record
  3. Run your workload.  A workload isn't necessary if all you want to see is a chart of an idle system.  :-)
  4. When your workload is done, interrupt the process started in step 2 using ^C, or kill -2.
Explanation:  Without arguments, perf timechart record runs forever gathering data, until stopped by SIGINT.  Note that only SIGINT works; SIGTERM will kill the process without producing the necessary output.  When the command completes, you'll see two new files:  perf.data and trace.out.

Lesson 2 - how to generate and view the chart

  1. In the directory where you ran Lesson 1, run this command:
    sudo perf timechart
  2. The output image will be stored in a file named output.svg. Use scp or some equivalent to copy the file to another system for viewing.
Tips for viewing:  Some browsers may have trouble displaying the image.  The author of timechart recommends the Inkscape image editor:
Inkscape does a good job of displaying the fine details, but it may be a bit slow for the large timechart images.  You should exercise patience when opening, magnifying, or scroling images.

Lesson 3 - how to avoid using SIGINT

  1. Run this command:  
    sudo perf timechart record sleep 5
  2. Run a workload that will finish within 5 seconds; for longer workloads, use a more appropriate sleep time in step 1.
  3. Generate and view the output as described in Lesson 2.
Explanation: If there are arguments to perf timechart record, the arguments are treated as a command to run as a subprocess of perf. perf gathers data until the process terminates.

If your workload is triggered by a single command, that command can be used in place of 'sleep 5'. Note that if the workload acts as a daemon (that is, forks a child and exits), perf will terminate with the parent terminates; this likely isn't what you'd want.

Lesson 4 - how to get a timechart of system boot

  1. Add the lines below to /sbin/chromeos_startup (somewhere after the line that mounts /sys/kernel/debug). This file can be found as src/platform/init/chromeos_startup in your source repository.  A patch to apply this change is attached at the bottom of this page. 
    ( cd /tmp ; exec perf timechart record sleep 10 >perf.out 2>&1 ) &
    sleep .2
  2. Add the line below to /etc/init/boot-complete.conf.  This file can be found as src/platform/init/boot-complete.conf in your source repository.  A patch to apply this change is attached at the bottom of this page. 
    pkill -2 '^perf$' || true
  3. Boot your system with the changes applied.
  4. Open a terminal, and run this command: 
    cd /tmp ; sudo perf timechart
    After doing this, you can copy the output.svg file for viewing.
Explanation:  This uses the trick from lesson 3, and applies it to early boot time.  The sleep .2 command is a hack needed to allow perf to finish initializing before startup scripts that load kernel modules have a chance to run.  Without the hack, perf frequently fails to come up.  To prevent gathering a full 10 second chart, the boot-complete upstart job kills perf as in lesson 2.
Richard Barnette,
Sep 22, 2010, 2:19 PM