The Chromium Projects

Search this site

Except as otherwise noted, the content of this page is licensed under a Creative Commons Attribution 2.5 license, and examples are licensed under the BSD License.
For Developers‎ > ‎

WebKit development workflow

Note: This page is deprecated thanks to the introduction of the Blink project. Please see the Blink Conversion Cheatsheet for the most up-to-date instructions.

This page describes a possible workflow for developing the Chromium WebKit port on Linux or Mac using git and ninja.

Setup

Check out the code:

git clone git://git.webkit.org/WebKit.git WebKit

You can also use the WebKit checked out inside an existing git Chromium tree, whose origin is https://chromium.googlesource.com/external/WebKit_trimmed.git

Configure git:

cd WebKit
# cd third_party/WebKit # if within a Chromium checkout
git svn init --prefix=origin/ -T trunk http://svn.webkit.org/repository/webkit
git config --replace svn-remote.svn.fetch trunk:refs/remotes/origin/master
git svn fetch
git config --global user.name "Foo Bar"
git config --global user.email "foo@chromium.org"
git config merge.changelog.driver "perl $(pwd)/Tools/Scripts/resolve-ChangeLogs --merge-driver %O %A %B"
git config diff.objcpp.xfuncname "^[-+@a-zA-Z_].*$"
git config diff.objcppheader.xfuncname "^[@a-zA-Z_].*$"
git config branch.autosetupmerge always

Login

For login, the scripts will automatically use your system keychain (GNOME Keyring on Linux, Keychain on Mac OS X), which is most secure, though it can be fiddly.
On Linux, first running webkit-patch without anything stored in git configuration and logging in manually stores username and password in the keyring, and then adding username only to git configuration gives automatic sign-in (if you initially log in with username set in git, it won't store all the information correctly and you'll still need to key in username).

webkit-patch upload # may need an actual patch/git commit
# enter credentials, cancel upload
git config bugzilla.username "foo@chromium.org"
webkit-patch upload # test that you don't need to sign in

Alternatively, you can store your password in plain text, which is easier, but insecure:

# git config bugzilla.password "yourpassword" # insecure, are you really sure you want to do this

Environment variables

Next we setup some environment variables. You can put these in ~/.bashrc or create a separate file such as ~/webkit.env to set the variables needed for WebKit development

export WEBKIT_DIR=$(pwd)
export PATH=$PATH:$WEBKIT_DIR/Tools/Scripts:$WEBKIT_DIR/Source/WebKit/chromium/depot_tools
export GYP_GENERATORS=ninja

If you also want to use goma on Mac, you'll also need

export CC=clang
export CXX=clang++
export PATH=$PATH:$WEBKIT_DIR/Source/WebKit/chromium/third_party/llvm-build/Release+Asserts/bin

On Linux goma only requires the usual goma setup.

If you want to use clang on Linux, you'll need

export CC=clang
export CXX=clang++
export PATH=$PATH:$WEBKIT_DIR/Source/WebKit/chromium/third_party/llvm-build/Release+Asserts/bin
export GYP_DEFINES="clang=1"

Now we're ready to pull in the Chromium dependencies

update-webkit --chromium

Done!

Development

Checking out code

First, update your checkout

cd $WEBKIT_DIR
git checkout master
update-webkit --chromium

Then create a branch to work on:

git checkout -b work

When you want to update your branch, update master as described above, and then rebase your branch on master:

cd $WEBKIT_DIR
git checkout master
update-webkit --chromium
git checkout work
git rebase master

The merge driver you configured above will take care of merging the ChangeLog files.

Development

Then you do the actual development. Keep in mind that WebKit uses a different coding style than Chromium: WebKit Coding Style Guidelines
See also Contributing Code and especially Writing New Tests

You can compile WebKit like this (the --makeargs parameter is only required when you use goma, otherwise ninja will use a reasonable default)

build-webkit --chromium --debug --makeargs=-jXXX

And run tests like this:

new-run-webkit-tests --chromium --debug --no-new-test-results --no-retry-failures
run-chromium-webkit-unit-tests --debug


Note that currently (when?) Linux baselines are for Ubuntu Lucid, Mac baselines for Mac 10.6, and Windows baselines for Win XP. If you don't have these operating systems available, you can try to pass --no-pixel-tests and hope that the text dumps are enough to catch regressions. On Ubuntu, you can setup a chroot as described here: Using a chroot.

If you added a new file, you'll need to rerun gyp

cd Source/WebKit/chromium; ./gyp_webkit

If you added a file to WebCore, you will need register it with all build systems used in WebKit which is surprisingly difficult and beyond this simple guide.

If you change the WebCore API, you will need to update exported symbol lists for various platforms as described here: Exporting Symbols

Uploading your patch

Once you're happy with your change, you can upload it to Bugzilla, either manually via the web page, or by using webkit-patch upload -- this automates many tasks, notably posting to the correct bug (or opening a new one if necessary), checking style, and preparing ChangeLogs, and it gives you a chance to review the patch before posting. Note that webkit-patch uses the same convention as git, namely that the command upload comes first, followed by options and then arguments.

There are many configuration options, which you can list with the help command via:

webkit-patch help upload

To post a patch to a new bug, without requesting review (such as for a WIP), use:

webkit-patch upload --no-review

This will prompt you for a one line description of your change, which will be used to create a bug on http://bugs.webkit.org/. It will also run some basic style checks on your patch, open an editor with ChangeLog entries, and show you the diff before uploading the patch. This will also store the URL https://bugs.webkit.org/show_bug.cgi?id=BUGID in the ChangeLogs, which will be used automatically determine the bug number on subsequent uploads. (It can also be stored in git commit messages, for instance, where it will also be picked up.)

You'll need EditBugs permission for the script to work properly (namely to assign the bug to you), else you'll get an error. You can request this (from anyone with EditUser permission) on the mailing list webkit-committers at lists.webkit.org or on #webkit on IRC (freenode).

There are a few conventions for the description/summary, such as using [WIP], or indicating subcomponents with [SubComponentName] or the like, and regressions with REGRESSION -- see Bug Reporting Guidelines.

There are various other fields you can fill in about the bug, detailed at Bug Reporting Guidelines. If you need to set these, it's easiest to use the web page. Of these, the most significant (and only one that can be set via the script) is the component, which you can set via --component=COMPONENT -- it may be useful to include this in a script if you generally work on a single component. You can also specify if it is only for a given platform, etc.


For subsequent uploads, you can just run this again (it will automatically determine the bug number). You likely want to automatically update the ChangeLogs (new date, revised bug description, touched files/functions), via:

webkit-patch upload --no-review --update-changelogs

Depending on the files you've touched, some people will get automatically cc'd on the issue. Otherwise, you can look at the files you've changed and try to find a good reviewer yourself, and cc them on the issue, or run

webkit-patch suggest-reviewers -g UPSTREAM....

Once your patch looks good to you and is ready for review, request review by setting the r? flag:

webkit-patch upload --update-changelogs

You can also set or clear the r (review) and cq (commit queue) flags on Bugzilla by clicking on the "Details" link next to the patch, and using the relevant drop-down.

On the bug tracker, the EWS (Early Warning System) will pick up your patch and check whether it compiles on a number of different configurations. The cr-linux bot will also execute tests. If anything breaks or there's anything wrong with the patch, substantively or stylistically you will likely get r- (review rejected) from a reviewer; if it's approved, you will get r+ and proceed to committing (see below).

If you want to upload your patch to an existing bug, you can specify the bug number on the command line, or manually include the URL https://bugs.webkit.org/show_bug.cgi?id=BUGID in the ChangeLog (or git commit message):

webkit-patch upload --no-review BUGID

Description and Comment

You can specify a one-line Description for the patch, and a multi-line Comment, further describing the patch and changes. webkit-patch upload does not automate this or provide an interactive facility (only command line parameters), but a brief script will do.

To get from git log (making sure you've committed, so it's not the previous, unrelated git message):

#!/bin/bash

if [[ -n $(git status --short) ]]
then
  echo "Uncommitted changes, please commit or stash."
  exit 1
fi

DESCRIPTION=`git log --format="%s" HEAD^..`
COMMENT=`git log --format="%b" HEAD^..`

webkit-patch upload --description="$DESCRIPTION" --comment="$COMMENT" --update-changelogs

To enter interactively:

#!/bin/bash

echo "Description:"
read DESCRIPTION
echo "Comment (end with ^D):"
COMMENT=`cat`

webkit-patch upload --description="$DESCRIPTION" --comment="$COMMENT" --update-changelogs

Alternatively, you can use webkit-patch post-commits which does automatically get from git.

Git

There are various git workflows, almost always starting by creating a local branch and switching to it. However, committing style varies.
  • Safest and most elaborate is to commit changes to your local branch (as new commits), as this lets you track the history of your own changes and revert to previous versions, and lets you store local notes on changes in the git commit log. webkit-patch upload will upload the combined diff, so you're free to make many local commits, only uploading a patch when it's in a good state. Recall that committing all of your changes -- after staging new files with git add -- is done via:
git commit --all
  • Many people prefer to have a single commit, which they amend. This simplifies local branch management, as you only have a single commit, though you lose your local history, which makes reverting and tracking your own changes impossible for revisions that haven't been upload. Any patch that has been uploaded can simply be downloaded and applied, however. This is done via:
git commit --all --amend
  • Some people don't even commit changes locally, only needing to git add new files. This makes it impossible to change branches, however, so this is primarily used in simple situations (e.g., only working on one bug, initial diff), or if using tools like git-new-workdir.
If you want webkit-patch upload to pick up the bug number from the git commit message (it should pick it up from the ChangeLogs, however), use a message such as:

DESCRIPTION

https://bugs.webkit.org/show_bug.cgi?id=XXX
COMMENT
...

post-commits

Instead of using webkit-patch upload you can alternatively use webkit-patch post-commits for slightly different behavior. post-commits is fundamentally for uploading a branch as a sequence of patches, and thus generates multiple attachments. For simple uses you don't want this behavior and thus must be using the "amend a single commit" workflow to get a single patch, or hack around this (if using multiple revisions on a local branch) by specifying the commitish appropriately, though this is tricky. One advantage is that post-commits will automatically set the Description (and optionally Comment) from the git commit, without needing the auxiliary scripts above. If using

If using the "amend a single commit" workflow, diff against HEAD via:

webkit-patch post-commits --add-log-as-comment HEAD

To only set the Description (no Comment), use:

webkit-patch post-commits HEAD

Manual operation

You can manually run the scripts that webkit-patch runs via:

check-webkit-style -g HEAD
prepare-ChangeLog -g HEAD

The first checks style; the second creates an updated structure for your ChangeLog entries, though you will still need to merge the old and new entries manually. You will need to update the ChangeLog entries for various reasons, such as the date changing, the bug description changing, adding tests, or change additional files or methods; this can be tedious to do this manually, hence the script

Landing patches

It's best to use the commit queue for landing patches. To get on the commit queue, you need to set the cq? flag on the patch (click on Details and set commit-queue to ?) or pass the --request-commit flag to webkit-patch upload or post-commits.

When you get a r+/cq+ on your patch, it will be automatically landed, and the commit queue will replace "Reviewed by nobody (OOPS)!" in the ChangeLogs with the respective reviewer.

Sometimes, you will get a r+/cq- (LGTM with nits). In that case, you can address the nits, and then you need to manually replace "Reviewed by nobody (OOPS)!" in the ChangeLogs with "Reviewed by Foo Bar", and upload that patch with cq? (but not requesting another review).

webkit-patch post-commits --no-review --request-commit HEAD

Once you're a committer, you can then set cq+ yourself, otherwise, any other committer can grant you cq+.

There's a number of situations where you can't use the commit queue, e.g. you might want to roll the chromium DEPS file:

git checkout -b deps
webkit-patch roll-chromium-deps

or you want to commit new baselines you created with garden-o-matic:

webkit-patch land-cowhand

or commit a patch from a reviewed bug:

webkit-patch land-from-bug BUG_ID

Cleaning up

Once your patch lands, you need to clean up by deleting your local branch. This is necessary because patches are committed outside the git flow (you don't git push them), so git doesn't know that they've been committed. You may want to wait a day or two after landing to make sure the patch didn't break anything and isn't reverted.

git branch -D work

If you have deleted your branch and need to revise the patch, just create a new branch, download the patch from Bugzilla, and apply it. This is also automated; note that you can download reviewed patches from a bug, or a specific attachment.

webkit-patch apply-from-bug BUGID
webkit-patch apply-attachment ATTACHMENT_ID

Testing your change in chromium

There are two ways to test how chromium does with your change: You can either patch your patch into the WebKit checkout in chromium:

webkit-patch apply-attachment --no-update ATTACHMENT_ID

Build Chromium as usual then.

Or you can build content_shell from within WebKit and test your change there: Running layout tests using the content shell
Comments