Everything you need to know about audio/video inside Chromium and Chromium OS! Who To ContactIt's best to have discussion on chromium-dev@chromium.org, but failing that feel free to email me (scherkus@chromium.org). We are Cr-Internals-Media on the Chromium bug tracker. DocumentationHere's our (somewhat outdated) design doc for HTML5 audio/video. Vorbis audio codec Theora video codec VP8 video codec PCM 8-bit unsigned integer PCM 16-bit signed integer little endian PCM 32-bit float little endian Ogg container format WebM container format WAV container format When building Google Chrome, the following codecs/containers are also included: MP3 audio codec AAC audio codec (Main only, not AAC-LC, AAC-SSR, HE-AAC) H.264 video codec MP4 container format In terms of file extensions, Chromium recognizes the following (defined in src/net/base/mime_util.cc): mp4, m4v, m4a, mp3, ogv, ogm, ogg, oga, webm, wav When build Google Chrome OS, the following codecs/containers are also included: FLAC audio codec MPEG-4 video codec AMR-NB audio codec AMR-WB audio codec PCM μ-law In terms of file extensions, Google Chrome OS recognizes the following (defined in src/net/base/mime_util.cc): amr, avi, flac, 3gp Code LocationChromium src/media/ - Home to all things multimedia!src/media/audio - OS audio input/output abstractions src/media/video/capture - OS camera input abstraction src/media/video - software/hardware video decoder interfaces + implementations deps/third_party/ffmpeg - Chromium's copy of FFmpeg deps/third_party/libvpx - Chromium's copy of libvpx WebKit Source/WebCore/html/HTMLMediaElement.{cpp,h,idl} - media element base class Source/WebCore/html/HTMLAudioElement.{cpp,h,idl} - audio element implementation Source/WebCore/html/HTMLVideoElement.{cpp,h,idl} - video element implementation Source/WebCore/platform/graphics/MediaPlayer.{cpp,h} - encapsulates media playback inside WebKit Particularly Interesting Bits src/net/base/mime_util.cpp - defines canPlayType() behaviour and file extension mapping src/webkit/media/buffered_data_source.{cc,h} - Chromium's main implementation of DataSource for the media pipeline src/webkit/media/buffered_resource_loader.{cc,h} - Implements the sliding window buffering strategy (see below) src/webkit/media/webmediaplayer_impl.{cc,h} - Chromium's implementation of WebMediaPlayer Source/WebKit/chromium/public/WebMediaPlayer.h - Chromium's interface for providing HTML5 audio/video functionality How does everything get instantiated?WebViewClient::createMediaPlayer(WebFrame*, WebMediaPlayerClient*) is the WebKit embedder API for creating a WebMediaPlayer and passing it back to WebKit. Every HTML5 audio/video element will ask to create a WebMediaPlayer object from the embedder. For Chromium (the entire browser) this is handled in RenderView: src/content/renderer/render_view.cc For Test Shell this is handled in TestWebViewDelegate: src/webkit/tools/test_shell/test_webview_delegate.h What's with all these abstraction layers!?Necessary in order to properly isolate Chromium from WebKit. Although it looks confusing, really all you need to worry about is WebKit's MediaPlayer and Chromium's WebMediaPlayerImpl. The rest of it amounts to simple method forwarding.  GYP FlagsThere are a few GYP flags which can alter the behaviour of Chromium's HTML5 audio/video implementation.ffmpeg_branding Overrides which version of FFmpeg to use Default: $(branding) Values: Chrome - includes additional proprietary codecs (MP3, etc..) for use with Google Chrome Chromium - builds default set of codecs proprietary_codecs Alters the list of codecs Chromium claims to support, which affects <source> and canPlayType() behaviour Default: 0 Values: 0 - <source> and canPlayType() assume the default set of codecs 1 - <source> and canPlayType() assume they support additional proprietary codecs use_system_ffmpeg Will build Chromium using the host system's version of FFmpeg Default: 0 Values: 0 - Build Chromium using the copy of FFmpeg included with Chromium 1 - Build Chromium using the host system's version of FFmpeg LayoutTestsWe run the following layout tests: Source/LayoutTests/media Source/LayoutTests/http/tests/media If these are failing, let us know! How the %#$& does buffering work?Chromium uses a combination of range requests and an in-memory sliding window to buffer media. We have a low and high watermark that is used to determine when to purposefully stall the HTTP request and when the resume the HTTP request. It's complicated, so here's a picture:  |
Audio/Video |