** This launched as of Chrome 31. Documentation: https://developer.chrome.com/extensions/tabCapture ** Proposal Date August 14th, 2012 Who is the primary contact for this API? mfoltz@chromium.org Who will be responsible for this API? (Team please, not an individual) mfoltz@chromium.org, keiger@google.com, markdavidscott@google.com Overview Do you know anyone else, internal or external, that is also interested in this API? Could this API be part of the web platform? Do you expect this API to be fairly stable? How might it be extended or changed in the future? Yes, we believe that this API will be fairly stable, since it is modelled after an existing API (getUserMedia) and has narrowly defined semantics. Extensions or changes to incorporate other capture scopes - e.g. window-level or desktop-level - is possible but should not result in significant changes to the API. If multiple extensions used this API at the same time, could they conflict with each others? If so, how do you propose to mitigate this problem? Nothing inherently prevents multiple tabs from being captured independently, but there may be implementation limits on how many tabs can be captured simultaneously (and in our initial implementation, we believe that capturing 1 tab is sufficient). To address this, the proposed API includes a notification to indicate when tab capture status changes so we can notify an application if a tab is no longer being captured due to a newer conflicting action. List every UI surface belonging to or potentially affected by your API: The proposed APIs don’t expose any new UI surfaces to extensions. It is proposed that we provide a visual indication to the user when a tab is being captured, though the exact design for this is still under consideration. One proposed approach is a color or animation on the tab itself or its favicon, another is use of the existing mechanism used by WebRTC to indicate when camera capture is active (this may be preferred for consistency). Actions taken with extension APIs should be obviously attributable to an extension. Will users be able to tell when this new API is being used? How? Per the above, we plan to have a visual indicator of the tab(s) being captured. Hovering over this indicator will indicate which extension is performing the capture. How could this API be abused? From a security/privacy perspective, since this API enables tab contents to be captured, it could be used to capture user browsing and to send that as a video stream to an external site. However, since we require the tabs permission, this is not substantively different from capturing the HTML/JS content of a tab (in fact, it’s less effective than the latter). It’s also no more sensitive than the tab snapshot API in terms of what it has access to. From a security perspective, creating a video stream that’s targetted at an arbitrary destination coud be used in DDOS-type attacks, though this is not significantly more effective than sending a large number of HTTP requests or transmitting a large binary file. From a performance perspective, too many simultaneous tabs being captured could impact performance. To mitigate this, we plan to limit the number of tabs that can be captured simultaneously (starting with a value of 1). Imagine you’re Dr. Evil Extension Writer, list the three worst evil deeds you could commit with your API (if you’ve got good ones, feel free to add more): 1) Per prior answer, scrape screen and send it to a malicious site. 2) Per prior answer, create “fake” video stream and use it for DDOS attack. 3) Per prior answer, create a bunch of bogus tab captures and send them somewhere meaningless, in an attempt to waste resources and trigger a crash. What security UI or other mitigations do you propose to limit evilness made possible by this new API? Per the above, we’re proposing a visual indicator for tabs that are being captured which identifies the extension performing the capture, and to ensure that the user is aware that capture is ongoing. Alright Doctor, one last challenge: Could a consumer of your API cause any permanent change to the user’s system using your API that would not be reversed when that consumer is removed from the system? No; no permanent changes are possible as a result of the proposed API. How would you implement your desired features if this API didn't exist? Existing alternatives depending on building native applications or using dangerous NPAPI plug-ins with full system access. Draft Manifest Changes At this point in time, we believe that the tabs permission is sufficient. We’re very open to input and advice on this, though! Draft API spec getTabMedia chrome.experimental.capture.getTabMedia(integer tabId, object options, function callback) Captures the visible area of the tab with the given tabId. Extensions must have the host permission for the given tab as well as the “tabs” permission to use this method. Parameters: tabId (optional integer): The tabId of the tab to capture. Defaults to the active tab. options (optional object): Configures the returned media stream. video (boolean): Whether to return video (default true) audio (boolean): Whether to return audio (default true) callback (function): Called when the media stream is ready. Callback function: function (stream) stream (optional LocalMediaStream): The LocalMediaStream that was created, or null if no stream was created. chrome.extension.lastError will be set if the stream could not be created (see below) Errors:
See below for details on the LocalMediaStream structure. onTabCaptured chrome.experimental.capture.onTabCaptured.addListener( function(object captureInfo) { ... }) Event fired when the capture status of a tab changes. This allows extension authors to keep track of the capture status of tabs, to keep UI elements like page actions and infobars in sync. Parameters: captureInfo (object): Details of the change. tabId (integer): The id of the tab whose status changed. status (string): The new capture status of the tab, one of “requested”, “cancelled”, “pending”, “active”, “muted”, “stopped” Status state diagram is attached. getCapturedTabs chrome.experimental.capture.getCapturedTabs(function callback) Returns a list of tabs that have requested capture or are being captured, i.e. status != stopped and status != cancelled. This allows extensions to inform the user that there is an existing tab capture that would prevent a new tab capture from succeeding (or, to prevent redundant requests for the same tab). Callback function: function(<array of CaptureInfo> result) { ... } CaptureInfo is as specified above. Open questions - Confirm permissions approach stated above - Can a WebKit object be returned through an extension API? - Control over aspect ratio, clipping, scaling of resulting media? |
For Developers > Design Documents > Extensions > Proposed & Proposing New Changes > API Proposals (New APIs Start Here) >
