The Chromium Projects

Search this site

Except as otherwise noted, the content of this page is licensed under a Creative Commons Attribution 2.5 license, and examples are licensed under the BSD License.

SyncFileSystem API

Proposal Date

Last updated 2013-02-27

(2013-02-11 -> 2013-02-27: Fixed the remote folder name from "Chrome Syncable Storage" to "Chrome Syncable FileSystem")

Who is the primary contact for this API?

kinuko@chromium.org

Overview

Sync FileSystem API is a storage API that appears as yet another local offline HTML5 FileSystem storage which gives app-private sandboxed file storage, except that the storage content is automatically synchronized across clients via a cloud backing service like Google Drive.  We initially only target Google Drive as its backing service.

Use cases

This API could be used to store user-generated data (or any other binary data) locally for offline or caching usage when the app also wants to save/synchronize the data on a cloud storage so that the same data can be available across different clients.

This API is NOT for accessing arbitrary user docs stored on a cloud service (e.g. not for opening your google docs document) but to provide app-specific syncable storage.

Do you know anyone else, internal or external, that is also interested in this API? 

I heard there could be some but I'm not aware of anyone at this time.

Could this API be part of the web platform?

Possibly yes, as this API has strong relationship with the existing owp proposal: File API: Directories and System (http://www.w3.org/TR/file-system-api/).

Do you expect this API to be fairly stable?  How might it be extended or changed in the future?

The API proposed in this API is supposed to be stable, while additional functionality to support more detailed manual conflict handling and sync behavior control might be added in the future.

If multiple extensions used this API at the same time, could they conflict with each others? If so, how do you propose to mitigate this problem?

No. Multiple extensions should be able to get its own private file storage.

List every UI surface belonging to or potentially affected by your API:

- Advanced Sync Settings (disabling 'Apps' sync will disable apps data synchronization)
- Apps settings UI (when http://crbug.com/141584 kicks in)

Actions taken with extension APIs should be obviously attributable to an extension. Will users be able to tell when this new API is being used? How?

In the current architecture users can tell the usage of this API by:
- Looking at the user's sync backend service storage (e.g. Drive).
  To be specific, current implementation saves the file in a folder named "Chrome Syncable FileSystem" on the signed-in user's Drive storage.
- Looking into the local sandboxed directory for the app.
- By querying the usage bytes returned by chrome.syncFileSystem.getUsageAndQuota API.

Once we have app settings UI for disk space users should also be able to tell when the API stores something locally.

How could this API be abused?

One could use up the user's local disk and/or sync backend storage but it cannot use more than the quota that is adjusted by the local and remote storage capacity.

Imagine you’re Dr. Evil Extension Writer, list the three worst evil deeds you could commit with your API (if you’ve got good ones, feel free to add more):
1)
2)
3)

What security UI or other mitigations do you propose to limit evilness made possible by this new API?

  • App settings UI could show storage usage information both on local and remote (i.e. sync backend storage) side, and could provide UI to purge data stored by this API.

Alright Doctor, one last challenge:  
Could a consumer of your API cause any permanent change to the user’s system using your API that would not be reversed when that consumer is removed from the system?

This API would leave the user data synchronized by this API on the remote sync backend storage even after the consumer is uninstalled, but the user should be able to delete the data by directly visiting the sync backend storage service (e.g. Drive Web interface for Drive).

How would you implement your desired features if this API didn't exist?

Technically I could combine offline storage API (e.g. FileSystem API or IndexedDB) and remote storage service API (e.g. Drive SDK API) and implement every synchronization logic in JS layer, but it usually requires more engineering power.  Integrating a custom JS app with server-side push change notifications and/or efficient background polling mechanism has several limitations too (though serious sync apps are doing that by themselves).

Also giving an app access permission to a cloud storage usually means allowing the app to access every file on the storage for the authenticated identity (e.g. giving an app access permission to Drive SDK effectively allows the app to access every docs on Drive of the user). This API hides such details from JS layer and gives isolated access only to the app's private storage.

Draft Manifest Changes 

This API adds a new permission: "syncFileSystem"

Draft API spec
  • chrome.syncFileSystem.requestFileSystem(function (filesystem) { ... });

    Requests a new 'syncable' file storage for the requesting app.      This returns FileSystem object which can be operated on in the same way as local offline filesystems in FileSystem API (http://www.w3.org/TR/file-system-api/).  This means that the app can create a DirectoryReader on the root directory and gets a list of files, can get a FileWriter to write binary data into the file, can copy or move a file within the filesystem or to/from other filesystems (including file entries that are drag-and-drop'ed, a file that is opened by chrome.fileSystem API, media files available in chrome.mediaGalleries API, and so on and so on).  Typical example code would look like following:

    chrome.syncFileSystem.requestFileSystem(function (fs) {
       // FileSystem API should just work on the returned 'fs'.
       fs.root.getFile('test.txt', {create:true}, getEntryCallback, errorCallback);
    });
  • chrome.syncFileSystem.getUsageAndQuota(function (storageInfo) { ... });

    Returns the current usage and quota in bytes for the 'syncable' file storage for the app. (Limitation: currently the returned quota size may not appropriately reflect the quota configuration on the cloud service side.)  Typical example code would look like following:

    chrome.syncFileSystem.getUsageAndQuota(fileSystem, function (storageInfo) {
       updateUsageInfo(storageInfo.usageBytes);
       updateQuotaInfo(storageInfo.quotaBytes);
    });

  • chrome.syncFileSystem.getFileStatus(fileEntry, function (status) { ... });

    Returns the current file sync status. The status value can be either one of: 'synced', 'pending' and 'conflicting'.

    • 'synced' status means the file is fully synchronized, meaning that there're no pending local changes that haven't been synchronized to the cloud storage.  (There might be pending changes on the cloud side that haven't been fetched yet even if the file status is 'synced'.)
    • 'pending' status means the file has pending changes that haven't been synchronized to the cloud storage. Usually local changes are (almost) immediately synchronized to the cloud if the app is running online.  When pending changes are synchronized onFileStatusChanged event is fired with 'synced' status (see below for more details about the event).
    • 'conflicting' status means there are conflicting changes both on the local and cloud (i.e. on other clients).  When a file becomes conflicting status onFileStatusChanged event is fired with 'conflicting' status.
      Currently the API does NOT provide automatic reconciliation mechanism for conflicting files, and if no action is taken the file will be kept detached from the remote changes made on other clients.  The app can continue to read/write the file as a detached local offline file.  The conflict is automatically resolved once either one of the local or conflicting version of the file is deleted.  When the conflicting status is resolved onFileStatusChanged event is fired with 'synced' status.

  • chrome.syncFileSystem.onFileStatusChanged.addListener(function (fileInfo) { ... });

    This event is fired when the sync status of a file is changed.  The 'fileInfo' object includes following fields:

    • fileEntry is a FileEntry object for the target file whose status is changed.  This object may be pointing to non-existing file if the sync action was 'deleted'.
    • status is the resulting file status by the event.  The value could be either one of 'synced', 'pending' or 'conflicting'.
    • action is the sync action taken to fire this event.  This field has a valid value only if the status is 'synced'.  The value could be either one of 'added', 'updated' or 'deleted'.
    • direction is the sync direction for the event.  This field has a valid value only if the status is 'synced'.  The value could be either one of 'local_to_remote' or 'remote_to_local'.
For example, assume a file has pending changes and in 'pending' state. The app may have been in offline state so that the change is about to be synchronized.  When the sync service detects the pending local change and uploads the change to the cloud storage, the service fires onFileStatusChanged event with following values: { fileEntry:a fileEntry for the file, status: 'synced', action: 'updated', direction: 'local_to_remote' }.

Similarly, regardless of the local activities the sync service may detect remote changes made by another client, and downloads the new image from the cloud storage to the local storage.  If the remote change was for adding a new file, an event with following values is fired: { fileEntry: a fileEntry for the file, status: 'synced', action: 'added', direction: 'remote_to_local' }.

If both the local and remote side have conflicting changes for the same file, the file status is changed to 'conflicting' state and it's detached from the sync service (won't be synchronized until the conflict is resolved).  In this case an event with following values is fired:  { fileEntry: a fileEntry for the file, status: 'conflicting', action: null, direction: null }.


Draft API IDL

=================================================================================
namespace syncFileSystem {
  enum SyncAction {
    added, updated, deleted
  };

  enum ServiceStatus {
    // The sync service is being initialized (e.g. restoring data from the
    // database, checking connectivity and authenticating to the service etc).
    initializing,

    // The sync service is up and running.
    running,

    // The sync service is not synchronizing files because the remote service
    // needs to be authenticated by the user to proceed.
    authentication_required,

    // The sync service is not synchronizing files because the remote service
    // is (temporarily) unavailable due to some recoverable errors, e.g.
    // network is offline, the remote service is down or not
    // reachable etc. More details should be given by |description| parameter
    // in OnServiceInfoUpdated (which could contain service-specific details).
    temporary_unavailable,

    // The sync service is disabled and the content will never sync.
    // (E.g. this could happen when the user has no account on
    // the remote service or the sync service has had an unrecoverable
    // error.)
    disabled
  };

  enum FileStatus {
    // Not conflicting and has no pending local changes.
    synced,

    // Has one or more pending local changes that haven't been synchronized.
    pending,

    // File conflicts with remote version and must be resolved manually.
    conflicting
  };

  enum SyncDirection {
    local_to_remote, remote_to_local
  };

  dictionary FileInfo {
    // |fileEntry| will contain name and path information of synchronized file.
    // On file deletion, fileEntry information will still be available but file
    // will no longer exist.
    [instanceOf=FileEntry] object fileEntry;
    
    FileStatus status;
    
    // Only applies if status is synced.
    SyncAction? action;
    
    // Only applies if status is synced.
    SyncDirection? direction;
  };

  dictionary StorageInfo {
    long usageBytes;
    long quotaBytes;
  };

  dictionary ServiceInfo {
    ServiceStatus state;
    DOMString description;
  };

  // [nodoc] A callback type for requestFileSystem.
  callback GetFileSystemCallback =
      void ([instanceOf=DOMFileSystem] object fileSystem);

  // [nodoc] A callback type for getUsageAndQuota.
  callback QuotaAndUsageCallback = void (StorageInfo info);

  // Returns true if operation was successful.
  callback DeleteFileSystemCallback = void (boolean result);

  // [nodoc] A callback type for getFileStatus.
  callback GetFileStatusCallback = void (FileStatus status);

  interface Functions {
    // Returns a syncable filesystem backed by Google Drive. 
    // The returned DOMFileSystem instance can be operated on in the same way as
    // the Temporary and Persistant file systems.
    // (http://www.w3.org/TR/file-system-api/). Calling this multiple times from
    // the same app will return the same handle to the same file system. 
    static void requestFileSystem(GetFileSystemCallback callback);

    // Get usage and quota in bytes for sync file system with |serviceName|.
    static void getUsageAndQuota([instanceOf=DOMFileSystem] object fileSystem,
                                 QuotaAndUsageCallback callback);

    // Deletes everything in the syncable filesystem.
    static void deleteFileSystem([instanceOf=DOMFileSystem] object fileSystem,
                                 DeleteFileSystemCallback callback);

    // Get the FileStatus for the given |fileEntry|.
    static void getFileStatus([instanceOf=FileEntry] object fileEntry,
                              GetFileStatusCallback callback);
  };

  interface Events {
    // Fired when an error or other status change has happened in the
    // sync backend. (e.g. the sync is temporarily disabled due to
    // network or authentication error etc).
    static void onServiceStatusChanged(ServiceInfo detail);

    // Fired when a file has been updated by the background sync service.
    static void onFileStatusChanged(FileInfo detail);
  };

};


Comments