For Developers‎ > ‎Code Reading‎ > ‎

How Uploading Works

Uploading a file from Chromium to a web server is a complex operation as many things happen across the boundary of Chromium and WebKit. This document describes how the whole thing works.

Disclaimer: the document was originally written in the late November 2011. The contents will be deprecated as Chromium code evolves.

Clicking on a File Upload Button

Let’s get started with figuring out what happens if a user clicks on a file upload button in a web page.

When the user clicks on <input type=file>, the browser process receives an IPC message named ViewHostMsg_RunFileChooser from the renderer, and opens the select file dialog. The process is initiated by RenderViewHost::OnRunFileChooser() in http://src.chromium.org/viewvc/chrome/trunk/src/content/browser/renderer_host/render_view_host.cc?view=markup and eventually propagated to FileSelectHelper, and SelectFileDialog::Create() will be called.

void RenderViewHost::OnRunFileChooser(
   const ViewHostMsg_RunFileChooser_Params& params) {
 delegate_->
RunFileChooser(this, params);
}

void Browser::RunFileChooser(TabContents* tab,
                            const ViewHostMsg_RunFileChooser_Params& params) {
 
RunFileChooserHelper(tab, params);
}


void Browser::RunFileChooserHelper(
   TabContents* tab, const ViewHostMsg_RunFileChooser_Params& params) {
 ..

 file_select_helper->RunFileChooser(tab->render_view_host(), tab, params);
}


void FileSelectHelper::RunFileChooser(
   RenderViewHost* render_view_host,
   TabContents* tab_contents,
   const ViewHostMsg_RunFileChooser_Params& params) {
 ..

}

void FileSelectHelper::RunFileChooserOnUIThread(
   const ViewHostMsg_RunFileChooser_Params& params) {
 ..
 if (!select_file_dialog_.get())
   select_file_dialog_ =
SelectFileDialog::Create(this);


Selecting a File from a Dialog

Now the user is asked to choose a file using a platform specific file selection dialog.

SelectFileDialog::Create(listener) is used to show “Open File” dialog.
http://src.chromium.org/viewvc/chrome/trunk/src/chrome/browser/ui/select_file_dialog.h?view=markup

Chrome OS’s implementation is SelectFileDialogExtension
http://src.chromium.org/viewvc/chrome/trunk/src/chrome/browser/ui/views/select_file_dialog_extension.h?view=markup

Once a file or files are selected, the listener is notified by FileSelected() or MultiFilesSelected() in http://src.chromium.org/viewvc/chrome/trunk/src/chrome/browser/file_select_helper.cc?view-markup

void FileSelectHelper::FileSelected(const FilePath& path,
                                   int index, void* params) {

 ..

 files.push_back(path);
 
NotifyRenderViewHost(render_view_host_, files, dialog_type_);

 // No members should be accessed from here on.
 RunFileChooserEnd();
}


void NotifyRenderViewHost(RenderViewHost* render_view_host,
                         const std::vector& files,
                         SelectFileDialog::Type dialog_type) {

 ..
 render_view_host->FilesSelectedInChooser(files, permissions);
}


Then, an IPC message named ViewMsg_RunFileChooserResponse is sent from the browser to the renderer:

void RenderViewHost::FilesSelectedInChooser(
   const std::vector& files,
   int permissions) {
 // Grant the security access requested to the given files.                    
 for (std::vector::const_iterator file = files.begin();
      file != files.end(); ++file) {
   ChildProcessSecurityPolicy::GetInstance()->GrantPermissionsForFile(
process()->id(), *file, permissions);
 }
 
Send(new ViewMsg_RunFileChooserResponse(routing_id(), files));
}


In the renderer, the response is handled in RenderViewImpl::OnFileChooserResponse() in http://src.chromium.org/viewvc/chrome/trunk/src/content/renderer/render_view_impl.cc?view=markup

void RenderViewImpl::OnFileChooserResponse(const std::vector& paths) {
 ..
 if (file_chooser_completions_.front()->completion)
   file_chooser_completions_.front()->completion->
didChooseFile(ws_file_names);

Here, |completion| is an object of WebFileChooserCompletion, which is a class in WebKit.

void WebFileChooserCompletionImpl::didChooseFile(const WebVector& fileNames)
{
   if (fileNames.size() == 1)
       m_fileChooser->
chooseFile(fileNames[0]);
   else if (fileNames.size() > 0) {
       Vector paths;
       for (size_t i = 0; i < fileNames.size(); ++i)
           paths.append(fileNames[i]);
       m_fileChooser->
chooseFiles(paths);
   }
   // This object is no longer needed.
   delete this;
}


Where m_fileChooser is an object of WebCore::FileChooser.

void FileChooser::chooseFiles(const Vector& filenames)
{
   // FIXME: This is inelegant. We should not be looking at settings here.     
   if (m_settings.selectedFiles == filenames)
       return;

   if (m_client)
       m_client->
filesChosen
(filenames);
}


Where m_client is an object of WebCore::FileChooserClient, implemented by FileInputType in WebKit/Source/WebCore/html/FileInputType.cpp

void FileInputType::filesChosen(const Vector& paths)
{
   ..

   
setFileList(paths);
}


void FileInputType::setFileList(const Vector& paths)
{
   m_fileList->clear();
   size_t size = paths.size();

   ..
   for (size_t i = 0; i < size; i++)
       
m_fileList->append(File::create(paths[i]));
}

Uploading a File - What happens in the Renderer?

Finally, when the user ‘submit’ button, Chromium sends the file to the web server.

When ‘submit’ button is clicked, in the renderer, FormSubmission object is created to perform form submission. This class collects form data to FormDataList form HTML form elements in http://trac.webkit.org/browser/trunk/Source/WebCore/loader/FormSubmission.cpp

PassRefPtr FormSubmission::create(HTMLFormElement* form, const \
Attributes& attributes, PassRefPtr event, bool lockHistory, FormSubmissi\
onTrigger trigger)
{

 ..
  for (unsigned i = 0; i < form->associatedElements().size(); ++i) {
FormAssociatedElement* control = form->associatedElements()[i];
       HTMLElement* element = toHTMLElement(control);
       if (!element->disabled())
           control->
appendFormData(*domFormData, isMultiPartForm);


AppendFormData() is implemented by sub classes of FormAssociatedElement. Here, it’s HTMLInputElement in http://trac.webkit.org/browser/trunk/Source/WebCore/html/HTMLInputElement.cpp

bool HTMLInputElement::appendFormData(FormDataList& encoding, bool multipart)
{
   return m_inputType->isFormDataAppendable() &&
m_inputType->appendFormData(encoding, multipart);
}


Here, appendFormData() is FileInputType::appendFormData in http://trac.webkit.org/browser/trunk/Source/WebCore/html/FileInputType.cpp

bool
FileInputType::appendFormData(FormDataList& encoding, bool multipart) const
{
   ..
   for (unsigned i = 0; i < numFiles; ++i)
       encoding.appendBlob(element()->name(), fileList->item(i));
   return true;
}

class FormDataList {
  ...
    void appendBlob(const String& key, PassRefPtr<Blob> blob, const String& filename = String())
    {
        appendString(key);
        appendBlob(blob, filename);
    }

void FormDataList::appendBlob(PassRefPtr<Blob> blob, const String& filename) { m_items.append(Item(blob, filename)); }

Where m_items is a vector of FormDataList::Item.



This FormListData is used in FormData::appendKeyValuePairItems()

void
FormData::appendKeyValuePairItems(const FormDataList& list, const TextEncoding& encoding, bool isMultiPartForm, Document* document, E\
ncodingType encodingType)
{

   ..
   for (size_t i = 0; i < formDataListSize; i += 2) {
 
const FormDataList::Item& key = items[i]; const FormDataList::Item& value = items[i + 1];
if (isMultiPartForm) {
           ..
           if (value.blob()) {
               if (value.blob()->isFile()) {
                    // For file blob, use the filename (or relative path if it is present) as the name.           
                    File* file = static_cast<File*>(value.blob());
#if ENABLE(DIRECTORY_UPLOAD)                
                    name = file->webkitRelativePath().isEmpty() ? file->name() : file->webkitRelativePath();
#else
                    name = file->name();
#endif

                   // Do not add the file if the path is empty.                                                                           
                   if (!static_cast(value.blob())->path().isEmpty())
                       
appendFile(static_cast<File*>(value.blob())->path(), shouldGenerateFile);
               }
..
                // We have to include the filename=".." part in the header, even if the filename is empty         
                FormDataBuilder::addFilenameToMultiPartHeader(header, encoding, name);



void
FormData::appendFile(const String& filename, bool shouldGenerateFile)
{
#if ENABLE(BLOB)
   
m_elements.append(FormDataElement(filename, 0, BlobDataItem::toEndOfFile, BlobDataItem::doNotCheckFileChange, shouldGenerateFile));
#else
   m_elements.append(FormDataElement(filename, shouldGenerateFile));
#endif
}


Where m_elements is Vector<FormDataElement>.

The FormData is then used to build a ResourceRequest object:

void FormSubmission::populateFrameLoadRequest(FrameLoadRequest& frameRequest)
{
   if (!m_target.isEmpty())
       frameRequest.setFrameName(m_target);

   if (!m_referrer.isEmpty())
       frameRequest.resourceRequest().setHTTPReferrer(m_referrer);

   if (m_method == FormSubmission::PostMethod) {
       frameRequest.resourceRequest().setHTTPMethod("POST");
      
 frameRequest.resourceRequest().setHTTPBody(m_formData)
;

The FormData is then used in FrameLoader::loadPostRequest() in http://trac.webkit.org/browser/trunk/Source/WebCore/loader/FrameLoader.cpp

void FrameLoader::loadPostRequest(const ResourceRequest& inRequest, const String& referrer, const String& frameName, bool lockHistory, FrameLoadType loadType, PassRefPtr event, PassRefPtr prpFormState)
{
   ..

   RefPtr formData = inRequest.httpBody();
   ..

   workingResourceRequest.setHTTPBody(formData)
;

setHTTPBody() here is tricky. This is WebURLRequest::setHTTPBody() in http://trac.webkit.org/browser/trunk/Source/WebKit/chromium/src/WebURLRequest.cpp which looks like:

void WebURLRequest::setHTTPBody(const WebHTTPBody& httpBody)
{
    m_private->m_resourceRequest->setHTTPBody(httpBody);
}

Why can this take formData, which is a FormData object? Because WebHTTPBody has a non-explicit constructor: http://trac.webkit.org/browser/trunk/Source/WebKit/chromium/public/platform/WebHTTPBody.h

    WebHTTPBody(const WTF::PassRefPtr<WebCore::FormData>&);
 
Remember that you are in the WebKit world, not Chromium, where non-explicit constructors like this are prohibited..




The renderer loads a new document using the form data. The flow is like:
FrameLoader::loadPostRequest() …
DocumentLoader::startLoadingMainResource() …
MainResourceLoader::loadNow() …
ResourceHandle::start() …
WebURLLoaderImpl::loadAsynchronously()


The ResourceRequest object we created earlier is converted to WrappedResourceRequest in http://trac.webkit.org/browser/trunk/Source/WebKit/chromium/src/ResourceHandle.cpp

void ResourceHandleInternal::start()
{
   if (m_state != ConnectionStateNew)
       CRASH();
   m_state = ConnectionStateStarted;

   m_loader = adoptPtr(webKitPlatformSupport()->createURLLoader());
   ASSERT(m_loader);

  
 WrappedResourceRequest wrappedRequest(m_request);
   wrappedRequest.setAllowStoredCredentials(allowStoredCredentials());
   m_loader->loadAsynchronously(wrappedRequest, this);
}


Note that WrappedResourceRequest is a sub class of WebURLRequest in


WrappedResourceRequest (== fWebURLResponse) is  used in WebURLLoaderImpl::Context::Start() to pass the file information to IPCResourceLoaderBridge  in http://src.chromium.org/viewvc/chrome/trunk/src/webkit/glue/weburlloader_impl.cc?view=markup

void WebURLLoaderImpl::Context::Start(
   const
WebURLRequest& request,
   ResourceLoaderBridge::SyncLoadResponse* sync_load_response) {
 ..

 if (!request.httpBody().isNull()) {
   // GET and HEAD requests shouldn't have http bodies.                        
   DCHECK(method != "GET" && method != "HEAD");
  
 const WebHTTPBody& httpBody = request.httpBody();
   size_t i = 0;
   WebHTTPBody::Element element;
   while (httpBody.elementAt(i++, element)) {
     switch (element.type) {
       ..

case WebHTTPBody::Element::TypeFile:
         if (element.fileLength == -1) {
           bridge_->
AppendFileToUpload(
               WebStringToFilePath(element.filePath));
         }


IPCResourceLoaderBridge stores the file path information in request_, which is an object of IPCResourceLoaderBridge in http://src.chromium.org/viewvc/chrome/trunk/src/content/common/resource_dispatcher.cc?view=markup

void IPCResourceLoaderBridge::AppendFileRangeToUpload(
   const FilePath& path, uint64 offset, uint64 length,
   const base::Time& expected_modification_time) {
 DCHECK(request_id_ == -1) << "request already started";

 if (!request_.upload_data)
   request_.upload_data = new net::UploadData();
 request_.upload_data->
AppendFileRange(path, offset, length,
                                       expected_modification_time);
}


Then, IPCResourceLoaderBridge issues an IPC message named ResourceHostMsg_RequestResource to the browser process, in http://src.chromium.org/viewvc/chrome/trunk/src/content/common/resource_dispatcher.cc?view=markup


// Writes a footer on the message and sends it                                  
bool
IPCResourceLoaderBridge::Start(Peer* peer) {
 ..

 // generate the request ID, and append it to the message                      
 request_id_ = dispatcher_->AddPendingRequest(
     peer_, request_.resource_type, request_.url);
 return dispatcher_->message_sender()->S
end(
     new ResourceHostMsg_RequestResource(routing_id_, request_id_, request_));
}


Uploading a File - What happens in the Browser?

Now the flow is moved to the browser process.

The IPC message is handled in the browser process in resource_dispatcher_host.cc: http://src.chromium.org/viewvc/chrome/trunk/src/content/browser/renderer_host/resource_dispatcher_host.cc?view=markup and the upload data is passed to net::URLRequest object.

void ResourceDispatcherHost::OnRequestResource(
   const IPC::Message& message,
   int request_id,
   const ResourceHostMsg_Request&
request_data) {
 BeginRequest(request_id, request_data, NULL, message.routing_id());
}
..
void ResourceDispatcherHost::
BeginRequest(
   int request_id,
   const ResourceHostMsg_Request& request_data,
   IPC::Message* sync_result,  // only valid for sync                          
   int route_id) {
 ..

 // Construct the request.                                                     
 net::URLRequest* request = new
net::URLRequest(request_data.url, this);
 ..

 // Set upload data.                                                           
 uint64 upload_size = 0;
 if (request_data.upload_data) {
   request->
set_upload(request_data.upload_data);
   upload_size = request_data.upload_data->GetContentLength();
 }


The request will be added into ResourceQueue in and then started immediately, or at a later time by ResourceQueue::StartDelayedRequests(), depending on whether interested delegates are presents or not, in http://src.chromium.org/viewvc/chrome/trunk/src/content/browser/renderer_host/resource_queue.cc?view=markup

void ResourceQueue::AddRequest(
   net::URLRequest* request,
   const ResourceDispatcherHostRequestInfo& request_info) {
 ..

 if (interested_delegates.empty()) {
   
request->Start();
   return;
 }

void ResourceQueue::StartDelayedRequests(ResourceQueueDelegate* delegate) {
 ..
  // If no more delegates want a delay, start the request.                                             
   if (interested_delegates_[request_id].empty()) {
     interested_delegates_.erase(request_id);
     net::URLRequest* request = i->second;
     // The request shouldn't have started (SUCCESS is the initial state).                              
     DCHECK_EQ(net::URLRequestStatus::SUCCESS, request->status().status());
    
 request->Start();
   }
 }


FWIW, As of now, there are only two delegates: chrome/browser/extensions/network_delay_listener.h and chrome/browser/extensions/user_script_listener.h

Uploading a File - Finally we are in the network layer!


URLRequest::Start() calls URLRequest::StartJob() that calls URLRequestJob::StartJob()

void URLRequest::StartJob(URLRequestJob* job) {
 ..
 if (upload_.get())
  
 job_->SetUpload(upload_.get());
 ..
 job->Start();
}

job here is actually an object of URLRequestHttpJob in http://src.chromium.org/viewvc/chrome/trunk/src/net/url_request/url_request_context.cc?view=markup

void URLRequestHttpJob::SetUpload(UploadData* upload) {
 DCHECK(!transaction_.get()) << "cannot change once started";
 
request_info_.upload_data = upload;
}


void URLRequestHttpJob::Start() {
 DCHECK(!transaction_.get());


Where request_info_ is an object of HttpRequestInfo in http://src.chromium.org/viewvc/chrome/trunk/src/net/http/http_request_info.h?view=markup

HttpTransaction is started in URLRequestHttpJob::StartTransactionInternal()

void URLRequestHttpJob::StartTransactionInternal() {
 ..
      rv = transaction_->Start(
           &request_info_, &start_callback_, request_->net_log());



For actual HTTP network transaction (as opposed to cache from the local drive), HttpNetworkTransaction in http://src.chromium.org/viewvc/chrome/trunk/src/net/http/http_network_transaction.cc?view=markup is used.

The HTTP request is built in HttpNetworkTransaction::DoBuildRequest():

int HttpNetworkTransaction::DoBuildRequest() {
 next_state_ = STATE_BUILD_REQUEST_COMPLETE;
 request_body_.reset(NULL);
 if (request_->upload_data) {
   int error_code;
   request_body_.reset(
       UploadDataStream::Create(request_->upload_data, &error_code));
   if (!request_body_.get())
     return error_code;
 }

As shown, the body of the HTTP request (i.e. POST data), is built using UploadDataStream in http://src.chromium.org/viewvc/chrome/trunk/src/net/base/upload_data_stream.cc?view=markup. File is handled as follows:

int UploadDataStream::FillBuf() {
 ..
 while (buf_len_ < kBufSize && next_element_ < elements.size()) {
   bool advance_to_next_element = false;

   UploadData::Element& element = elements[next_element_];
   if (element.type() == UploadData::TYPE_BYTES ||
       element.type() == UploadData::TYPE_CHUNK) {
     ..   
   } else {
     DCHECK(element.type() ==
UploadData::TYPE_FILE);

     if (!next_element_remaining_) {
       // If the underlying file has been changed, treat it as error.                                   
       // Note that the expected modification time from WebKit is based on                              
       // time_t precision. So we have to convert both to time_t to compare.                            
       if (!element.expected_file_modification_time().is_null()) {
         base::PlatformFileInfo info;
         if (
file_util::GetFileInfo(element.file_path(), &info) &&
             element.expected_file_modification_time().ToTimeT() !=
                 info.last_modified.ToTimeT()) {
           return ERR_UPLOAD_FILE_CHANGED;
         }
       }
       next_element_remaining_ = element.GetContentLength();
       
next_element_stream_.reset(element.NewFileStreamForReading());
     }

     int rv = 0;
     int count =
         static_cast(std::min(next_element_remaining_,
                                   static_cast(size_remaining)));
     if (count > 0) {
       // Temporarily allow until fix: http://crbug.com/72001.                                          
       base::ThreadRestrictions::ScopedAllowIO allow_io;
       if (next_element_stream_.get())
         rv =
next_element_stream_->Read(buf_->data() + buf_len_, count,
                                         CompletionCallback());
       if (rv <= 0) {
         // If there's less data to read than we initially observed, then                               
         // pad with zero.  Otherwise the server will hang waiting for the                              
         // rest of the data.                                                                           
         memset(buf_->data() + buf_len_, 0, count);
         rv = count;
       }
       buf_len_ += rv;
     }

     if (static_cast(next_element_remaining_) == rv) {
       advance_to_next_element = true;
     } else {
       next_element_remaining_ -= rv;
     }
   }


The file is opened in UploadData::Element::NewFileStreamForReading() in http://src.chromium.org/viewvc/chrome/trunk/src/net/base/upload_data.cc?view=markup

FileStream* UploadData::Element::NewFileStreamForReading() {
 ..

 scoped_ptr file(new FileStream());
 int64 rv =
file->Open(file_path_,
                     base::PLATFORM_FILE_OPEN | base::PLATFORM_FILE_READ);
Comments