NaCl Manifest File Format for GLibC

Overview

This document describes the changes needed for compiling a Native Client (NaCl) executable (nexe) using Dynamic Shared Objects (DSOs) and the glibc toolchain, instead of nexes created using static linking and the newlib toolchain.

Compiling differences between glibc and newlib

GCC command-line flags for the dynamic linking of an executable and objects files (i.e. *.o files) in the nacl-glibc toolchain are identical to the ones used in the nacl-newlib toolchain.  It is not recommended to statically link executable binaries with the nacl-glibc toolchain (the resulting .nexe is overly large), however it is possible to build a statically linked binary.  In this case, together with the argument -static, two additional arguments should be provided for the compiler/linker:
  • for x86-64: -T $TOOLCHAIN/x86_64-nacl/lib/ldscripts/elf64_nacl.x.static
  • for x86-32: -T $TOOLCHAIN/x86_64-nacl/lib/ldscripts/elf_nacl.x.static

This requirement is not common among systems compiling with GCC and may be changed soon.

Note: with the nacl-glibc toolchain it is now possible to build dynamic libraries and load them via a manifest file.  Doing so is helpful for caching code between multiple invocations of the same .nexe. However, there is no caching support in Chromium yet.  Building dynamic libraries with nacl-glibc toolchain is mostly the same as on a GNU system, more details on building dynamic libraries can be found here: How to Build Shared Libraries (PDF).

NaCl Manifest File for glibc

As of October 2011, it is necessary to manually generate the nmf files when using glibc. However, we hope to add support into a future version of the NaCl SDK (Software Developer's Kit) to automatically generate the nmf for glibc.

Here's an example of what a glibc-compatible nmf file looks like:

Example nmf file with glibc

{
 "program": {
   "x86-64": {"url": "lib64/runnable-ld.so"},
   "x86-32": {"url": "lib32/runnable-ld.so"}
 },
 "files": {
   "libpthread.so.5055067a" : {
     "x86-64": {"url": "lib64/libpthread.so.5055067a"},
     "x86-32": {"url": "lib32/libpthread.so.5055067a"}
   },
   "libppapi_cpp.so" : {
     "x86-64": {"url": "lib64/libppapi_cpp.so"},
     "x86-32": {"url": "lib32/libppapi_cpp.so"}
   },
   "libstdc++.so.6" : {
     "x86-64": {"url": "lib64/libstdc++.so.6"},
     "x86-32": {"url": "lib32/libstdc++.so.6"}
   },
   "libm.so.5055067a" : {  
     "x86-64": {"url": "lib64/libm.so.5055067a"},
     "x86-32": {"url": "lib32/libm.so.5055067a"}
   },
   "libgcc_s.so.1" : {
     "x86-64": {"url": "lib64/libgcc_s.so.1"},
     "x86-32": {"url": "lib32/libgcc_s.so.1"}
   },
   "libc.so.5055067a" : {  
     "x86-64": {"url": "lib64/libc.so.5055067a"},
     "x86-32": {"url": "lib32/libc.so.5055067a"}
   },
   "main.nexe" : {
     "x86-64": {"url": "pi_generator_x86_64.nexe"},
     "x86-32": {"url": "pi_generator_x86_32.nexe"}
   }
 }
}


Couple things to point out:
  • The “program” section needs to now point to runnable-ld.so instead of your nexe.
  • Each DSO needs to be included in the “files” section.
  • The nexe that you run needs to be mapped to “main.nexe”. Currently there is no way to tell runnable-ld.so to look for other files.
  • Since this is JSON, the order of the keys doesn’t matter.

General process for creating glibc nmf file

Note: These instructions are oriented towards Linux and Mac users. Windows users can execute equivalent commands, but will need to make minor changes.
  1. Compile the nexe using the same flags as you would for newlib
  2. Determine what DSOs are needed for this nexe.  One way is to use sel_ldr and runnable-ld.so.  Example:

sel_ldr_x86_64 -a lib64/runnable-ld.so --list --library-path lib64 ./my_program.nexe

libpthread.so.5055067a => lib64/libpthread.so.5055067a (0x000001030000)

libppapi_cpp.so => lib64/libppapi_cpp.so (0x000001090000)

libstdc++.so.6 => lib64/libstdc++.so.6 (0x0000010f0000)

libm.so.5055067a => lib64/libm.so.5055067a (0x0000011e0000)

libgcc_s.so.1 => lib64/libgcc_s.so.1 (0x000001270000)

libc.so.5055067a => lib64/libc.so.5055067a (0x0000012c0000)

NaClMain (0x000000020000)

    1. Note: You need to have a slash in the path to my_program.nexe, otherwise runnable-ld will search the PATH variable.
    2. On Windows, it is still necessary to use forward slashes to the nexe. This is because runnable-ld.so expects a posix-style path.
    3. You also need to include any file that you dlopen().  These libraries will not be detected by runnable-ld.  There’s not an easy way to find all these files automatically.  The user will need to provide this list manually.
  1. Create an nmf file that has these DSOs in the “files” section.  See example above.
  2. Either copy or set up symlinks to the x86_64-nacl/lib32 and x86_64-nacl/lib64 toolchain directories to the directory that has the nexe.  This isn’t absolutely necessary, but you need to somehow setup the web server to serve these DSOs.


Alternatively, it’s possible to generate a rough-draft list of required libraries using objdump like this:

$ x86_64-nacl-objdump -p hello_world_x86_64.nexe | grep NEEDED
 NEEDED               libpthread.so.5055067a
 NEEDED               libppapi_cpp.so
 NEEDED               libstdc++.so.6
 NEEDED               libm.so.5055067a
 NEEDED               libgcc_s.so.1
 NEEDED               libc.so.5055067a


(Windows users who don't have grep installed can omit piping the output to grep, but will need to manually search for lines starting with 'NEEDED')

Make sure to run objdump on all the dependencies (and their dependencies) to get a complete list.

Debugging

For general debugging, see Debugging Native Client Modules.  To debug, you need to launch Chrome from a terminal with some of these environment variables set:
  • NACL_SRPC_DEBUG=2
  • NACL_PLUGIN_DEBUG=2
  • NACL_DEBUG_ENABLE=1
  • NACL_PPAPI_PROXY_DEBUG=2
  • PPAPI_BROWSER_DEBUG=1
  • NACLVERBOSITY=4

Troubleshooting

If the nexe isn’t loading, look for error messages in the terminal.  Here are some examples:
  • /main.nexe: error while loading shared libraries: /main.nexe: failed to allocate code and data space for executable:  The nexe may not have been compiled correctly. For example, nexe is statically linked.  Try cleaning and recompiling with glibc toolchain.
  • /main.nexe: error while loading shared libraries: libpthread.so.5055067a: cannot open shared object file: Permission denied:  This could be caused by having the wrong path set in the nmf file.  Double-check that the nmf file is correct.
  • /main.nexe: error while loading shared libraries: /main.nexe: cannot open shared object file: No such file or directory: If there are no obvious problems with your main.nexe entry in the manifest, check Wrench -> Tools -> Developer Tools -> Network tab and see where the main.nexe is being requested from.  
  • NaCl module load failed: ELF executable text/rodata segment has wrong starting address: This error happens when using a regular newlib-style nmf file instead of a glibc-style nmf file (as shown above).
Also, check the JavaScript error console.
Comments