Debugging x86-32 Native Client Modules with nacl-gdb


Quick Start Notes

Where can I find it?

The 32bit Native Client GDB can be only built on Linux in a NaCl source checkout:
cd native_client/tools
make gdb

Overview

When debugging NaCl programs nacl-gdb works pretty much like you're use to. There are two modes in nacl-gdb. A NaCl program can either be in service runtime (see Glossary) code or in native client code. The prompt shows which mode the program is in, "(sr-gdb)" or "(nc-gdb)".

Remember, apropos is your friend. If you can't remember the name of a command, type apropos <word> and hopefully you'll see the command you are looking for in the output. For nacl commands, try apropos nacl.

Simple Example

Assuming you have downloaded the SDK. Though downloading or building the modified GNU toolchain should be enough. You will also need to fetch and build Native Client Sources to run this example. Note: the hello_world example from the SDK examples is not suitable for command-line debugging. Although you can attach to the browser process to debug it, for this simple demonstration we take a more trivial hello_world module that only outputs data to stdout.

shell> cd native_client
shell> ./scons --mode=dbg-linux,nacl platform=x86-32 sdl=none --verbose run_hello_world_test nacl_ccflags=-g
[...]
shell> /path/to/nativeclient-sdk/toolchain/linux_x86/bin/nacl-gdb --loader scons-out/dbg-linux-x86-32/staging/sel_ldr scons-out/nacl-x86-32/obj/tests/hello_world/hello_world.nexe
GNU gdb 6.8 20100521 (Native Client r2316)
Copyright (C) 2008 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.  Type "show copying"
and "show warranty" for details.
This GDB was configured as "--host=i686-pc-linux-gnu --target=nacl"...
(nc-gdb) b hello_world
Breakpoint 1 at 0x20340: file tests/hello_world/hello_world.c, line 13.
(nc-gdb) r
Starting program: /path/to/native_client/scons-out/nacl-x86-32/obj/tests/hello_world/hello_world.nexe 
warning: Lowest section in system-supplied DSO at 0xffffe000 is .hash at ffffe0b4
[Thread debugging using libthread_db enabled]
[New Thread 0xf7ae7a50 (LWP 17001)]
[New Thread 0xb7a9cb90 (LWP 17004)]
[Switching to Thread 0xb7a9cb90 (LWP 17004)]

Breakpoint 1, hello_world () at tests/hello_world/hello_world.c:13
13      void hello_world() {
(nc-gdb) disp/i $pc
1: x/i $pc
0x20340 <hello_world>:  sub    $0x1c,%esp
(nc-gdb) si
14        printf("Hello, World!\n");
1: x/i $pc
0x20343 <hello_world+3>:        movl   $0x30000,(%esp)
(nc-gdb) 
0x0002034a      14        printf("Hello, World!\n");
1: x/i $pc
0x2034a <hello_world+10>:       jmp    0x2035b <hello_world+27>
(nc-gdb) 
0x0002035b      14        printf("Hello, World!\n");
1: x/i $pc
0x2035b <hello_world+27>:       call   0x20b20 <puts>
(nc-gdb) 
0x00020b20 in puts ()
1: x/i $pc
0x20b20 <puts>: push   %ebp
(nc-gdb) c
Continuing.
Hello, World!
[17001,3081358224:17:28:35.140413] Exit syscall handler: 0

Program exited normally.
(nc-gdb) 

Known Issues

Examining core files

Examining core files of Native Client programs has not been well tested yet.

Hardware watchpoints do not work

Hardware watchpoints in native-client mode currently do not work. They should work in service-runtime mode.

Thread Local Storage (TLS)

Access to thread local variables in native-client mode does not currently work.

They should work in service-runtime mode.

Transition between sr and nc modes

During the transition between service-runtime and native-client modes GDB can get confused about what the current mode is. The symptom is memory accesses give errors. If one is stepi-ing, just continue to stepi a few more instructions until the transition is complete.

Incomplete display of breakpoint symbols

When stopped in one mode, "info breakpoints" won't print symbolic addresses of breakpoints in the other mode.

longjmp

There's a FIXME in the code regarding restoring longjmp breakpoints. So assume stepping over a longjmp may not work.

Native Client specific features

Overview

From GDB's point of view, when running a Native Client program there are actually two programs: sel_ldr and the Native Client program itself. The Native Client program runs in its own protected space, and assumes that all of its symbols won't collide with sel_ldr (the Service Runtime). Therefore GDB needs to keep the symbols of each separate. It does this by becoming modal. At any particular point in a thread's execution, it is either in the Service Runtime side of things or it is in the Native Client side of things. GDB tracks this state (lazily to not affect performance).

To assist the user in knowing what state the program is currently in, and thus for example what set of symbols are accessible, nacl-gdb includes the current state in the prompt. When a thread is stopped in Native Client mode the prompt is (nc-gdb). When a thread is stopped in Service Runtime mdoe the prompt is (sr-gdb). As one switches among threads GDB updates its record of what state the thread is in, and updates the prompt accordingly.

New startup options

--loader

Use the --loader option to specify a different path for sel_ldr. Example:

gdb --loader=/path/to/my/sel_ldr hello.nexe

Commands

nacl apply-runtime command [args ...]

Temporarily switch to service-runtime mode and then run command there.

This is useful if you are sitting at a (nc-gdb) prompt and want to run a command "as if" you are sitting at a (sr-gdb) prompt. Examples are examining service-runtime state or setting a breakpoint in sel_ldr.

nacl apply command [args ...]

Temporarily switch to native-client mode and then run command there.

This is useful if you are sitting at a (sr-gdb) prompt and want to run a command "as if" you are sitting at a (nc-gdb) prompt. Examples are examining the state of the native-client program or setting a breakpoint there.

Example:

shell> /path/to/nacl-sdk/toolchain/linux_x86/bin/nacl-gdb --loader scons-out/dbg-linux-x86-32/staging/sel_ldr scons-out/nacl-x86-32/obj/tests/hello_world/hello_world.nexe
GNU gdb 6.8 20100521 (Native Client r2316)
Copyright (C) 2008 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.  Type "show copying"
and "show warranty" for details.
This GDB was configured as "--host=i686-pc-linux-gnu --target=nacl"...
(nc-gdb) nacl apply-runtime b main
Breakpoint 1 at 0x804a7e8: file src/trusted/service_runtime/sel_main.c, line 177.
(nc-gdb) i b
Num     Type           Disp Enb Address       What
1       breakpoint     keep y   sr:0x0804a7e8 src/trusted/service_runtime/sel_main.c:177
(nc-gdb) r
Starting program: /path/to/native_client/scons-out/nacl-x86-32/obj/tests/hello_world/hello_world.nexe 
warning: Lowest section in system-supplied DSO at 0xffffe000 is .hash at ffffe0b4
[Thread debugging using libthread_db enabled]
[New Thread 0xf7b81a50 (LWP 20808)]
[Switching to Thread 0xf7b81a50 (LWP 20808)]

Breakpoint 1, main (ac=2, av=0xffe45644) at src/trusted/service_runtime/sel_main.c:186
186       char                          *nacl_file = 0;
(sr-gdb) nacl apply break main
Breakpoint 2 at 0x20380: file tests/hello_world/hello_world.c, line 17.
(sr-gdb) i b
Num     Type           Disp Enb Address       What
1       breakpoint     keep y   sr:0x0804a804 in main at src/trusted/service_runtime/sel_main.c:177
        breakpoint already hit 1 time
2       breakpoint     keep y   nc:0x00020380 tests/hello_world/hello_world.c:17
(sr-gdb) 

set nacl service-runtime-args [args ...]

Set the arguments to sel_ldr, NOT including the NaCl program or its arguments. Use this to specify arguments for sel_ldr itself, e.g., -h d:D. Do not specify the -f argument, gdb supplies that itself.

NOTE: This option is only useful when debugging NaCl programs, i.e., when the "main" executable is a .nexe program. In other words, you started your debugging session with something like:

nacl-gdb program.nexe


Comments