FreeBSD Handbook : Kernel Debugging : On-line Kernel Debugging Using Remote GDB
Previous: On-line Kernel Debugging Using DDB
Next: Debugging a Console Driver

22.4. On-line Kernel Debugging Using Remote GDB

This feature has been supported since FreeBSD 2.2, and it's actually a very neat one.

GDB has already supported remote debugging for a long time. This is done using a very simple protocol along a serial line. Unlike the other methods described above, you will need two machines for doing this. One is the host providing the debugging environment, including all the sources, and a copy of the kernel binary with all the symbols in it, and the other one is the target machine that simply runs a similar copy of the very same kernel (but stripped of the debugging information).

You should configure the kernel in question with config -g, include DDB into the configuration, and compile it as usual. This gives a large blurb of a binary, due to the debugging information. Copy this kernel to the target machine, strip the debugging symbols off with strip -x, and boot it using the -d boot option. Connect the first serial line of the target machine to any serial line of the debugging host. Now, on the debugging machine, go to the compile directory of the target kernel, and start gdb:

% gdb -k kernel
GDB is free software and you are welcome to distribute copies of it
 under certain conditions; type "show copying" to see the conditions.
There is absolutely no warranty for GDB; type "show warranty" for details.
GDB 4.16 (i386-unknown-freebsd), 
Copyright 1996 Free Software Foundation, Inc...
(kgdb) 

Initialize the remote debugging session (assuming the first serial port is being used) by:

(kgdb) target remote /dev/cuaa0

Now, on the target host (the one that entered DDB right before even starting the device probe), type:

Debugger("Boot flags requested debugger")
Stopped at	Debugger+0x35: movb	$0, edata+0x51bc
db> gdb

DDB will respond with:

Next trap will enter GDB remote protocol mode

Every time you type ``gdb'', the mode will be toggled between remote GDB and local DDB. In order to force a next trap immediately, simply type ``s'' (step). Your hosting GDB will now gain control over the target kernel:

Remote debugging using /dev/cuaa0
Debugger (msg=0xf01b0383 "Boot flags requested debugger")
    at ../../i386/i386/db_interface.c:257
(kgdb) 

You can use this session almost as any other GDB session, including full access to the source, running it in gud-mode inside an Emacs window (which gives you an automatic source code display in another Emacs window) etc.

Remote GDB can also be used to debug LKMs. First build the LKM with debugging symbols:

# cd /usr/src/lkm/linux
# make clean; make COPTS=-g

Then install this version of the module on the target machine, load it and use modstat to find out where it was loaded:

# linux
# modstat
Type     Id Off Loadaddr Size Info     Rev Module Name
EXEC      0   4 f5109000 001c f510f010   1 linux_mod

Take the load address of the module and add 0x20 (probably to account for the a.out header). This is the address that the module code was relocated to. Use the add-symbol-file command in GDB to tell the debugger about the module:

(kgdb) add-symbol-file /usr/src/lkm/linux/linux_mod.o 0xf5109020
add symbol table from file "/usr/src/lkm/linux/linux_mod.o" at
text_addr = 0xf5109020?
(y or n) y
(kgdb)

You now have access to all the symbols in the LKM.


FreeBSD Handbook : Kernel Debugging : On-line Kernel Debugging Using Remote GDB
Previous: On-line Kernel Debugging Using DDB
Next: Debugging a Console Driver