一、啟動GDB調試
使用 GDB 調試程序一般有三種方式: gdb filename gdb attach pid gdb filename corename
1、直接調試目標程序
2、附加進程
3、調試 core 文件
各個參數的說明如下: 參數名稱 參數含義(英文) 參數含義(中文) %p insert pid into filename 添加 pid 到 core 文件名中 %u insert current uid into filename 添加當前 uid 到 core 文件名中 %g insert current gid into filename 添加當前 gid 到 core 文件名中 %s insert signal that caused the coredump into the filename 添加導致產生 core 的信號到 core 文件名中 %t insert UNIX time that the coredump occurred into filename 添加 core 文件生成時間(UNIX)到 core 文件名中 %h insert hostname where the coredump happened into filename 添加主機名到 core 文件名中 %e insert coredumping executable name into filename 添加程序名到 core 文件名中
二、GDB 常用的調試命令概覽
先給出一個常用命令的列表,後面會結合具體的例子詳細介紹每個命令的用法。
命令名稱 命令縮寫 命令說明 run r 運行一個程序 continue c 讓暫停的程序繼續運行 next n 運行到下一行 step s 如果有調用函數,進入調用的函數內部,相當於 step into until u 運行到指定行停下來 finish fi 結束當前調用函數,到上一層函數調用處 return return 結束當前調用函數並返回指定值,到上一層函數調用處 jump j 將當前程序執行流跳轉到指定行或地址 print p 列印變量或寄存器值 backtrace bt 查看當前線程的調用堆棧 frame f 切換到當前調用線程的指定堆棧,具體堆棧通過堆棧序號指定 thread thread 切換到指定線程 break b 添加斷點 tbreak tb 添加臨時斷點 delete del 刪除斷點 enable enable 啟用某個斷點 disable disable 禁用某個斷點 watch watch 監視某一個變量或內存地址的值是否發生變化 list l 顯示源碼 info info 查看斷點 / 線程等信息 ptype ptype 查看變量類型 disassemble dis 查看彙編代碼 set args 設置程序啟動命令行參數 show args 查看設置的命令行參數
三、GDB 常用命令詳解
本課的核心內容如下:
run 命令 continue 命令 break 命令 backtrace 與 frame 命令 info break、enable、disable 和 delete 命令 list 命令 print 和 ptype 命令
為了結合實踐,這裡以調試 Redis 源碼為例來介紹每一個命令,先介紹一些常用命令的基礎用法,某些命令的高級用法會在後面講解。 Redis 源碼下載與 debug 版本編譯 Redis 的最新源碼下載地址可以在 Redis 官網獲得,使用 wget 命令將 Redis 源碼文件下載下來:
[root@localhost gdbtest]# wget http://download.redis.io/releases/redis-4.0.11.tar.gz –2018-09-08 13:08:41– http://download.redis.io/releases/redis-4.0.11.tar.gz Resolving download.redis.io (download.redis.io)… 109.74.203.151 Connecting to download.redis.io (download.redis.io)|109.74.203.151|:80… connected. HTTP request sent, awaiting response… 200 OK Length: 1739656 (1.7M) [application/x-gzip] Saving to: 『redis-4.0.11.tar.gz』 54% [======================> ] 940,876 65.6KB/s eta 9s
解壓:
[root@localhost gdbtest]# tar zxvf redis-4.0.11.tar.gz
進入生成的 redis-4.0.11 目錄使用 makefile 命令進行編譯 為了方便調試,我們需要生成調試符號並且關閉編譯器優化選項,操作如下:
[root@localhost gdbtest]# cd redis-4.0.11 [root@localhost redis-4.0.11]# make CFLAGS=」-g -O0」 -j 4
注意:由於 redis 是純 C 項目,使用的編譯器是 gcc,因而這裡設置編譯器的選項時使用的是 CFLAGS 選項;如果項目使用的語言是 C++,那麼使用的編譯器一般是 g++,相對應的編譯器選項是 CXXFLAGS。這點請讀者注意區別。 另外,這裡 makefile 使用了 -j 選項,其值是 4,表示開啟 4 個進程同時編譯,加快編譯速度。 編譯成功後,會在 src 目錄下生成多個可執行程序,其中 redis-server 和 redis-cli 是需要調試的程序。 進入 src 目錄,使用 GDB 啟動 redis-server 這個程序:
[root@localhost src]# gdb redis-server Reading symbols from /root/gdbtest/redis-4.0.11/src/redis-server…done.
1、run 命令
默認情況下,前面的課程中我們說 gdb filename 命令只是附加的一個調試文件,並沒有啟動這個程序,需要輸入 run 命令(簡寫為 r)啟動這個程序:
(gdb) r Starting program: /root/gdbtest/redis-4.0.11/src/redis-server [Thread debugging using libthread_db enabled] Using host libthread_db library 「/lib64/libthread_db.so.1」. 46455:C 08 Sep 13:43:43.957 # oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo 46455:C 08 Sep 13:43:43.957 # Redis version=4.0.11, bits=64, commit=00000000, modified=0, pid=46455, just started 46455:C 08 Sep 13:43:43.957 # Warning: no config file specified, using the default config. In order to specify a config file use /root/gdbtest/redis-4.0.11/src/redis-server /path/to/redis.conf 46455:M 08 Sep 13:43:43.957 * Increased maximum number of open files to 10032 (it was originally set to 1024). [New Thread 0x7ffff07ff700 (LWP 46459)] [New Thread 0x7fffefffe700 (LWP 46460)] [New Thread 0x7fffef7fd700 (LWP 46461)] . _.-__ ''-.__.- .. 」-. Redis 4.0.11 (00000000/0) 64 bit .-.-```. ```\/ _.,_ ''-._( ' , .-` | `, ) Running in standalone mode|`-._`-...-` __...-.-._|』_.-'| Port: 6379|-._ ._ / _.-' | PID: 46455 -._ -._-./ .-』 .-』 |-._-._ -.__.-' _.-'_.-'||-._-._ _.-'_.-' | http://redis.io -._ -._-._.-『.-』 _.-』 |-._-._ -.__.-' _.-'_.-'||-._-._ _.-'_.-' | -._ -._-._.-『.-』 _.-』 -._-._.-』 .-』 -._ _.-' -.__.-『46455:M 08 Sep 13:43:43.965 # WARNING: The TCP backlog setting of 511 cannot be enforced because /proc/sys/net/core/somaxconn is set to the lower value of 128. 46455:M 08 Sep 13:43:43.965 # Server initialized 46455:M 08 Sep 13:43:43.965 # WARNING overcommit_memory is set to 0! Background save may fail under low memory condition. To fix this issue add 『vm.overcommit_memory = 1』 to /etc/sysctl.conf and then reboot or run the command 『sysctl vm.overcommit_memory=1』 for this to take effect. 46455:M 08 Sep 13:43:43.965 # WARNING you have Transparent Huge Pages (THP) support enabled in your kernel. This will create latency and memory usage issues with Redis. To fix this issue run the command 『echo never > /sys/kernel/mm/transparent_hugepage/enabled』 as root, and add it to your /etc/rc.local in order to retain the setting after a reboot. Redis must be restarted after THP is disabled. 46455:M 08 Sep 13:43:43.965 * Ready to accept connections
這就是 redis-server 啟動界面,假設程序已經啟動,再次輸入 run 命令則是重啟程序。我們在 GDB 界面按 Ctrl + C 快捷鍵讓 GDB 中斷下來,再次輸入 r 命令,GDB 會詢問我們是否重啟程序,輸入 yes 確認重啟。
^C Program received signal SIGINT, Interrupt. 0x00007ffff73ee923 in epoll_wait () from /lib64/libc.so.6 Missing separate debuginfos, use: debuginfo-install glibc-2.17-196.el7_4.2.x86_64 (gdb) r The program being debugged has been started already. Start it from the beginning? (y or n) yes Starting program: /root/gdbtest/redis-4.0.11/src/redis-server
2、continue 命令
當 GDB 觸發斷點或者使用 Ctrl + C 命令中斷下來後,想讓程序繼續運行,只要輸入 continue 命令即可(簡寫為 c)。當然,如果 continue 命令繼續觸發斷點,GDB 就會再次中斷下來。
^C Program received signal SIGINT, Interrupt. 0x00007ffff73ee923 in epoll_wait () from /lib64/libc.so.6 (gdb) c Continuing.
3、break 命令
break 命令(簡寫為 b)即我們添加斷點的命令,可以使用以下方式添加斷點:
break functionname,在函數名為 functionname 的入口處添加一個斷點; break LineNo,在當前文件行號為 LineNo 處添加一個斷點; break filename:LineNo,在 filename 文件行號為 LineNo 處添加一個斷點。
這三種方式都是我們常用的添加斷點的方式。舉個例子,對於一般的 Linux 程序來說,main() 函數是程序入口函數,redis-server 也不例外,我們知道了函數的名字,就可以直接在 main() 函數處添加一個斷點:
(gdb) b main Breakpoint 1 at 0x423450: file server.c, line 3709.
添加好了以後,使用 run 命令重啟程序,就可以觸發這個斷點了,GDB 會停在斷點處。
(gdb) r The program being debugged has been started already. Start it from the beginning? (y or n) y Starting program: /root/gdbtest/redis-4.0.11/src/redis-server [Thread debugging using libthread_db enabled] Using host libthread_db library 「/lib64/libthread_db.so.1」.Breakpoint 1, main (argc=1, argv=0x7fffffffe648) at server.c:3709 3709 int main(int argc, char **argv) { (gdb)
redis-server 默認埠號是 6379 ,我們知道這個埠號肯定是通過作業系統的 socket API bind() 函數創建的,通過文件搜索,找到調用這個函數的文件,其位於 anet.c 441 行。
我們使用 break 命令在這個地方加一個斷點:
(gdb) b anet.c:441 Breakpoint 3 at 0x426cf0: file anet.c, line 441
由於程序綁定埠號是 redis-server 啟動時初始化的,為了能觸發這個斷點,再次使用 run 命令重啟下這個程序,GDB 第一次會觸發 main() 函數處的斷點,輸入 continue 命令繼續運行,接著觸發 anet.c:441 處的斷點:
(gdb) r The program being debugged has been started already. Start it from the beginning? (y or n) y Starting program: /root/gdbtest/redis-4.0.11/src/redis-server [Thread debugging using libthread_db enabled] Using host libthread_db library 「/lib64/libthread_db.so.1」.Breakpoint 1, main (argc=1, argv=0x7fffffffe648) at server.c:3709 3709 int main(int argc, char **argv) { (gdb) c Continuing. 46699:C 08 Sep 15:30:31.403 # oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo 46699:C 08 Sep 15:30:31.403 # Redis version=4.0.11, bits=64, commit=00000000, modified=0, pid=46699, just started 46699:C 08 Sep 15:30:31.403 # Warning: no config file specified, using the default config. In order to specify a config file use /root/gdbtest/redis-4.0.11/src/redis-server /path/to/redis.conf 46699:M 08 Sep 15:30:31.404 * Increased maximum number of open files to 10032 (it was originally set to 1024).Breakpoint 3, anetListen (err=0x746bb0 <server+560> 「」, s=10, sa=0x75edb0, len=28, backlog=511) at anet.c:441 441 if (bind(s,sa,len) == -1) { (gdb)
anet.c:441 處的代碼如下:
現在斷點停在第 441 行,所以當前文件就是 anet.c,可以直接使用「break 行號」添加斷點。例如,可以在第 444 行、450 行、452 行分別加一個斷點,看看這個函數執行完畢後走哪個 return 語句退出,則可以執行:
440 static int anetListen(char *err, int s, struct sockaddr *sa, socklen_t len, int backlog) { 441 if (bind(s,sa,len) == -1) { 442 anetSetError(err, 「bind: %s」, strerror(errno)); 443 close(s); 444 return ANET_ERR; (gdb) l 445 } 446 447 if (listen(s, backlog) == -1) { 448 anetSetError(err, 「listen: %s」, strerror(errno)); 449 close(s); 450 return ANET_ERR; 451 } 452 return ANET_OK; 453 } 454 (gdb) b 444 Breakpoint 3 at 0x426cf5: file anet.c, line 444. (gdb) b 450 Breakpoint 4 at 0x426d06: file anet.c, line 450. (gdb) b 452 Note: breakpoint 4 also set at pc 0x426d06. Breakpoint 5 at 0x426d06: file anet.c, line 452. (gdb)
添加好這三個斷點以後,我們使用 continue 命令繼續運行程序,發現程序運行到第 452 行中斷下來(即觸發 Breakpoint 5):
(gdb) c Continuing.Breakpoint 5, anetListen (err=0x746bb0 <server+560> 「」, s=10, sa=0x7e34e0, len=16, backlog=511) at anet.c:452 452 return ANET_OK;
說明 redis-server 綁定埠號並設置偵聽(listen)成功,我們可以再打開一個 SSH 窗口,驗證一下,發現 6379 埠確實已經處於偵聽狀態了:
[root@localhost src]# lsof -i -Pn | grep redis redis-ser 46699 root 10u IPv6 245844 0t0 TCP *:6379 (LISTEN)
4、backtrace 與 frame 命令
backtrace 命令(簡寫為 bt)用來查看當前調用堆棧。接上,redis-server 現在中斷在 anet.c:452 行,可以通過 backtrace 命令來查看當前的調用堆棧:
(gdb) bt#0 anetListen (err=0x746bb0 <server+560> "", s=10, sa=0x7e34e0, len=16, backlog=511) at anet.c:452#1 0x0000000000426e35 in _anetTcpServer (err=err@entry=0x746bb0 <server+560> "", port=port@entry=6379, bindaddr=bindaddr@entry=0x0, af=af@entry=10, backlog=511)at anet.c:487#2 0x000000000042793d in anetTcp6Server (err=err@entry=0x746bb0 <server+560> "", port=port@entry=6379, bindaddr=bindaddr@entry=0x0, backlog=511)at anet.c:510#3 0x000000000042b0bf in listenToPort (port=6379, fds=fds@entry=0x746ae4 <server+356>, count=count@entry=0x746b24 <server+420>) at server.c:1728#4 0x000000000042fa77 in initServer () at server.c:1852#5 0x0000000000423803 in main (argc=1, argv=0x7fffffffe648) at server.c:3862(gdb)
這裡一共有 6 層堆棧,最頂層是 main() 函數,最底層是斷點所在的 anetListen() 函數,堆棧編號分別是 #0 ~ #5 ,如果想切換到其他堆棧處,可以使用 frame 命令(簡寫為 f),該命令的使用方法是「frame 堆棧編號(編號不加 #)」。在這裡依次切換至堆棧頂部,然後再切換回 #0 練習一下:
(gdb) f 1#1 0x0000000000426e35 in _anetTcpServer (err=err@entry=0x746bb0 <server+560> "", port=port@entry=6379, bindaddr=bindaddr@entry=0x0, af=af@entry=10, backlog=511)at anet.c:487487 if (anetListen(err,s,p->ai_addr,p->ai_addrlen,backlog) == ANET_ERR) s = ANET_ERR;(gdb) f 2#2 0x000000000042793d in anetTcp6Server (err=err@entry=0x746bb0 <server+560> "", port=port@entry=6379, bindaddr=bindaddr@entry=0x0, backlog=511)at anet.c:510510 return _anetTcpServer(err, port, bindaddr, AF_INET6, backlog);(gdb) f 3#3 0x000000000042b0bf in listenToPort (port=6379, fds=fds@entry=0x746ae4 <server+356>, count=count@entry=0x746b24 <server+420>) at server.c:17281728 fds[*count] = anetTcp6Server(server.neterr,port,NULL,(gdb) f 4#4 0x000000000042fa77 in initServer () at server.c:18521852 listenToPort(server.port,server.ipfd,&server.ipfd_count) == C_ERR)(gdb) f 5#5 0x0000000000423803 in main (argc=1, argv=0x7fffffffe648) at server.c:38623862 initServer();(gdb)
通過查看上面的各個堆棧,可以得出這裡的調用層級關係,即:
main() 函數在第 3862 行調用了 initServer() 函數 initServer() 在第 1852 行調用了 listenToPort() 函數 listenToPort() 在第 1728 行調用了 anetTcp6Server() 函數 anetTcp6Server() 在第 510 行調用了 _anetTcpServer() 函數 _anetTcpServer() 函數在第 487 行調用了 anetListen() 函數 當前斷點正好位於 anetListen() 函數中
5、info break、enable、disable 和 delete 命令
在程序中加了很多斷點,而我們想查看加了哪些斷點時,可以使用 info break 命令(簡寫為 info b):
(gdb) info b Num Type Disp Enb Address What 1 breakpoint keep y 0x0000000000423450 in main at server.c:3709 breakpoint already hit 1 time 2 breakpoint keep y 0x000000000049c1f0 in _redisContextConnectTcp at net.c:267 3 breakpoint keep y 0x0000000000426cf0 in anetListen at anet.c:441 breakpoint already hit 1 time 4 breakpoint keep y 0x0000000000426d05 in anetListen at anet.c:444 breakpoint already hit 1 time 5 breakpoint keep y 0x0000000000426d16 in anetListen at anet.c:450 breakpoint already hit 1 time 6 breakpoint keep y 0x0000000000426d16 in anetListen at anet.c:452 breakpoint already hit 1 time
通過上面的內容片段可以知道,目前一共增加了 6 個斷點,除了斷點 2 以外,其他的斷點均被觸發一次,其他信息比如每個斷點的位置(所在的文件和行號)、內存地址、斷點啟用和禁用狀態信息也一目了然。如果我們想禁用某個斷點,使用「disable 斷點編號」就可以禁用這個斷點了,被禁用的斷點不會再被觸發;同理,被禁用的斷點也可以使用「enable 斷點編號」重新啟用。
(gdb) disable 1 (gdb) info b Num Type Disp Enb Address What 1 breakpoint keep n 0x0000000000423450 in main at server.c:3709 breakpoint already hit 1 time 2 breakpoint keep y 0x000000000049c1f0 in _redisContextConnectTcp at net.c:267 3 breakpoint keep y 0x0000000000426cf0 in anetListen at anet.c:441 breakpoint already hit 1 time 4 breakpoint keep y 0x0000000000426d05 in anetListen at anet.c:444 breakpoint already hit 1 time 5 breakpoint keep y 0x0000000000426d16 in anetListen at anet.c:450 breakpoint already hit 1 time 6 breakpoint keep y 0x0000000000426d16 in anetListen at anet.c:452 breakpoint already hit 1 time
使用 disable 1 以後,第一個斷點的 Enb 一欄的值由 y 變成 n,重啟程序也不會再次觸發:
(gdb) r The program being debugged has been started already. Start it from the beginning? (y or n) y Starting program: /root/gdbtest/redis-4.0.11/src/redis-server [Thread debugging using libthread_db enabled] Using host libthread_db library 「/lib64/libthread_db.so.1」. 46795:C 08 Sep 16:15:55.681 # oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo 46795:C 08 Sep 16:15:55.681 # Redis version=4.0.11, bits=64, commit=00000000, modified=0, pid=46795, just started 46795:C 08 Sep 16:15:55.681 # Warning: no config file specified, using the default config. In order to specify a config file use /root/gdbtest/redis-4.0.11/src/redis-server /path/to/redis.conf 46795:M 08 Sep 16:15:55.682 * Increased maximum number of open files to 10032 (it was originally set to 1024).Breakpoint 3, anetListen (err=0x746bb0 <server+560> 「」, s=10, sa=0x75edb0, len=28, backlog=511) at anet.c:441 441 if (bind(s,sa,len) == -1) {
如果 disable 命令和 enable 命令不加斷點編號,則分別表示禁用和啟用所有斷點:
(gdb) disable (gdb) info b Num Type Disp Enb Address What 1 breakpoint keep n 0x0000000000423450 in main at server.c:3709 2 breakpoint keep n 0x000000000049c1f0 in _redisContextConnectTcp at net.c:267 3 breakpoint keep n 0x0000000000426cf0 in anetListen at anet.c:441 breakpoint already hit 1 time 4 breakpoint keep n 0x0000000000426d05 in anetListen at anet.c:444 5 breakpoint keep n 0x0000000000426d16 in anetListen at anet.c:450 6 breakpoint keep n 0x0000000000426d16 in anetListen at anet.c:452 (gdb) enable (gdb) info b Num Type Disp Enb Address What 1 breakpoint keep y 0x0000000000423450 in main at server.c:3709 2 breakpoint keep y 0x000000000049c1f0 in _redisContextConnectTcp at net.c:267 3 breakpoint keep y 0x0000000000426cf0 in anetListen at anet.c:441 breakpoint already hit 1 time 4 breakpoint keep y 0x0000000000426d05 in anetListen at anet.c:444 5 breakpoint keep y 0x0000000000426d16 in anetListen at anet.c:450 6 breakpoint keep y 0x0000000000426d16 in anetListen at anet.c:452 (gdb)
使用「delete 編號」可以刪除某個斷點,如 delete 2 3 則表示要刪除的斷點 2 和斷點 3:
(gdb) delete 2 3 (gdb) info b Num Type Disp Enb Address What 1 breakpoint keep y 0x0000000000423450 in main at server.c:3709 4 breakpoint keep y 0x0000000000426d05 in anetListen at anet.c:444 5 breakpoint keep y 0x0000000000426d16 in anetListen at anet.c:450 6 breakpoint keep y 0x0000000000426d16 in anetListen at anet.c:452
同樣的道理,如果輸入 delete 不加命令號,則表示刪除所有斷點。