Embedded Development Essential: A Complete Guide to GDB Debugging

I. GDB Basics: Core Commands and Debugging Workflow

1.1 Preparations: Add Debug Information During Compilation

GDB debugging relies on debug information (line numbers, variable names, function names, etc.) in the program. The -g parameter must be added during compilation:

# Take doubly linked list code as an example to generate an executable with debug information
gcc main.c doulinklist.c -o link_test -Wall -g

• -g: Generates debug information (core parameter);
• -Wall: Displays all warnings to troubleshoot potential issues in advance;
• link_test: Name of the generated executable file.

1.2 Cheat Sheet for Common Commands (High-Frequency in Embedded Development)

Command	Description	Embedded Scenario Example (Based on Linked List Program)
gdb <executable>	Starts GDB debugging	gdb ./link_test
r/run [args]	Runs the program, supports passing command-line arguments	r (run directly), r 10 20 (simulate passing args to embedded device)
l/list [line]	Displays source code (10 lines by default; supports specifying line numbers/function names)	l (display code near main), l ReverseDouLinkList (display reverse function)
b/break	Sets breakpoints (by line number, function name, or multi-file specification)	b 50, b doulinklist.c:80, b InsertTailDouLinkList
n/next	Executes step by step, skipping function calls (quickly traverse code)	Use n in loops to execute quickly without entering library functions
s/step	Executes step by step, entering custom function bodies (deep dive into function logic)	s to enter FindKthFromTail (function to find the k-th node from the end)
p/print	Views variables, pointers, and structure contents	p dl->clen (linked list length), p slow->data.name (node data)
display <var>	Continuously displays variables, auto-refreshes after each step (no need to repeatedly enter p)	display tmp->prev (monitor pointer pointing changes)
undisplay <num>	Cancels continuous display (a number is shown after display)	undisplay 1 (cancel the first monitoring item)
c/continue	Continues running to the next breakpoint (break out of loops, skip irrelevant code)	Use c to locate the next breakpoint after finding one issue
return	Forces exit from the current function and returns to the caller (no need to complete function execution)	Exit early when debugging linked list insertion functions
where/bt	Views stack trace and displays function call relationships (reverse order)	Use bt to locate the error line after a segmentation fault
q/quit	Exits GDB debugging	q (end debugging)
layout src	Enables source code visualization interface (intuitively view code execution position)	Enter during debugging and use with step-by-step execution
layout reg	Displays register status (commonly used in embedded low-level debugging)	Check register values when troubleshooting pointer exceptions
x/[n][f][u] <addr>	Views memory data (n=count, f=format, u=unit)	x/10xw dl->head (display 10 4-byte memory blocks in hexadecimal)

1.3 Basic Debugging Workflow (Example: Logic Error in Doubly Linked List Reversal)

Scenario: Abnormal output order after doubly linked list reversal; need to troubleshoot the ReverseDouLinkList function

1. Start GDB and load the program:

   gdb ./link_test
   layout src  # Enable source code visualization (optional, more intuitive)
   

2. Set a breakpoint (at the entry of the reversal function):

   b ReverseDouLinkList  # Set breakpoint by function name
   # Or set by line number: b doulinklist.c:120 (assuming the reversal function starts at line 120)
   

3. Run the program:

   r  # The program will pause at the breakpoint
   

4. Step into the function:

   s  # Enter the interior of the ReverseDouLinkList function
   

5. View key variables:

   p curr->data.name  # Check current node data (confirm if it is the linked list head node)
   display curr->prev  # Continuously monitor prev pointer changes
   display curr->next  # Continuously monitor next pointer changes
   

6. Step through execution and observe:

   n  # Execute line by line and check if prev and next pointers are swapped correctly
   # If you find that curr->next is not updated, you can correct the code immediately
   

7. Continue to the next breakpoint:

   c  # If there are multiple breakpoints, run to the next pause point
   

8. Exit debugging:

   q  # After debugging, modify the code based on troubleshooting results
   

1.4 Segmentation Fault Debugging (High-Frequency Error in Embedded Development)

A segmentation fault (Segmentation fault) is the most common error in embedded development, usually caused by wild pointers, memory out-of-bounds, or null pointer dereferencing. The debugging steps are as follows:

Scenario: Segmentation fault triggered during linked list insertion (Segmentation fault (core dumped))

1. Compile with debug information (same as 1.1);
2. Start GDB and run the program (no need to set breakpoints in advance):

   gdb ./link_test
   r  # Run the program directly to trigger the segmentation fault
   

3. View stack trace to locate the error line:

   bt  # Output the error call stack
   

   Example output:

   #0  0x0000555555555289 in InsertTailDouLinkList (dl=0x5555555592a0, data=0x7fffffffe150) at doulinklist.c:86
   #1  0x00005555555548e6 in main (argc=1, argv=0x7fffffffe2a8) at main.c:15
   

   From the output, the segmentation fault occurs at line 86 of doulinklist.c, triggered by the InsertTailDouLinkList call at line 15 of the main function.
4. View code at the error line:

   l 86  # Display code near line 86
   

   Root cause found: The original loop condition while (tmp->next == NULL) caused tmp to point to NULL in advance, and the subsequent tmp->next = newnode triggered a null pointer dereference.
5. Fix the code: Change the loop condition to while (tmp->next != NULL), recompile, and run to resolve the issue.

II. Advanced GDB: Practical Operation of High-Frequency Features in Embedded Development

2.1 Conditional Breakpoints (Precisely Locate Errors in Specific Scenarios)

Feature Description

A regular breakpoint pauses every time at the specified location, while a conditional breakpoint only pauses when a specific condition is met. It is suitable for debugging:

• Specific iterations in loops (e.g., errors occurring in the 5th loop);
• Specific parameter scenarios (e.g., errors when inserting a node named “guanerge”);
• Boundary conditions (e.g., head insertion with pos=0, tail insertion with pos=clen).

Practical Scenario: Data Corruption When Inserting at pos=2 in the Linked List

1. Start GDB and load the program: gdb ./link_test;
2. Set a conditional breakpoint (pause only when pos=2 at the core logic line of the insertion function):

   # Assume the pointer binding logic for inserting nodes is at line 105 of doulinklist.c
   b doulinklist.c:105 if pos == 2
   

3. Run the program: r;
4. After the program pauses, view key information:

   p pos  # Confirm current pos=2
   p data->name  # Check if the inserted data is correct
   p tmp->data.name  # Check the previous node at the insertion position
   p newnode->prev  # Verify if the prev pointer is bound to tmp
   

5. Step through execution to observe logic: Use n to execute line by line and troubleshoot pointer binding errors.

Common Usage of Conditional Breakpoints

Scenario	GDB Command Example
Pause when loop variable i=3	b main.c:25 if i == 3 (i is the loop variable)
Pause when the inserted node’s gender is ‘f’	b doulinklist.c:90 if data->sex == 'f'
Pause when linked list length clen>5	b doulinklist.c:100 if dl->clen > 5
Pause when pointer tmp is not null	b doulinklist.c:85 if tmp != NULL

2.2 Variable Monitoring (Track Variable/Memory Changes)

Feature Description

The watch command monitors changes to variables or memory addresses. The program automatically pauses when the variable is modified, which is suitable for troubleshooting:

• Unexpected modification of variables (e.g., the linked list length clen increases or decreases inexplicably);
• Tampering of data pointed to by pointers;
• Abnormal changes to structure members.

Derived commands:

• rwatch <var>: Pauses only when the variable is read;
• awatch <var>: Pauses when the variable is read or modified (most commonly used).

Practical Scenario: Monitor Whether the Linked List Length clen is Updated Correctly

1. Start GDB and load the program: gdb ./link_test;
2. First run to the completion of linked list creation (to avoid monitoring uninitialized variables):

   b CreateDouLinkList  # Set a breakpoint at the end of the linked list creation function
   r
   c  # Run to the completion of linked list creation, where the dl pointer has been initialized
   

3. Monitor dl->clen (linked list length):

   watch dl->clen  # Monitor variable modification
   

4. Continue running the program to trigger linked list insertion/deletion operations:

   c
   

5. When dl->clen is modified, the program pauses and prompts the change:

   Hardware watchpoint 2: dl->clen
   
   Old value = 3
   New value = 4
   InsertTailDouLinkList (dl=0x5555555592a0, data=0x7fffffffe180) at doulinklist.c:98
   98      dl->clen++;
   

6. Troubleshoot the modification logic:
   • Check who modified the variable: bt (confirm it is the InsertTailDouLinkList function);
   • Verify if the modification is reasonable: p dl->clen (confirm if the new value meets expectations).

Notes

• Monitoring local variables: Set watch only after running into the function (when local variables take effect);
• Monitoring structure members: Ensure the structure pointer is not null (e.g., first confirm dl != NULL with p dl);
• Monitoring content pointed to by a pointer: watch *tmp (monitor memory data pointed to by pointer tmp);
• Monitoring array elements: watch arr[0] (monitor changes to the first element of the array).

2.3 Multi-Thread Debugging (Concurrent Scenarios in Embedded Development)

Embedded development often involves multi-threaded operations on shared resources (e.g., one thread inserting into a linked list and another deleting from it). GDB supports multi-thread debugging with the following core commands:

Command	Description	Example
info threads	Views the status of all threads (ID, running function, state)	info threads
thread <thread_id>	Switches to the thread with the specified ID	thread 2 (switch to thread 2)
b <func> thread <thread_id>	Sets a breakpoint only for the specified thread	b DeleteDoulinkList thread 3
set scheduler-locking on	Locks the current thread, allowing only the current thread to execute (avoid interference from other threads)	Execute when debugging thread 1 to prevent interruption by thread 2
set scheduler-locking off	Unlocks thread locking	Restore concurrent execution of all threads

Practical Scenario: Multi-Threaded Linked List Operation Conflict (Segmentation Fault)

1. Compile with thread support:

   gcc main.c doulinklist.c -o link_test -g -pthread  # -pthread enables the thread library
   

2. Start GDB and set breakpoints:

   gdb ./link_test
   b InsertTailDouLinkList  # Breakpoint for insertion function
   b DeleteDoulinkList      # Breakpoint for deletion function
   

3. Run the program: r;
4. View thread status:

   info threads  # Assume 3 threads are output, thread 2 executes deletion, thread 3 executes insertion
   

5. Switch to thread 2 (deletion thread):

   thread 2
   

6. Lock the current thread (avoid interference from the insertion thread):

   set scheduler-locking on
   

7. Step through the deletion logic:

   s  # Enter the DeleteDoulinkList function
   p tmp  # Check if the node to be deleted is valid (non-null, in the linked list)
   p tmp->prev  # Verify if the prev pointer is normal
   

8. Troubleshoot conflicts: If it is found that the node has been modified by the insertion thread when the deletion thread accesses it, add thread synchronization (e.g., mutex lock pthread_mutex_t).

III. Specialized Debugging: Troubleshooting Embedded Memory Errors

Memory errors (wild pointers, memory out-of-bounds, memory leaks) account for a high proportion of issues in embedded development. The following details troubleshooting methods combining GDB and auxiliary tools (valgrind).

3.1 Wild Pointer Debugging (Most Common Error)

Definition

A wild pointer refers to a pointer that points to freed memory, uninitialized memory, or an illegal memory address. Accessing a wild pointer triggers a segmentation fault.

Troubleshooting Steps (Example: Accessing After Linked List Destruction)

1. Start GDB and load the program: gdb ./link_test;
2. Set a breakpoint (at the position where the linked list is accessed after destruction):

   b main.c:100  # Assume line 100 of the main function accesses dl->head after destruction
   

3. Run the program to the breakpoint: r;
4. View pointer status:

   p dl->head  # If output is `0x5555555592a0 <error: Cannot access memory at address 0x5555555592a0>`, it is a wild pointer
   bt  # View where dl->head was freed (confirm if it was freed in DestroyDouLinkList)
   

5. Fix solution: Avoid accessing the dl pointer again after destroying the linked list, or check if dl is NULL before accessing.

3.2 Memory Out-of-Bounds Debugging (Common in Arrays/Linked Lists)

Definition

Memory out-of-bounds refers to accessing memory beyond the allocated range of an array/linked list (e.g., array subscript out-of-bounds, linked list node pointer pointing to an illegal address).

Troubleshooting Steps (Example: Out-of-Bounds Insertion in Linked List)

1. Start GDB and load the program: gdb ./link_test;
2. Set a conditional breakpoint (pause when the insertion position pos exceeds the linked list length):

   b InsertPosDouLinkList if pos > dl->clen  # Pause only when pos is out of bounds
   

3. Run the program: r;
4. View parameters:

   p pos  # View the out-of-bounds pos value
   p dl->clen  # View the current linked list length
   

5. Fix solution: Add pos validity check in the InsertPosDouLinkList function (e.g., if (pos < 0 || pos > dl->clen) return 1;).

3.3 Memory Leak Debugging (Combined with valgrind)

Definition

A memory leak refers to dynamically allocated memory (malloc/calloc) that is not freed. Long-term operation leads to memory exhaustion, which is extremely harmful in embedded devices.

Troubleshooting Steps (Example: Linked List Destruction Function)

1. Compile the program with debug information (same as 1.1);
2. Use valgrind to detect memory leaks:

   valgrind --leak-check=full ./link_test
   

3. View valgrind output:
   • If All heap blocks were freed -- no leaks are possible is displayed: No memory leaks;
   • If definitely lost: 120 bytes in 3 blocks is displayed: Explicit memory leaks exist (3 nodes not freed).
4. Use GDB to locate the leak location:

   gdb ./link_test
   b DestroyDouLinkList  # Breakpoint at the destruction function
   r
   s  # Step through the destruction function to check if all nodes are freed
   p curr  # Confirm each node is freed with `free`
   

5. Fix solution: Ensure all nodes are traversed and freed in DestroyDouLinkList, and finally free the linked list structure itself.

IV. GDB Practical Tips and Pitfall Avoidance Guide

4.1 Common Shortcuts (Improve Debugging Efficiency)

Shortcut	Corresponding Command	Description
Enter	Repeat the previous command	No need to re-enter the same command
Ctrl+L	Clear screen	Clear screen when there is too much debugging information
Ctrl+C	Interrupt program execution	Force pause when the program is in an infinite loop
Tab	Command completion	Press Tab to complete after entering part of a command

4.2 Pitfall Avoidance Guide

1. Forgetting to add the -g parameter: Without -g during compilation, GDB cannot display line numbers and variable names; recompile is required;
2. Monitoring uninitialized variables: The value of an uninitialized variable is random, so monitoring is meaningless; set watch only after the variable is initialized;
3. Not locking threads during multi-thread debugging: Concurrent execution of multiple threads causes frequent breakpoint switching; use set scheduler-locking on to lock the current thread;
4. Ignoring compiler warnings: Warnings displayed by -Wall (e.g., “uninitialized variable”, “pointer type mismatch”) are often precursors to errors and need to be fixed in advance;
5. Debugging cross-compiled embedded programs: Use the GDB of the cross-compilation toolchain (e.g., arm-linux-gdb) and connect to the development board via target remote.

4.3 Applicable Scenario Summary

GDB Feature	Applicable Scenarios in Embedded Development
Basic commands (n/s/p)	Simple logic errors (e.g., linked list pointer binding errors)
Conditional breakpoints	Specific loop iterations, boundary condition errors
Variable monitoring (watch)	Unexpected variable modification, abnormal pointer pointing
Multi-thread debugging	Conflicts in multi-threaded operations on shared resources (linked lists, queues)
Segmentation fault debugging (bt)	Wild pointers, memory out-of-bounds, null pointer dereferencing
Memory viewing (x command)	Low-level memory data verification (e.g., registers, hardware register mapping)

V. Summary

GDB is an indispensable debugging tool for embedded C language development. Mastering its basic commands + advanced functions + specialized debugging can greatly improve error troubleshooting efficiency. Based on actual embedded scenarios and combined with cases such as doubly linked lists and multi-threading, this article covers debugging methods from simple logic errors to complex memory issues. The core points are:

1. The -g parameter must be added during compilation; otherwise, normal debugging is not possible;
2. For segmentation faults, use bt first to locate the error line; for memory issues, combine with valgrind for detection;
3. Advanced functions (conditional breakpoints, watch, multi-thread debugging) are key to solving complex problems;
4. After debugging, correct the code in a timely manner to avoid high-frequency embedded errors such as wild pointers and memory leaks.