Skip to content

Debugging

Jump to bottom
Enrico Fraccaroli (Galfurian) edited this page Jan 29, 2026 · 7 revisions

Debugging techniques for MentOS kernel and userspace programs.

GDB Debugging

Quick Start

# Terminal 1: Start QEMU with GDB server
cd build
make qemu-gdb

# Terminal 2: Connect with GDB
gdb --quiet --command=gdb.run

Or use cgdb (recommended):

cgdb --quiet --command=gdb.run

Setup

First, generate the gdb.run file (only needed once or after rebuild):

cd build
make gdbinit

This creates a GDB command file that tells GDB where to find symbol information for all binaries.

Debugging Session

When you start GDB, MentOS is paused at the bootloader entry. You'll see two breakpoints:

Breakpoint 1: Bootloader entry (boot_main)

(gdb) continue

MentOS will run until it hits the first breakpoint in the bootloader.

Breakpoint 2: Kernel entry (kmain)

(gdb) continue

MentOS will continue to the kernel's main function.

Common GDB Commands

# Breakpoints
(gdb) break kmain              # Break at function
(gdb) break kernel.c:100       # Break at line
(gdb) break sys_fork           # Break at syscall
(gdb) info breakpoints         # List breakpoints
(gdb) delete 1                 # Delete breakpoint 1

# Execution
(gdb) continue                 # Continue running
(gdb) step                     # Step into function
(gdb) next                     # Step over function
(gdb) finish                   # Run until function returns

# Inspection
(gdb) print variable           # Print variable value
(gdb) print *pointer           # Dereference pointer
(gdb) print/x value            # Print as hex
(gdb) x/10xw 0xC0000000        # Examine memory (hex words)

# Call stack
(gdb) backtrace                # Show call stack
(gdb) frame 2                  # Switch to frame 2
(gdb) info locals              # Show local variables

# Registers
(gdb) info registers           # Show all registers
(gdb) print $eax               # Print EAX register

# Process info
(gdb) info threads             # Show threads (limited in MentOS)

Debugging Specific Components

Debug Bootloader

(gdb) break boot_main
(gdb) continue
(gdb) step                     # Step through boot process
(gdb) print multiboot_magic    # Check multiboot

Debug Kernel Initialization

(gdb) break kmain
(gdb) continue
(gdb) break vmem_init          # Break at memory init
(gdb) break vfs_init           # Break at VFS init

Debug System Calls

(gdb) break sys_fork           # Break at fork syscall
(gdb) break sys_read           # Break at read syscall
(gdb) continue
# Trigger syscall from MentOS shell
(gdb) backtrace                # See call stack

Debug Process Scheduler

(gdb) break scheduler_pick_next_task
(gdb) continue
# Let scheduler run
(gdb) print runqueue.curr->pid
(gdb) print runqueue

VSCode Debugging

MentOS includes a VSCode launch configuration.

  1. Open MentOS folder in VSCode
  2. Run make qemu-gdb in terminal
  3. Press F5 or use "Run > Start Debugging"
  4. Select "(gdb) Attach" configuration

The debugger will connect to QEMU and stop at breakpoints.

Kernel Logging

The kernel provides logging functions for debug output.

Log Levels

#define LOGLEVEL_EMERG   0    // System unusable
#define LOGLEVEL_ALERT   1    // Action must be taken immediately
#define LOGLEVEL_CRIT    2    // Critical conditions
#define LOGLEVEL_ERR     3    // Error conditions
#define LOGLEVEL_WARNING 4    // Warning conditions
#define LOGLEVEL_NOTICE  5    // Normal but significant (default minimum)
#define LOGLEVEL_INFO    6    // Informational
#define LOGLEVEL_DEBUG   7    // Debug-level messages

Logging Functions

pr_emerg("System is unusable\n");
pr_alert("Action required\n");
pr_crit("Critical error\n");
pr_err("Error: %d\n", errno);
pr_warning("Warning: %s\n", msg);
pr_notice("Normal but significant\n");
pr_info("Informational message\n");
pr_debug("Debug: value=%d\n", value);

Default: Only pr_notice and below are displayed.

Changing Log Level

Add at the top of your source file:

// Include the kernel log levels
#include "sys/kernel_levels.h"

// Change header prefix
#define __DEBUG_HEADER__ "[MYMOD]"

// Set log level
#define __DEBUG_LEVEL__ LOGLEVEL_DEBUG

// Include debugging header
#include "io/debug.h"

Now all levels from DEBUG down to EMERG will be shown with [MYMOD] prefix.

Example

// kernel/src/fs/myfs.c
#include "sys/kernel_levels.h"
#define __DEBUG_HEADER__ "[MYFS ]"
#define __DEBUG_LEVEL__ LOGLEVEL_INFO
#include "io/debug.h"

void myfs_init(void)
{
    pr_info("Initializing MyFS\n");
    pr_debug("Debug info: %d\n", some_value);  // Won't show (INFO level)
}

Viewing Logs

Kernel logs appear in the terminal where you ran make qemu (or in build/serial.log when EMULATOR_OUTPUT_TYPE=OUTPUT_LOG):

[BOOT  ] Bootloader starting...
[KERN  ] Kernel initialized
[MEM   ] Memory: 128MB
[VFS   ] Mounted root filesystem
[MYFS  ] Initializing MyFS

User-space Debugging

Printf Debugging

Use standard printf() in userspace programs:

// userspace/bin/myprog.c
#include <stdio.h>

int main(void)
{
    printf("Debug: starting\n");
    int result = some_function();
    printf("Debug: result = %d\n", result);
    return 0;
}

Output appears in the MentOS shell.

Assertions

Use assert() for runtime checks:

#include <assert.h>

void my_function(int *ptr)
{
    assert(ptr != NULL);  // Check pointer
    assert(*ptr > 0);     // Check value
    // ...
}

If assertion fails, program aborts with message.

Return Value Checking

Always check system call return values:

int fd = open("/etc/passwd", O_RDONLY);
if (fd < 0) {
    perror("open failed");
    return 1;
}

Memory Debugging

Kernel Memory Tracing

Enable memory tracing at build time:

cmake .. -DENABLE_KMEM_TRACE=ON
cmake .. -DENABLE_PAGE_TRACE=ON
make

This adds logging to all kmalloc(), kfree(), and page allocation calls.

Checking for Leaks

Look for allocation/free imbalances in kernel logs:

[MEM] kmalloc(256) = 0xC0100000
[MEM] kfree(0xC0100000)

Unmatched kmalloc = memory leak.

Page Fault Debugging

When a page fault occurs, kernel logs:

[PAGE FAULT] addr=0x12345678, error=0x02
  EIP: 0xC0001234
  CR2: 0x12345678

Error code bits:

  • Bit 0: Page present (0 = not present, 1 = protection fault)
  • Bit 1: Write operation (0 = read, 1 = write)
  • Bit 2: User mode (0 = kernel, 1 = user)

Common causes:

  • Not present + kernel: Accessing unmapped memory
  • Not present + user: Accessing kernel memory from userspace
  • Protection + user: Writing to read-only memory

Crash Analysis

Kernel Panic

When kernel panics:

KERNEL PANIC: Unable to mount root fs
  EIP: 0xC0001234
  EAX: 0x00000000  EBX: 0xC0100000
  Call trace:
    0xC0001234: vfs_mount+0x42
    0xC0002000: kmain+0x120

  1. Note EIP (instruction pointer)

  2. Use objdump to find function:

    objdump -d build/mentos/kernel.bin | grep C0001234

  3. Check call trace for root cause

Triple Fault (QEMU Resets)

If QEMU keeps rebooting:

  • CPU exception during critical operation

  • Usually GDT, IDT, or paging issue

  • Enable GDB and break early:

    (gdb) break boot_main
(gdb) continue
(gdb) step

Common Issues and Solutions

Issue: "No such file or directory"

Cause: Program not in filesystem image

Solution:

make filesystem
make qemu

Issue: "Permission denied"

Cause: File permissions or not running as root

Solution: Check file permissions in filesystem/

Issue: Infinite Loop in Kernel

Symptoms: MentOS hangs, no response

Debug:

# Connect with GDB
(gdb) Ctrl+C              # Interrupt
(gdb) backtrace           # See where it's stuck
(gdb) info registers      # Check EIP

Issue: Syscall Returns -1

Cause: Syscall failed, check errno

Debug:

#include <errno.h>

int fd = open("/file", O_RDONLY);
if (fd < 0) {
    printf("Error: %d (%s)\n", errno, strerror(errno));
}

Issue: Segmentation Fault in Userspace

Cause: Invalid memory access

Debug:

  1. Use GDB:

    (gdb) break main
(gdb) continue
(gdb) step

  2. Check pointers before dereferencing

  3. Add assertions:

    assert(ptr != NULL);

Performance Profiling

Time Measurement

Use clock() or gettimeofday():

#include <time.h>

clock_t start = clock();
// ... code to measure ...
clock_t end = clock();

double cpu_time = ((double)(end - start)) / CLOCKS_PER_SEC;
printf("Time: %f seconds\n", cpu_time);

Instrumentation

Add timing logs in critical paths:

pr_info("Starting operation\n");
uint64_t start = read_tsc();  // Read CPU timestamp counter

// ... operation ...

uint64_t end = read_tsc();
pr_info("Operation took %llu cycles\n", end - start);

Hands-On Exercises

Note: For each example program below, add the source file to userspace/bin/CMakeLists.txt, then rebuild:

make programs
make filesystem

Exercise 1: Debug a Segmentation Fault

Goal: Use GDB to find and fix a crash.

Buggy Program - userspace/bin/buggy_segfault.c:

#include <stdio.h>
#include <string.h>

int main(int argc, char *argv[]) {
    // BUG: Not checking if argv[1] exists
    char *input = argv[1];
    
    printf("Input: %s\n", input);  // CRASH if no argument!
    printf("Length: %lu\n", strlen(input));
    
    return 0;
}

Steps:

  1. Compile and rebuild the filesystem: make programs && make filesystem

  2. In MentOS, run without arguments: /bin/buggy_segfault → Crashes!

  3. Connect GDB and set breakpoint:

    make qemu-gdb
# In another terminal (from build/):
gdb --quiet --command=gdb.run
(gdb) b main
(gdb) c
(gdb) n              # Step through
(gdb) print argv     # Inspect arguments
(gdb) print argv[1]  # NULL!

  4. Fix: Add argument checking:

    if (argc < 2) {
    printf("Usage: %s <input>\n", argv[0]);
    return 1;
}

Exercise 2: Add Debug Logging

Goal: Use kernel logging to trace execution.

Program - userspace/bin/trace_demo.c:

#include <stdio.h>
#include <unistd.h>

int main() {
    printf("Program started\n");
    
    for (int i = 0; i < 5; i++) {
        printf("Loop iteration %d\n", i);
        sleep(1);
    }
    
    printf("Program ending\n");
    return 0;
}

With Debugging:

# Boot with GDB
make qemu-gdb

# Set breakpoint at specific line
(gdb) b trace_demo.c:8
(gdb) c

# Inspect state
(gdb) print i
(gdb) list     # Show source code

Exercise 3: Debug a Data Structure

Goal: Inspect complex data structures in GDB.

Program - userspace/bin/debug_struct.c:

#include <stdio.h>
#include <stdlib.h>

struct Person {
    char name[50];
    int age;
    int id;
};

int main() {
    struct Person people[] = {
        {"Alice", 25, 1},
        {"Bob", 30, 2},
        {"Charlie", 35, 3}
    };
    
    printf("People array:\n");
    for (int i = 0; i < 3; i++) {
        printf("  %s, age %d, ID %d\n", 
               people[i].name, 
               people[i].age, 
               people[i].id);
    }
    
    return 0;
}

Debug:

(gdb) b debug_struct.c:17      # Breakpoint in loop
(gdb) c
(gdb) print people[0]          # Show struct
(gdb) print people[0].name     # Show field
(gdb) x/50c people[0].name     # Show raw memory
(gdb) n                        # Step through

Exercise 4: Memory Debugging

Goal: Detect memory errors (uninitialized, leaks, etc.).

Buggy Program - userspace/bin/memory_bug.c:

#include <stdio.h>
#include <stdlib.h>

int main() {
    // BUG 1: Use uninitialized variable
    int *data = malloc(10 * sizeof(int));
    printf("Value: %d\n", data[0]);  // Garbage value!
    
    // BUG 2: Access out of bounds
    data[100] = 42;                  // Overflow!
    
    // BUG 3: Memory leak
    free(data);
    printf("Value after free: %d\n", data[0]);  // Use-after-free!
    
    return 0;
}

Find Problems:

(gdb) b memory_bug.c:7
(gdb) c
(gdb) print data[0]        # Uninitialized!
(gdb) print &data[100]     # Out of bounds
(gdb) n                    # Execute free
(gdb) print data[0]        # Use-after-free

Fix:

// Initialize
for (int i = 0; i < 10; i++) data[i] = 0;

// Check bounds
if (index >= 10) {
    printf("Index out of bounds!\n");
    return 1;
}

// Don't access after free
free(data);
data = NULL;  // Set to NULL for safety

Exercise 5: Breakpoint and Inspection

Goal: Master GDB breakpoints and inspection.

Test Program:

#include <stdio.h>

int add(int a, int b) {
    return a + b;
}

int main() {
    int x = 10;
    int y = 20;
    int sum = add(x, y);
    printf("Sum: %d\n", sum);
    return 0;
}

GDB Session:

# Set breakpoint before line
(gdb) b <filename>.c:3           # In add() function

# Continue to breakpoint
(gdb) c

# Inspect local variables
(gdb) info locals
(gdb) print a
(gdb) print b

# Inspect register
(gdb) info registers
(gdb) disassemble               # Show assembly

# Modify variable during execution
(gdb) set variable a = 100
(gdb) print a

# Continue
(gdb) n                         # Next line
(gdb) s                         # Step into function
(gdb) finish                    # Run to function return

Exercise 6: Kernel Debugging

Goal: Debug kernel code with GDB.

Steps:

  1. Start GDB kernel debugging:

    make qemu-gdb
# In another terminal (from build/):
gdb --quiet --command=gdb.run

  2. Set kernel breakpoint:

    (gdb) b sys_fork              # Break on fork syscall
(gdb) c

  3. In MentOS:

    /bin/shell &                  # Background shell

  4. Back in GDB:

    # GDB will break when fork is called
(gdb) print pid              # Inspect variable
(gdb) bt                      # Backtrace - how did we get here?

  5. Analyze backtrace:

    #0 sys_fork () at kernel/src/process/process.c:123
#1 syscall_handler () at kernel/src/system/syscall.c:45
#2 0x00001234 in [userspace]

Exercise 7: Using Breakpoints Effectively

Common GDB Commands:

# Set conditional breakpoint
(gdb) b kernel/src/fs/vfs.c:200 if fd > 10

# Breakpoint with command
(gdb) b sys_read
(gdb) commands
> print fd
> print buf
> print count
> continue
> end

# Watchpoint (break on variable change)
(gdb) watch process_count

# List all breakpoints
(gdb) info break

# Delete breakpoint
(gdb) delete 1

# Disable/enable
(gdb) disable 1
(gdb) enable 1

Challenge: Debug a Complex Issue

Scenario: Your program crashes inconsistently.

Steps:

  1. Run program until crash
  2. Use GDB to inspect crash point
  3. Look at backtrace (bt command)
  4. Examine local variables
  5. Set breakpoints before crash
  6. Trace execution step-by-step
  7. Find root cause
  8. Fix and verify

Tools

objdump

Disassemble binaries:

objdump -d build/mentos/bootloader.bin | less
objdump -d build/mentos/kernel.bin | grep sys_fork

nm

List symbols:

nm build/mentos/kernel.bin | grep kmalloc

readelf

Inspect ELF files:

readelf -h build/mentos/kernel.bin
readelf -S build/mentos/kernel.bin  # Show sections

Tips

  1. Use meaningful log prefixes (__DEBUG_HEADER__)
  2. Check return values of all functions
  3. Initialize variables to catch uninitialized usage
  4. Use assertions for invariant checking
  5. Enable tracing during development
  6. Commit often so you can bisect bugs
  7. Test incrementally after each change

Next Steps

Previous: Development Guide | Next: Scheduling →

Clone this wiki locally