Writing Your Own Windows Debugger - Overview

发表于 2017-04

Introduction

Debuggers are the apple of the hacker's eye. We benefit a lot from the debugger, but few of us know the principle of it.

In the book Gray Hat Python , the author has constructed a simple debugger. However, it is too simple, it is only a machine language level debugger, and can only set basic breakpoints and show CPU register information. We also want to know how to

Show source code
Set breakpoint based on lines, not memory address
Set Step In, Step Out, Step Over
Show stack trace
Show global and local variables

In this Chinese blog Zplutor's, I find a excellent series which has covered most above topics. I decide to write a English blog about it, and I will turn his code into a C++ version.

Before getting started, let's make some limitations:

It is only a user mode debugger.
It is only a Windows debugger. Although the principle is quite same, but Windows has offered lots of convenient APIs. The implementation will be different on Linux.
It is only a terminal-based debugger.
Different from Gray Hat Python , the debugger will be implemented by C++.
The debuggee program is single thread.

The modified debugger can be found here. It is only tested under Windows 10 + Visual Studio 2013.

To Start the Debuggee Program

The so-called user mode debugger is to debug the program in user mode. Windows has provided a series of open API for debugging, and they can be devided into three categories:

API for starting the debuggee program
API for handling debug event during debug loop
API for inspecing and modifying debuggee program

The first thing to do before debugging a program is to start it. On Windows, we use CreateProcess to start to program:

STARTUPINFO startupinfo = { 0 };
startupinfo.cb = sizeof(startupinfo);
PROCESS_INFORMATION processinfo = { 0 };
unsigned int creationflags = DEBUG_ONLY_THIS_PROCESS | CREATE_NEW_CONSOLE;

if (CreateProcess(
    "L:\\git_up\\anotherDebugger\\anotherDebugger\\Debug\\test.exe",
    //path,
    NULL,
    NULL,
    NULL,
    FALSE,
    creationflags,
    NULL,
    NULL,
    &startupinfo,
    &processinfo) == FALSE)
{
    std::cout << "CreateProcess failed: " << GetLastError() << std::endl;
    return;
}

DEBUG_ONLY_THIS_PROCESS means the subprocess of the debuggee will not be debugged. If you need subprocess, use DEBUG_PROCESS.
CREATE_NEW_CONSOLE means the debuggee's and debugger's output will be separated in two consoles.
If the debugger process exits, the debuggee will also exit.

Debugger loop

The debugger loop is a bit like Windows GUI message loop, some operations and exceptions will stop the debuggee and send event to the debugger. We always use

DEBUG_EVENT debugEvent;
WaitForDebugEvent(&debugEvent, INFINITE)

to capture the debug event.

There are 9 debug event in total:

CREATE_PROCESS_DEBUG_EVENT. Reports a create-process debugging event.
CREATE_THREAD_DEBUG_EVENT. Reports a create-thread debugging event.
EXCEPTION_DEBUG_EVENT. Reports an exception debugging event.
EXIT_PROCESS_DEBUG_EVENT. Reports an exit-process debugging event.
EXIT_THREAD_DEBUG_EVENT. Reports an exit-thread debugging event.
LOAD_DLL_DEBUG_EVENT. Reports a load-dynamic-link-library (DLL) debugging event.
OUTPUT_DEBUG_STRING_EVENT. Reports an output-debugging-string debugging event.
RIP_EVENT. Reports a RIP-debugging event (system debugging error).
UNLOAD_DLL_DEBUG_EVENT. Reports an unload-DLL debugging event.

If the debug event has been handled correctly, then

ContinueDebugEvent(debuggeeprocessID, debuggeethreadID, DBG_CONTINUE);

to continue the debuggee process. Let's combine the above to construct the debug loop:

while (WaitForDebugEvent(&debugEvent, INFINITE) == TRUE)
{
    debuggeeprocessID = debugEvent.dwProcessId;
    debuggeethreadID = debugEvent.dwThreadId;
    if (dispatchDebugEvent(debugEvent) == TRUE)
    {
        ContinueDebugEvent(debuggeeprocessID, debuggeethreadID, FLAG.continueStatus);
    }
    else {
        break;
    }
}

bool dispatchDebugEvent(const DEBUG_EVENT & debugEvent)
{
    switch (debugEvent.dwDebugEventCode)
    {
    case CREATE_PROCESS_DEBUG_EVENT:
        // TBD
        break;

    case CREATE_THREAD_DEBUG_EVENT:
        // TBD
        break;

    case EXCEPTION_DEBUG_EVENT:
        // TBD
        break;

    case EXIT_PROCESS_DEBUG_EVENT:
        // TBD
        break;

    case EXIT_THREAD_DEBUG_EVENT:
        // TBD
        break;

    case LOAD_DLL_DEBUG_EVENT:
        // TBD
        break;

    case UNLOAD_DLL_DEBUG_EVENT:
        // TBD
        break;

    case OUTPUT_DEBUG_STRING_EVENT:
        // TBD
        break;

    case RIP_EVENT:
        // TBD
        break;

    default:
        cout << "Unknown debug event." << endl;
        return false;
        break;
    }
}

In the next part of the series, I intend to give a brief introduction about the 9 debug events.

lyyyuna 的小花园

动静中之动, by lyyyuna

Writing Your Own Windows Debugger - Overview

Introduction

To Start the Debuggee Program

Debugger loop