Adi Levin's Blog for programmers

July 19, 2009

Debugging tips and tricks

Filed under: programming — Adi Levin @ 1:06 pm
Tags: , , ,

What do you do when you encounter a bug that is really hard to debug? Here are a few tips & tricks:

Breakpoints with counter

If your program consistently throws an exception at a specific line, you want to break before the crash happens, and monitor how the program behaves. But just putting a breakpoint is not good, because it will stop whenever you reach that line, and not necessarily in the iteration that throws the exception. To know where to stop, place a breakpoint with a counter just before the suspicious line, set the counter to a high number, and wait for the program to crash. Then, the debugger will show you how many times you passed through the counter, and you’ll use that number to set the target to your breakpoint. In the next run, the breakpoint will stop exactly when you wanted.

Breakpoints with tracing

In Visual Studio, you can instruct the debugger to print values of certain expressions as well as the thread ID and even the call-stack, whenever it reaches a breakpoint. Right-click the breakpoint and choose “when hit…”. The result will appear in the “output” window in the IDE.

Divide and conquer

Suppose your program consistenly throws an exception at a specific line, because of corrupted data (for example, using a pointer that has been deleted). Suppose that you don’t even know the reason, but you know that somewhere before that line, the data got corrupted. So, the reason for the bug could be far away from the line where the exception is thrown. To detect which line of code caused the crash, use a divide-and-conquer approach, if possible.

Let’s regard the program as a series of commands, and suppose the the exception is thrown at line 10,000. You know that the reason for the bug happens somewhere between line 1 and line 10,000. Start by commenting-out lines 5,000-9,999 (assuming it is legal to do so). If the exception is still thrown, you know that the reason for the bug is between line 1 and 4,999. Otherwise, it is between 5,000 and 10,000. Uncomment, and continue by commenting-out half of the suspicious code, until the suspicious area is small enough.

Minidump files in Visual Studio

If your program crashes from time to time, but you cannot reconstruct the bug on your debugger, it is important to get as much information as possible from each crash. You need to write a side application that monitors your application for exceptions, and writes the call-stack and other needed information when an exception occurs.

The best way to do this, is to write a program that attaches to your application as a debugger using the DebugActiveProcess command (see MSDN for documentation). In this debugger program run a function that constantly waits for debug events, using WaitForDebugEvent. A debug event includes a thrown exception, threads being created or deleted, and other events. When you get the debug event, you can check if it is an exception, and get the type of exception (e.g. access violation or division by zero) and thread ID from which the exception was thrown. You should output this information to a text file. Next, you should also output a dump-file that will include the call-stack of all running threads using the command MiniDumpWriteDump.

The result is a “.dmp” file that you can load into Visual Studio, and see the running threads and call-stacks at the time the exception was thrown. To do this, you need to place the dmp file near the executable, and make sure you have PDB files for the relevant executables (EXE and DLL) that you wish to debug. The PDB files should be next to the DLLs. You also need to have the source-code available. Of course, source-code and PDB files should be exactly the ones that were used and created when building the specific version of the application for which the dmp file was created.

If your application does not throw a lot of exceptions on a regular basis, then running such a process that montiors exceptions in your application does not slow it down.It is safe to even distribute your application with this process attached. This way, when a customers complain about a crash, you can ask them to send you the dmp file (or do it automatically), andany other information that your process generates automatically, and you’ll be able to see excatly where your program crashed.

It is important to configure Visual Studio to save debug information (PDB files) even in the release version, for this purpose. It doesn’t slow down the application. You don’t need to distribute the PDB files – you only need to store them on your side. You’ll need one copy of the entire collection of PDB files for each build.


Breakpoints are sometimes not useful, because they block execution, and thereby change the timing of the application. I find it useful to place “Beep” commands at certain places in the code, so that I hear a beep, at a different tone, every time certain lines of code are executed. This way I use my hearing to monitor the execution of my application.


For crashes that are hard to reproduce, it is sometimes necessary to keep a detailed log of the operations that preceeded the crash. Writing a log-file isn’t trivial, especially in a multithreaded application. You need your logger to be able to write lines to the log-file from different threads, without slowing down the application. Simply writing a line to an open file is not good, because when the application crashes, the file will not be closed properly, so the log-file will be missing the last lines, which are the most important ones for debugging.

A good way to write to a log file is via a second process, called a “logger process“. In the application that you wish to debug, perform logging by posting messages to the logger process, using the non-blocking command PostMessage. Use the three parameters of the message – message identifier, wparam and lparam, to send the important information that you want to write to the log file. In the logger process, the window procedure or message loop should handle the incoming messages by writing the appropriate data to the log file.

The data that should appear in a log-file includes the time, thread ID, and the string you want to write. There should be a mapping between message identifier and the string that appears in the log-file. To transfer the mapping to the logger process, you can use WM_COPYDATA.

If you need to pass more information than fits in the three message parameters, consider using named pipes for communicating with the logger proceses.

Alternatively, you can implement a logger not as a different process, but as a dedicated thread inside your application. You can use APCs (the command QueueUserAPC) to send log requests to the logger thread. A dedicated thread is more efficient than accessing the log-file from different threads, because it doesn’t require explicit synchronization (i.e. mutex, semaphore or critical section). The downside of this approach is that you have to close the log-file every time you add a line to it (because you want to close properly upon crash), and this means slowing the application.


If you suspect data corruption due to memory buffer overrun, I propose that you run your application with the pageheap flag turned on. Pageheap is a flag that Windows can apply globally or to a specific executable. It tells Windows to change the way it does heap allocations. It allocates buffers on the heap such that they are follows by a reserved page. If your program tried to access an entry outside the valid range of the buffer, it will throw an exception of type access violation, and you can use your debugger to understand why it happened.

To use it, you will need gflags.exe, which is part of Windows Debugging Tools. You can download a 32 bit version and a 64 bit version for free. After installing the Winbdows Debugging Tools, run gflags.exe (as in global flags), to see the flags that can be turned on and off.

Turning on Pageheap will slow-down your application, but not as much as other memory monitoring applications do (such as Bounds Checker and Rational Purify).


Leave a Comment »

No comments yet.

RSS feed for comments on this post. TrackBack URI

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

Blog at

%d bloggers like this: