Formatting Error in Process Explorer

The other day I was trying to reproduce a bug with 3dsmax involving altered regional settings. Many of our customers of course run max in countries where it is common in the language to use the comma as a decimal symbol. This for instance is a common practice in Quebec where the majority of the people speak French. It seems that some people have been having problems with max crashing on them when their O.S. regional settings are set to English but use a comma instead of a decimal symbol. So in attempting to reproduce this bug, I set my regional settings to use the comma. I didn’t reproduce any crashes with 3dsmax, at least not with our current production version. So anyways, it did have some strange unintended fallout however with process explorer from SysInternals.

What happened, is that all the numbers that process explorer reported were either truncated or just plain old cut off. So, for instance, instead of reporting 12 Gigabytes of Virtual memory for SQLServer.exe, it would simply report 2K. Obviously something was not right there.

Here are some links showing a screen shot of a normal and abnormal of process explorer.

Unfortunately, I had been installing and messing with a bunch of other debugging tools (Don’t you love playing with new tools?), and thought the problem was with them. Such was not the case. By reverting my regional settings to use a period for the decimal symbol, everything worked again.

Using the Windows Performance Toolkit to find memory leaks

The Windows Performance Toolkit can be used to monitor Heap activity, including monitoring the stacks of heap allocations. You can find out more about the Windows Performance Toolkit here:

It is downloaded via the SDK installer that comes with the Windows 7 SDK. To install, simply start the SDK installer, and select the Windows Performance Toolkit option as shown here:


Its default install location, for a 64 bit system is: C:Program FilesMicrosoft Windows Performance Toolkit.

I will not explain how the Windows Performance Toolkit captures, processes or stores heap allocation data. The pages on MSDN explain that pretty thoroughly. What I want to explain here is a quick and simple way to capture heap information, and find a memory leak, with a basic stack trace. The main tool I use for this is called XPerf, short for XPerf.exe.


First I want to demonstrate this by writing some simple code that will leak some memory. I included a few functions that get called from main, so as to display a non-trivial stack trace.

int* AllocSmallAmount(int val)
    return new int(val);
void RunForLoop(int max)
    printf("Test Leak: Start\n");
    for (int i = 0; i < max; i++)
        int* ptr = AllocSmallAmount(i);
        printf("i: %d\n", i);
    printf("Test Leak: End\n");

int _tmain(int argc, _TCHAR* argv[])
    return 0;

Note, that  works for both debug and release builds. But be sure that the compiler settings have ‘omit frame pointers’ turned off. This tool needs it off in order to provide correct stack traces when a heap event occurs.

So with that, I compile the code, and navigate to the directory where my binary (test_leak.exe) resides. I then created a batch script (which I call xperf_run.bat) that contains the following commands:

Code Snippet
  1. @echo off
  2. rem my binary that I want to test leaks for
  3. set bin=%CD%\test_leak.exe
  4. rem store the results here
  5. set savefile=%CD%\test_leak.etl
  7. pushd "C:\Program Files\Microsoft Windows Performance Toolkit"
  9. rem start the session
  10. xperf -on Base -BufferSize 512 -MinBuffers 5 -MaxBuffers 5
  11. xperf -start HeapSession -heap -PidNewProcess "%bin%" -WaitForNewProcess -BufferSize 512 -MinBuffers 5 -MaxBuffers 5 -stackwalk HeapAlloc+HeapRealloc+HeapFree
  13. rem stop the session
  14. xperf -stop HeapSession -stop -d "%savefile%"
  16. popd
  17. pause

Notice first, I set the path to my application I want to test, and I set the output path for where I want to store the results (line 5). Line 11 contains the code that will actually launch my app when I start this batch script. The parameter –PidNewProcess is followed by the full file path of application that will get launched by xperf. The parameter –WaitForNewProcess means that xperf will wait until my test application is done, before returning control to the batch script.

The first time you run this, you will get a warning from xperf:

xperf: warning: This system is not fully configured for x64 stack tracing.
Please modify the registry under:

  HKLMSystemCurrentControlSetControlSession ManagerMemory Management

and set the value:

  DisablePagingExecutive (REG_DWORD) = 1

Then reboot before retrying tracing.

Note: Tracing has been enabled, this is just a warning.

Which means you have to modify the registry value, and reboot your computer. Once you do that, you can continue to run the application. Which gives the following results:

Test Leak: Start
i: 0
i: 1
i: 2
i: 3
i: 4
i: 5
i: 6
i: 7
i: 8
i: 9
Test Leak: End
Merged Etl: C:Users<user name>DocumentsVisual Studio 2010Projectstest_leakReleasetest_leak.etl
The trace you have just captured "C:Users<user name>DocumentsVisual Studio 2010Projectstest_leakReleasetest_leak.etl" may contain personally identifiable information,
o files accessed, paths to registry accessed and process names. Exact information depends on the events that were logged. Please be aware of this when sharing out this trac
Press any key to continue . . .

So that is it. The rest is the hard part: deciphering the results.

Analyzing the results

Double click the file test_leak.etl, and select yes to any warning dialog popup’s that may appear. Will will show you two progress bars, and then you will see a window with a lot of graphs or charts in it.


There are actually quite a few charts to look at here. With possibilities of adding many more if you choose different command line parameters. For the purposes of a memory however, we are interested in two things:

  1. Outstanding heap allocation size
  2. Outstanding heap allocation count

So I like to filter out everything except those two. Click on the side bar to expand the FrameList, and uncheck everything except Heap Outstanding Allocation Size and Heap Outstanding Allocation Count;


Next, take the splitter bar under the graph for Heap Outstanding Allocation Count and drag it down. You might have to expand the window size to make it fit. Doing this, you can see the stair step effect of a heap slowly growing in regular, even increments. That is the heap that we are interested in.


It is obvious that the other two heaps (There are three total) are not allocating or freeing memory during the majority of the process lifetime, but rather maintain a steady state. The heap in green is the one we are interested in. You can turn on or off the display of any heap by clicking on the legend, and checking or unchecking various heaps. As the following picture shows.


Summary Table

To examine the stack trace of the memory leaks, you first select Load Symbols from the Trace menu:


Then you right click over the graph and select Summary Table.


This summary table is a listview that contains records and fields of data. By default stack traces are turned off. Choosing what data to display is similar to choosing data to display for the graphs. You open the tool panel on the left, and check Stack from the available checkboxes. In this example I am interested in looking at: Type, HeapHandle, Process, Stack, Count and Size. I then drag the Type column over to the far left, and then place HeapHandle to the right that.

If I left the selection blank when I right clicked, and selected summary table, I would get a graph that included the entire lifetime of the process, from start to finish. If you did that all memory allocations were allocated and freed inside your ‘selected’ timeblock, which in this case is the entire length of time that the Kernel was logging heap events for this application. In this case, the x axis on the graph shows about 1.95 seconds, of which all memory allocations took place between about 0.25 seconds and were freed about 0.95 seconds. The windows performance analyzer calls this Allocated Inside / Freed Inside or AIFI for short. If you look at the summary table, you will see a type called AIFI:


Expand it, and you will see that there were five heaps active during the lifetime of this process:


You will also see some stacks associated with each heap, with the words [Root] next to it.

Here open up the one of the stack traces, starting at root until you find a call to __tmainCRTStartup.


This is the heap that all the memory was leaked on. There underneath __tmainCRTStartup is a call to wmain (assuming you compiled in unicode). expand that to drill down into the callstacks that came out of main, until you find the memory leak:


There you can see the call stack for the memory that leaked 10 times for a total of 40 bytes of memory over the course of the applications lifetime.


And that brings me to the conclusion that this tool is worthless for finding memory leaks for any app larger than the directions on a shampoo bottle. This memory leak is completely buried with all sorts of other memory data. As you can see, it is mixed in the middle of 5 process heaps. Getting to it involved drilling down deep into code that you had to previously be familiar with. It is not at all obvious that this is a memory leak, unless you knew it was there in the first place. Which completely defeats the purpose of using this tool. Since you want a tool to find memory leaks in places where you do not know where they are.

There may be other tools for finding memory leaks, but this is not one of them.

Solving Link error 1112

Yesterday I brushed off some old code I had not touched in quite a while. It was a layer manager plugin for Autodesk 3dsmax that I had written years ago. I don’t think I had even opened any project files for this project in the last 12 months. So I re-acquainted with the code, made some changes to the files, and went to recompile it. Now this project compiles to both 32 and 64 bit targets. So I randomly chose a 64 bit target, and built the sucker.

Imagine my shock when I encountered a linker error stating:

module machine type ‘X86’ conflicts with target machine type ‘x64’

I knew enough about this in the past, and never had a problem before. Like I said before, I hadn’t touched the code in long time. What’s more, is my code was under source control!

I opened up the project settings and double and triple checked my Linker Target Machine settings. Indeed it was properly set at: MachineX64. Perfect!

Still the problem persisted. Then I got worried that the library files I was linking in were corrupted, and somehow 32 bit libraries were intermixed with 64 bit libraries. I didn’t want to download the entire 3dsMax SDK again, in the off chance this hypothesis was true. So I researched a solution to check what type of libraries I had!

So I used the utility DumpBin.exe that is found here:

"C:Program Files (x86)Microsoft Visual Studio 9.0VCbindumpbin.exe"

And by the way, in windows 7 I did a search a few times for ‘dumpbin’, which failed every time. Turns out you have to search for dumpbin.exe. Go figure.

Anyways, I used this tool to dump out the functions in my lib file. Then I was able to inspect the functions and see what platform there were compiled for.

For instance, notice the ‘Machine’ entry in a function I dumped out using dumpbin.exe:

Version      : 0
Machine      : 8664 (x64)
TimeDateStamp: 49B974BD Thu Mar 12 14:46:53 2009
SizeOfData   : 0000002D
DLL name     : maxutil.dll
Symbol name  : ??0Path@Util@MaxSDK@@QEAA@PEBD@Z (public: __cdecl MaxSDK::Util::Path::Path(char const *))
Type         : code
Name type    : name
Hint         : 18
Name         : ??0Path@Util@MaxSDK@@QEAA@PEBD@Z

Notice the second line contains x64, a dead give away this is a function compiled for an x64 target platform. So the problem was, my project linked in a lot of libraries from the 3dsmax sdk. The 3dsmax sdk contains about 49 library files. How was I supposed to inspect every function from every library file?

Sounds like a job for a script:

@echo off
call "C:Program Files (x86)Microsoft Visual Studio 10.0VCvcvarsall.bat"

for %%f in (*.lib) do dumpbin.exe -headers %%f | findstr /c:"Machine      :" >> _lib-exports.txt"

@echo on

So I put the above script into a batch file, and placed the batch file in the directory that contained all my library files. I iterated through all the library files, dumping out the header information. This was piped to find string, which searched for the machine information. I then appended that to a log file which I could inspect at my leisure. The log file contained 13642 lines looking like this:

Machine      : 8664 (x64)
Machine      : 8664 (x64)
Machine      : 8664 (x64)
Machine      : 8664 (x64)
Machine      : 8664 (x64)
Machine      : 8664 (x64)
Machine      : 8664 (x64)
Machine      : 8664 (x64)
Machine      : 8664 (x64)
Machine      : 8664 (x64)

So that was it. I opened the file in my favorite text editor, and a search for x86 turned up nothing. It was x64 all the way down.

So now what was I supposed to do? I was stuck. Let me review all I had done.

  1. My target machine linker setting was properly set.
  2. My build configuration setting was properly calling ‘x64’ and not win32.
  3. All the libraries I was importing were actually 64 bit libraries.

All appeared lost and hopeless until I ran across a tidbit on an MSDN forum.

Here the gentleman answered:

If you have left the ide at the default settings, then it will actually use the x86_amd64 compiler only. This is set under tools->options->projects and solutions->vc++ directories. Under amd64 executables the directory should be $(VCInstallDir)binx86_amd64 followed by $(VCInstallDir)bin.

This method works for all versions of windows so its a default setting. If you want to though you can change the x86_amd64 to just amd64.

So I looked at the executable settings for my x64 build configurations in VS 2008. It looked like this:


I changed it to this by adding the x86_amd64 directory:


And everything now works. My project now compiles and links just fine.

So I am left to surmise that some external add-on to visual studio hosed my executable tools settings for 64 bit configurations. I had off and on over the last year been using bullseye, which messes around with the settings in the project executable directories. So Perhaps that is what messed up that setting in visual studio. But now it’s fixed, and I am very happy.