Pointer Truncation

Pointer truncation is an evil that all developers of native C++ code need to worry about. This is simply storing a wide piece of memory in a narrower bit of memory. This can become an issue on applications that are ported from 32 bit to 64 bit. In 32 bit applications, pointers are of course 32 bits wide. So it is no big deal to do something like this:

int* foo = new int[45];
int bad = (int)foo;

However in 64 bit applications, that is bad. Since that pointer could be using the high bits of it’s memory address, and when cast or truncated to a 32 bit int, it will lose information. The high bits of a pointer are the bits that pointer to regions higher than 0xFFFFFFFF. For instance a 32 bit pointer has 8 digits (in hexadecimal notation) and can look like this:

0x12345678

But a 64 bit pointer has 16 digits (in hex) and can look like this:

0x1111111122222222

The bits above that are 1 are the high bits. Since a 64 bit app can address more than 2^32 bits of memory, these bits can get used if the 64 bit app requires lots of memory. There is a way to force pointers to be assigned to the higher regions of memory without the expense of allocating all that memory. That way is to use VirtualAlloc in the Windows API, and reserve (but not commit) the lower 4 Gigs of memory. 

The results of a random debugging session involving the code below, shows that the high bits are completely lost when stored in the variable bad.

// a pointer, which is 64 bits wide on a 64 bit platform
int* foo = new int[45];
// this will truncate the memory pointer
int bad = (int)foo; // PURE Evil! Integers are 32 bits wide

Notice how the integer loses the leading value from the pointer (circled) as shown in the debugger watch window below: image

If the coder really gets adventurous they can try converting that integer back to a pointer again, with especially disastrous results:

// a pointer, which is 64 bits wide on a 64 bit platform
int* foo = new int[45];
// this will truncate the memory pointer
int bad = (int)foo; // PURE Evil! Integers are 32 bits wide

// an accident waiting to happen
int* worse = (int*)bad; // catastrophic!
// worse now looks like this: 0xffffffffa0040140
// now the high bits of 'worse' are filled with 0xff, which automatically
// makes it point to invalid regions of memory
// Any accessing of this pointer will result in an access violation.

which shows this in the debug watch window:

image

Notice how the high bits of the pointer worse are now pointing to regions of memory that the windows kernal has flagged as off limits. Any attempt to do anything with this pointer will result in an access violation, and if not correctly handled will result in the termination of the application.

One good way to find these is to call VirtualAlloc with very big values when your application begins. (I’m simplifying greatly here, trust me). This will reserve large chunks of memory below 4 Gigs. So that by the time your application makes actual memory allocations, all or most of the pointers will be above the 4 Gigs boundary mark. Then you have to test your application quite thoroughly to flush out these hot spots.

Well you might ask, should not the compiler aid you in this situation? Not the MS VC++ 10.0 compiler. Even at warning level 4, nothing is emitted to warn about the truncation.

Also Code Analysis will not help as it not enabled on 64 bit builds. i.e.

1>—— Build started: Project: VirtualMemTest, Configuration: Debug x64 ——
1>cl : Command line warning D9040: ignoring option ‘/analyze’; Code Analysis warnings are not available in this edition of the compiler

And on 32 bit builds of the same code code analysis still emits no warning.

5 thoughts on “Pointer Truncation

  1. Hi.
    I am one of the developers of PVS-Studio static code analyzer for C/C++/C++0x. As means of improving the quality of our product we are offering you a 3-month single user license for free. In case you find the analyzer interesting we would appreciate if you could mention your experience with it in one of your articles/blog posts.
    PVS-Studio is a static code analysis tool for the CC++C++0x languages designed to locate source code issues in 3 distinctive areas: migration to 64-bit platforms, OpenMP related errors and general-purpose C++ errors.
    If you are interested in our offer please reply to this message and we will provide you with a free registration key for PVS-Studio analyzer.
    Looking forward to your reply
    Paul Eremeeff, PVS-Studio development team
    OOO “Program Verification Systems” (Co Ltd)

    Like

  2. There’s a global registry key you can set to aid in catching exactly these issues:
    HKLMSystemCurrentControlSetControlSession ManagerMemory ManagementAllocationPreference REG_DWORD = 0x100000 . (reference: http://msdn.microsoft.com/en-us/library/windows/desktop/bb613473(v=vs.85).aspx or http://msdn.microsoft.com/en-us/library/windows/hardware/Dn613975(v=vs.85).aspx) .

    It might slow down the app quite a bit (https://randomascii.wordpress.com/2011/08/05/making-virtualalloc-arbitrarily-slower/) but for testing purposes it might be tolerable.

    Like

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s