MiniDumpWriteDump for an external process

I have written an app to generate a minidump on an external process. This will be ultimately be called by our test harness on process’s that take too long to execute its unit tests. Or for our a process that hung while executing unit tests. (i.e. timed out) So I would like to create a minidump before terminating the process.

The minidump will be used to inspect the callstacks of the hung process. By looking at the callstacks you can get an idea of what went wrong: Or in our case, where did the hang occur.

So I wrote a handy little command line tool to do that. It takes as it’s (only) parameter the full path name of the executable you want a dump for. The cool part of this tool is you can also execute it while an application is running. And if you have symbols for your app, you will get nice callstacks in your minidump report.

getminidump (Source code here)

Advertisements

Lockup in call DialogBoxIndirectParam at call to NtUserCallHwndParamLock

One day, while running some of our automated test suites, I suddenly got a lockup in while running the following script: fat_rendering_exposurecontrol.ms.

I had seen this before last year. But it was a problem that would come and go at whim. Even recompiling would make it go away.

The root problem was caused by a ‘less than’ operator failing. I’m not really interested in why this is failing, but it helps explain the rest of the problem.

Anyways, it all starts with a call to an STL min function like this:

std::min( a, b )

Eventually the less than operator devolves down to this grotesque mess:

c:Program Files (x86)Microsoft Visual Studio 9.0VCincludexutility [line 264]

template<class
_Ty1, class
_Ty2> inline

bool
__CLRCALL_OR_CDECL
_Debug_lt(const
_Ty1& _Left, const
_Ty2& _Right,

const
wchar_t *_Where, unsigned
int
_Line)

{    // test if _Left < _Right and operator< is strict weak ordering

if (!(_Left < _Right))

return (false);

else
if (_Right < _Left)

_DEBUG_ERROR2(“invalid operator<“, _Where, _Line); <– here

return (true);

}

_Right was 3814.9946

_Left was 6405.5967

_Left < _Right    is false
!(_Left < _Right)    is true

Therefore the code had absolutely no business hitting the else clause of the if statement and calling _DEBUG_ERROR2 . Weird!! Perhaps it’s a compiler error… But anyways…

The error eventually reports an assert error to the Debug CRT. Which calls our own debug CRT report hook. There-upon we simply pop up an assert. The problem arises when the assert dialog doesn’t appear, but apparently locks on something.

The call stack for the assert:

>    user32.dll!NtUserCallHwndParamLock()     <– That locks up here:

user32.dll!InternalDialogBox()

user32.dll!DialogBoxIndirectParamAorW()

user32.dll!DialogBoxIndirectParamW()

BugslayerUtil.dll!JWnd::Dialog() Line 53    C++

BugslayerUtil.dll!JModalDlg::DoModal() Line 72    C++

BugslayerUtil.dll!PopTheFancyAssertion() Line 474    C++

BugslayerUtil.dll!RealSuperAssertion() Line 281    C++

BugslayerUtil.dll!SuperAssertionA() Line 566    C++

maxutil.dll!assert1() Line 237    C++ <– here we pop up a normal assert.

3dsmax.exe!CrashHandler::Imp::RTLReportHook() Line 1725    C++

msvcr90d.dll!_VCrtDbgReportW() Line 576    C

msvcr90d.dll!_CrtDbgReportWV() Line 242    C++

msvcr90d.dll!_CrtDbgReportW(…) Line 42    C++

msvcp90d.dll!std::_Debug_message() Line 22    C++ <– less than operator flubs it here.

dltonerep.dlu!std::_Debug_lt<float,float>() Line 265    C++

dltonerep.dlu!std::min<float>() Line 3399    C++

So I’m wondering how a call to

::DialogBoxIndirectParam

could lock up?

The NtUserCallHwndParamLock function assembly is this:

NtUserCallHwndParamLock:

00000000778DB7C0 mov r10,rcx

00000000778DB7C3 mov eax,1027h

00000000778DB7C8 syscall

00000000778DB7CA ret

00000000778DB7CB nop

00000000778DB7CC nop

00000000778DB7CD nop

00000000778DB7CE nop

00000000778DB7CF nop

It seems that I cannot step through assembly code because I get this error:

The dialog box states: “Unable to step. The process has been soft broken.”

[Edit]

Eventually I solved this by eliminating the use of the std::min and std::max. I simply wrote my own functions to return a minimum and maximum value, and that solved everything.

As for the lockup, well, it could have been because of some contention over resources, as I was trying to pop up a dialog, and perhaps the CRT was trying to as well.