Unwound Stack

This is another recycled post from the old blog.

In any windowing library, the question is always how to connect instances of whatever C++ class is representing a Window, and the HWNDs that the OS actually uses. I was curious as to how ATL & WTL do that, so I did a little digging.

I started with ATL’s support for Property Sheets. Looking at the implementation of CPropertySheetImpl, we see:

ATL::_AtlWinModule.AddCreateWndData(&pT->m_thunk.cd, pT); // 1
INT_PTR nRet = ::PropertySheet(&m_psh);                   // 2

Line number 1 obviously looks interesting. So, what the heck is AddCreateWndData? Like so many ATL methods, it ends up in a global:

AtlWinModuleAddCreateWndData(_ATL_WIN_MODULE* pWinModule,
                             _AtlCreateWndData* pData,
                             void* pObject)
{
    pData->m_pThis = pObject;
    pData->m_dwThreadID = ::GetCurrentThreadId();
    pData->m_pNext = pWinModule->m_pCreateWndList;
    pWinModule->m_pCreateWndList = pData;
...

Here’s what’s happening; ATL maintains a per-thread, singly linked list of AtlCreateWndData structures. These structures record the current thread and the current class instance. Here’s the structure definition:


struct _AtlCreateWndData
{
void* m_pThis;
DWORD m_dwThreadID;
_AtlCreateWndData* m_pNext;
};


for every kind of window (plain-jane Windows, Dialogs, Property Pages, &c), ATL finds some kind of “hook” that will be called after the window is actually created, but before it begins receiveing messages. There, (when it has an HWND laying around) it pops the head of the current list to find the C++ object corresponding to the window being created.

The current head of the list (for the current thread) is retrieved by calling AtlWinModuleExtractCreateWndData. This functon is called by CAtlWinModule::ExtractCreateWndData, which is in turn called by:

• CWindowImplBaseT< TBase, TWinTraits >::StartWindowProc
• CDialogImplBaseT< TBase >::StartDialogProc
• CCommonDialogImplBase::HookProc
• CColorDialogImpl::HookProc
• CPropertySheetImpl::PropSheetCallback

Let’s dig into StartWindowProc. This is the WNDPROC that gets registered with the Window class WNDCLASS. What does it do?

It pops the head off the current thread’s list of AtlCreateWndData, sets pThis to the corresponding member, and then:


pThis->m_thunk.Init(pThis->GetWindowProc(), pThis);


Alright, so what’s m_thunk? This is a member variable of type CWndProcThunk. The actual implementation is hidden behind a typedef or two, but the upshot is this: it’s a little structure whose member variables actually make up executable code! Specifically:


mov dword ptr [esp+0x4], pThis
jmp relwndproc


The Init member sets up the pThis & the address of the actual WNDPROC we’re going to call. Here’s the trick: if this code is executed at the beginning of a function in which the first parameter on the stack is an HWND (like, say, a window procedure), this code will write the address of the C++ class instance representing the window that’s just been created over top of the HWND and then jump to the beginning of the window procedure returned from the GetWindowProc function.

Remember that while we get back to StartWindowProc. StartWindowProc next gets the address at which this new code resides by calling:

WNDPROC pProc = pThis->m_thunk.GetWNDPROC();
WNDPROC pOldProc = (WNDPROC)::SetWindowLongPtr(hWnd, GWLP_WNDPROC,
                                               (LONG_PTR)pProc);

What we’ve just done is substituted our little thunk for the new window’s window procedure. With the thunk in place, we can now do this first off in the real WNDPROC:

CWindowImplBaseT< TBase, TWinTraits >* pThis = (CWindowImplBaseT< TBase, TWinTraits >*)hWnd;

which is actually kind of slick (although a maintenance headache, I’d guess…).

Trackback URI | Comments RSS