[xplorer] Fast forward VS6 64 bit
home blog 26 October 2008 [programming]

"Noone will ever need more than 640KB RAM" Bill Gates

In this lengthy post I will describe the procedure and the tools required to build 64 bit executables using Visual Studio 6 (VS6) and the old Active Template Library v3 classes (ATL3). While building the 64 bit version of xplorer I found many generic guides with 64 bit programming tips, but very few describing the build process itself. Most probably it's trivial to build 64 bit programs with newer versions of visual studio, but the common conception is that VS6/ATL3 combo is a non starter, just on account of ATL thunking. As it turns out all you need is the 2003 platform SDK (PSDK) that contains the 64-bit compatible ATL v3.1 and a few tweaks explained step by step below.

I have not examined Intel's Itanium 64 bit processor, only AMD64. It may be easy to target itanium too but you'll have to test it yourselves <g>. I am building on a standard 32 bit windows XP SP2 system using C++ (ATL/WTL template classes).

  1. Put together the x64 build environment
    VS6 is patched with SP6 and linked to the 2003 PSDK
  2. Patch ATL and WTL
    ATL31 requires a few small changes for DEP compatibility; for WTL I use v7.0 but v7.5 should be fine too
  3. Build a test 64 bit application
    _WIN64 must be defined and a few compiler and linker switches to avoid a few common build errors
  4. Port your 32 bit source code to 64 bit
    There are countless articles on this subject, so I'll just discuss what I found important for xplorer
  5. Remote debugging tips
    I don't have 64 bit hardware to test on, but all is not lost with beta testers, minidumps and windbg debugger.

Put together the x64 build environment

VS6 may be 10 years old but it was built to last. Not only it is fast and lightweight, it can learn new tricks too. If you don't have it first install service pack 6 with cumulative patches. In case you can't install it, rename sp698ent.stf to acmsetup.stf and run the setup program again.

Then download and install PSDK for windows server 2003 SP1 that includes the 64-bit compatible ATL and a set of 64-bit tools that can be used from VS6. The only snag is that VS6 generated code (with its original C++ compiler) isn't library compatible to this PSDK so for your 32 bit executables you'll have to continue using your older PSDK (I have July 2002).

As with any PSDK, the integration involves adjusting the INCLUDE, LIB and PATH environmental variables. Most important is the path where the 64 bit compiler and linker are in (aka Executable files in VS6 Tools > Options > Directories). The names haven't changed since 1998 so putting the PSDK path in front ensures that VS6 calls the correct 64 bit toolset. For the x64 target the correct path is [PSDK2003]\Bin\Win64\x86\AMD64.

If you are like me and you want to work from VS6 GUI without messing with makefiles and such, you can launch the studio from a batch file, setting up the environmental variables and forcing VS6 to use them passing it /USEENV command line switch. Here's the 64 bit script I use, a combination of the original VCVARS32.BAT and PSDK's SETENV.CMD for the AMD64 target. You will have to adjust it slightly for your installation locations of VS6 and PSDK 2003.

If you prefer external projects and makefiles see this guide.

Patch ATL and WTL

People ask me why do I stick to a 10 year old compiler and C++ class framework. The answer is "better the devil you know" ok I am a laggard too <g>. xplorer is a rock solid program that buys bread and beer, and I don't want to introduce (possible) bugs switching to a new ATL and a new GUI.

The header files in [PSDK2003]\Include\atl contain an almost x64-ready ATL, a precursor to version 7 that ships with visual studio .NET. The bigger changes I see in this _ATL_VER 0x0301 are new window creation thunks, and a set of USES_CONVERSION_EX string conversion macros that use a safer (viz: less efficient) _alloca that may hit the heap if too much stack space is requested.

All that's left is a slight but important change to ATLBASE.H so that thunks will run from executable memory avoiding DEP faults, which are not tolerated in 64 bit windows versions. You must comment out the definitions of AllocStdCallThunk and FreeStdCallThunk and replace them with:

// for details see http://www.codeproject.com/KB/wtl/WTLExpress.aspx
#define AllocStdCallThunk() VirtualAlloc(0, sizeof(_stdcallthunk), MEM_COMMIT | MEM_RESERVE, \\
#define FreeStdCallThunk(p) VirtualFree(p, 0, MEM_RELEASE)

There are at least two relevant instances you have to edit on lines 303 and 330. If you get any linker errors just comment out the missing LIB files (e.g. #pragma comment(lib, "atlthunk.lib")).

To use the same ATL with your WIN32 projects, you will need to define OLD_ATL_CRITSEC_CODE to avoid compiler "error C2899: typename cannot be used outside a template declaration". I guess the rusty VS6 compiler is in too deep water here. You don't need this defined for 64 bit builds though as the PSDK compiler is smarter.

Finally if your code relies in any way on ATL's setting child windows IDs equal to the CWindowImplBase this pointer (atlwin.h line 2159), you'll have to change strategy since 64-bit pointers cannot fit in the 4 bytes used by GetDlgCtrlID.

Coming to Windows Template Library (WTL) in case you are using it (the one they made google chrome with), it has to be said that it was 64-bit compatible for some time, using all those DWORD_PTR and what have you, so there's little to do there. I personally use version 7.0 with a few bugfixes introduced with later versions I have no need for ATL7 compatibility or windows CE code present in WTL 7.5. As for the current WTL v8.0 I am not even sure if it is VS6 compatible! But I digress.

Build a test 64 bit application

VS6 has no clue about anything but WIN32 that's the only platform available in Build > Configurations menu. But if you use your (modified) 64 bit script to launch VS6 it will generate 64 bit executables like a charm.

Create a new empty WTL project and accept all the defaults. Then add a 64 bit target from Build > Configurations, say based on "Win32 Debug". You can't get rid of "win32" so your target will be called something lame like "Win32 Debug AMD64" but that's not a problem. Select it as the active configuration.

At minimum you'll need to define the following preprocessor symbols with project settings dialog <alt+F7>:

  • _WIN64. WIN32 is also automatically defined in the PSDK header files
  • UNICODE and _UNICODE. Time to drop the old _MBCS compatibility, I'm sure x64 won't be available for win9x :)

For compiler settings, use /Zi (debug info: program database) instead of /ZI which is no longer supported. Turn on maximum warnings with /W4 /Wp64 especially for your old 32 bit code. Other notable switches:

  • Remove /GZ. The compiler will complain anyway that /RTC1 should be used for the runtime checks. The linker will also fail with "error LNK2001: unresolved external symbol _RTC_Shutdown" and other silimar _RTC unresolved externals.
  • Add /GS-. This is to turn off the default buffer overflow checks and avoid linker "error LNK2001: unresolved external symbol __security_cookie". If you want this feature make sure you link to bufferoverflowU.lib

Coming to linker settings, replace /machine:I386 with /machine:AMD64. VS6 will insist adding its /machine:IX86 to the list but that doesn't seem to matter. The recommended /NXCOMPAT switch does not have any effect for the 2003 version so I left it out. Even if you use /GS- as recommended above you may have to link to bufferoverflowU.lib since it is used by libraries added with the CRT. Finally if you see "error LNK2001: unresolved external symbol "void * __cdecl ATL::__AllocStdCallThunk" you haven't done the change to atlbase.h I mentioned above, so it's time to do it now :)

After all these changes you should get a clean build for your test 64 bit program. I attach a sample VS6 project for you to download (12 KB)

Port your 32 bit source code to 64 bit

There have been countless articles about porting source code issues so I won't bore you with a repeat here. See the further reading links below if you need a refresher course. What is striking is how little you need to do; you have to hand it to whoever devised the 64 bit migration path, good job lads! The only thing I couldn't port to x64 was inline assembly __asm statements in C++ source code. But that was the exception.

The 64 bit PSDK header files are full of remnants of the 32 bit era, data structures like WIN32_FIND_DATA, constants like _WIN32_WINNT etc. All of these work without change for 64 bit, and there are no equivalent WIN64_FIND_DATA definitions. You just set WINVER and all the rest PSDK basic constants to the minimum target platform as usual.

On a similar tack, all system DLLs keep their old names like KERNEL32.DLL. There is no KERNEL64.DLL on windows 64, so all your explicit LoadLibrary calls for dynamic loading needn't be modified. Behind the scenes win64 uses the filesystem redirector to route 32 bit applications to C:\windows\SysWOW64 folder whenever they try to access a DLL in SYSTEM32. A 32 bit program can use Wow64DisableWow64FsRedirection API to see the real filesystem I don't know how this affects its LoadLibrary calls!

The registry redirector does a similar job transparently referring 32 bit applications to Wow6432Node subnode whenever they access parts of HKLM and HKCR registry hives. There are flags that allow cross access, for example KEY_WOW64_32KEY allows a 64 bit executable to access the 32 bit hive. Note that HKCU is a shared hive so if your program saves its settings in the registry the 32 and 64 bit versions will share the same data.

I haven't found any information for updating the VS_VERSION_INFO resource to mark the executable version block as 64 bit. But the manifest file seems to be adequate. The change you need to make is swap "x86" for "processorArchitecture=AMD64"; see the sample manifest in RES folder of the test64 project above. You'll notice that the 64 bit EXE file grows in size for no apparent reason: xplorer 64 bit is a whopping 1.13MB 35% up from the 804KB 32 bit version for the same source code! Perhaps there are some compiler switches I need to discover for size.

Your installer should also be 64-bit aware so that it places your program in the real c:\program files folder and not get fooled by the redirector. I hope I'll get away with a 32 bit installer since NSIS offers a couple of neat switches for 64 bit installations (EnableX64FSRedirection and SetRegView).

After all these preliminaries it's time to bite the bullet and try your source code through the 64 bit compiler. Take heed of all the warnings generated by /Wp64 switch, especially pointer truncations. And good luck chasing subtle bugs! In my first run, xplorer source code generated 37 errors and 2486 warnings. Most of the warnings are irrelevant though. I have given up hope of a clean build; 4.2MB is a lot of source code for one man to tidy up it's #pragma warning(disable: xxx) to the rescue!


Remote debugging tips

Porting to 64 bit will inevitably reveal subtle bugs and your program will crash a lot to begin with, so a small diversion on remote debugging skills is in order.

For a long time I've been using mapfiles to track down crashes on remote user computers. Using the crash address and the MAP file it is possible to find the approximate line number in the code that blew up. With a Dr Watson crash log you can even get a stack trace. The downfall started with vista dropping dr watson altogether, and the final nail is the x64 compiler discontinuing /MAPINFO:LINES support. MAP files are infeasible for 64 bit crash investigation.

as we say in Greece and my lament for MAP files quickly turned to joy when I discovered the ease of remote debugging with minidumps and windbg. Here are the required steps:

  • Enable full debug information even for release builds, and store the program database (PDB) file matching the distributed executable.
  • Install a crash handler that saves a minidump file when the program crashes. This is a small (<100KB) file that can be easily sent back by a user via email.
  • Minidumps can be opened in latest versions of developer studio (not VS6) but 32 bit versions cannot open 64 bit minidumps. Just get the 32 bit version of windbg part of the windows debugging tools, which does load both 32 and 64 bit minidumps.
  • Put the minidump in the VS6 output folder (where the executable was first created) with its matching EXE and PDB files. Fire up windbg and open said minidump with File > Open crash dump menu command. At the prompt type .ecxr. to jump to the crash location. Open extra panels as necessary using the toolbar icons (they are similar to VS6 icons e.g. for Call Stack, Processes and Threads etc). Through the PDB file windbg will jump straight to your source code offending the user's CPU, just like the VS6 debugger.

The beauty of it is that you don't have to learn a lot to use windbg and the information it extracts from the minidump is in technicolor compared to dr watson crash logs. Mapfiles, RIP! If you need to see symbols from Windows DLLs involved in the crash use File > Symbol file path to add the internet location SRV*E:\srvcache*http://msdl.microsoft.com/download/symbols this assumes you have a folder E:\srvcache that will be used to cache debug symbols.


Who said you can't teach an old dog new tricks? With all the information above you should extend the life of your
(t)rusty VS6 for another 10 years easy! <g>

Post a comment on this topic



What would you like to do next?

Reclaim control of your files!
  • browse
  • preview
  • manage
  • locate
  • organize
Download xplorer2 free trial
"This powerhouse file manager beats the pants off Microsoft's built-in utility..."

20022008 Nikos Bozinis, all rights reserved