We are looking into updating our application with /CETCOMPAT liker flag. After reading some articles (like Developer Guidance for Hardware-enforced Stack Protection and Understanding Hardware-enforced Stack Protection) and spending some time with running dumpbin /headers against various Windows OS DLL’s and EXE’s, as well as Office, Edge and SQL Server binaries, it is still not clear to us whether we should use this linker flag on just DLL projects or EXE projects as well. What are the rules?
Related
I have a 32-bit application and I have a problem with it on Windows 7 x64. I'm loading a DLL. LoadLibraryW succeeds and the subsequent call to GetProcAddress fails with the error code 127 ("procedure not found" or something like that).
The funny part is that I know for a fact the function is exported by the DLL. I made no typos in the GetProcAddress call. I can see the function with Depends.exe and DllExp.exe. The exact same application binary successfully loads the function from the exact same DLL on Windows 10 x64, but not on Windows 7 x64.
Some more details: the library is dbghelp.dll and the "missing" function is MiniDumpWriteDump.
And the fun bit: dbghelp.dll provides API for inspecting the modules loaded into the process and for enumerating functions exported by those modules. So, first I took the HMODULE for this problematic dbghelp.dll and ran
auto ptrSymInitialize = (decltype(&SymInitialize))GetProcAddress(hDbgHelpDll, "SymInitialize");
It worked, this function did load! Then I loaded SymEnumSymbols, written the enumerator callback and finally ran the following to enumerate all the functions in this very `dbghelp.dll":
ptrSymEnum(GetCurrentProcess(), 0, "dbghelp*!*", &Enumerator, nullptr);
And what do you know, MiniDumpWriteDump is, in fact, listed there. Go figure.
Thoughts?
I can see your intent is to use MiniDumpWriteDump. We also make minidumps in our product, and I'm the one to support this.
I would suggest against using dbghelp.dll supplied with OS. First, they tend to be outdated and not support the latest minidump capabilities, which you would want to have. Second, they have proven to be rather unreliable. I believe they simply lack too many bugfixes.
What I found to work quite well is to take dbghelp.dll from Debugging Tools for Windows package (currently part of Windows SDK) and ship it along with our product. This way, I can be sure minidumps will have all the latest features and it works reliably on all OS. It has been some 8 years now, with OS ranging from WinXP to Win10, and I didn't have any issues.
I'm not exactly sure which version of SDK I used to extract the currently used dbghelp.dll, probably it was Win7 SDK. I simply didn't have a reason to update since then. However, we do use Debugging Tools for Windows package from Win10 SDK on Win7 without any issues, so I guess you can use Win10 version as well.
that's exactly what I've been doing, and I didn't bring dbgcore.dll
This was just a plain bad idea. Microsoft makes no effort to make the DLLs that are included with the OS to be backwards compatible. They don't have to. In their implementation, only the interface needs to be compatible. They do take advantage of new capabilities or design changes to improve the implementation.
Like you saw here, a side-effect of the MinWin project. There is no reasonable guess where that ended, if it happens to work now on the Win7 machine then you got lucky. Maybe you won't be so lucky on a Win7 machine without SP1, maybe some minwin glue DLLs are missing on a clean install, maybe the minidump itself is affected negatively some way. Impossible to predict.
So never do this. Afaik you should not be doing this at all, a Win7 machine already has dbghelp.dll available. Not 100% sure, it has been too long and Win7 is rapidly turning into the new XP. If you find it to be necessary then always use the redistributable version. Included with the SDK's Debugging Tools for Windows. Copy it into the same folder as the EXE that needs it so you don't mess up a machine.
My scenario is I'm supporting a VB6 app at the place I work and in the last few weeks it has started crashing more often than it ever used to. It uses both a local Access MDB database and a remote SQL Server DB for different types of storage. The good news is we are writing a replacement app, the band news I need to support this one in the meantime and the vendor is long gone from this world.
What are some ways I could try and diagnose what is causing the crash? For example so far I've tried ODBC tracing (For the MDB component), SQL Profiler tracing and ProcMon on a client PC.
Is there anything else I could try to discover what the app was trying to do at the time of the crash?
You can also start in a debugger.
windbg or ntsd (ntsd is a console program and maybe installed). Both are also from Debugging Tools For Windows.
Download and install Debugging Tools for Windows
http://msdn.microsoft.com/en-us/windows/hardware/hh852363
Install the Windows SDK but just choose the debugging tools.
Create a folder called Symbols in C:\
Start Windbg. File menu - Symbol File Path and enter
srv*C:\symbols*http://msdl.microsoft.com/download/symbols
then
windbg -o -g -G c:\windows\system32\cmd.exe /k batfile.bat
You can press F12 to stop it and kb will show the call stack (g continues the program). If there's errors it will also stop and show them.
Type lm to list loaded modules, x *!* to list the symbols and bp symbolname to set a breakpoint
Use db address (as in db 01244 to see what's at that memory.
If programming in VB6 then this environmental variable link=/pdb:none stores the symbols in the dll rather than seperate files. Make sure you compile the program with No Optimisations and tick the box for Create Symbolic Debug Info. Both on the Compile tab in the Project's Properties.
I have a mixed managed and unmanaged C++ application that's working quite well. I'm using Visual Studio 2013 to compile it, and all is well. Recently I upgraded my computer to Windows 10, and now it's not working.
If I get the executable compiled on Windows 8, it runs properly on Windows 10. It only fails if I compile it on Windows 10.
The failure is peculiar as well. I run the EXE and nothing happens. When I run it from Visual Studio it doesn't even reach the first line of main. Breakpoints are all marked as 'disabled'. When I break the running process the debugger shows an empty stack trace.
UPDATE: Hunch about DLL loading turned into a fact:
I used Process Explorer and I see the process has two threads. The one starting at !CorExeMain is stuck at !LdrLoadDll, but I can't tell which DLL that is.
OK, found the DLL that causes the problem. I've created a C++/CLI console application, used that DLL and got the same behavior. The DLL is part of the application (and part of the VS solution). It's a native C++ DLL, compiled with the same compiler and settings. This DLL references other DLLs unfortunately.
This is a generic problem called "LoaderLock". The operating system makes very strong guarantees when it calls the DllMain() entrypoint of a DLL. Strictly in loading order, they never run at the same time. There is a lock in the OS loader that ensures these promises are kept.
And a lock always has the potential to cause deadlock. It will happen when the DllMain() entrypoint does something unwise like loading the DLL itself with LoadLibrary(). Or call a function that requires the OS to have a DLL already loaded. Can't work, its DllMain() entrypoint can't be called because the loader lock is held. The program will freeze. C++/CLI apps are prone to this problem, lots of stuff tends to happen in DllMain(). Indirectly, you can't see it in your code.
You can only see it with the debugger. You must change its flavor, Project > Properties > Debugging > Debugger Type, change it from "Auto" to "Mixed". You'll now also see the unmanaged code that is running including the OS loader functions, name starts with "Ldr". Be sure to enable the Microsoft Symbol Server with Tools > Options > Debugging > Symbols. And be sure to use the Debug > Windows > Threads debugging window as well, the truly tricky loader deadlocks that don't repeat well or appear to be affected by the OS version are caused by another thread loading a DLL.
Diagnosing and fixing it can be difficult, be sure to reserve the time you need to dig in. If you can't make heads or tails of the stack traces then post them in your question.
Before taking #Hans Passant advice I carefully combed through the code, dumpbin /dependentsed the executables and DLLs and made sure there were no custom DllMains. There were none. DLLs were indeed loaded with LoadLibrary, but that was happening long after the loading DLLs were loaded.
So I took #Hans Passant advice. Set up the debugger properly and checked the state of the process during the deadlock. One of the threads was stuck in LdrLoadDll.
It took a little tinkering to find the name of the DLL that was passed to LdrLoadDll. It was AVGHOOK.DLL .
I disabled AVG, and lo and behold - everything is back to normal.
This is the second time AVG is messing with me. The previous time I nearly replaced a printer until I figured out all the PCL errors disappeared when I disabled AVG. I think I'm not going to use it any more.
I have put our comment ping pong into a full text:
#1:
As you found out, that your application did not load, you needed to check if applications on your system (W10, VS2013) run at all.
Reply: A test console app is running fine.
#2:
If your application doesn't run, build up a similar application and step-by-step put code of your app into the new app until it fails.
If the failure is causes by a DLL (which cannot be loaded, as it was in your case), remove DLLs from your app until it works. Alternatively build a dummy console app, include DLL #1 and use some functions of that DLL. Compile, run, check. Go on until DLL #n...
Reply: faulty DLL was found.
#3:
Reference only this DLL in the test app to ensure it's this DLL only and not a combination of DLLs.
Is it a managed DLL or a native C++ DLL?
If the faulty DLL is from a foreign source: bad luck. Ask the developer for support.
If it's your own: Did you compile the faulty DLL on W10+VS2013, too, or did you copy it from your previous system? I suggest you compile this DLL again on the new system.
Reply: it's a native C++ DLL, which is part of the solution and is compiled together with the main app.
This DLL references other DLLs unfortunately.
#4:
Create a new console app, that references not your faulty application DLL, but the DLLs which are referenced by YOUR DLL. Omit the intermediate step to detect if the failure comes from the other DLLs.
The general procedure is: Split up faulty code to find out out which part is causing trouble. Romans already knew this 2000 years ago: Divide et impera ;-) Though they did it in a different context...
We had the exact same problem here during several weeks ! And we finally found a solution !
In our case, it was the anti-virus Avast which was corrupting the generated .exe !
The solution was to simply disable all agents while generating the release !
If you use another anti-virus, try to disable it.
I have an executable compiled by MSVS2013 on Windows 7 SP1 32 bit with C runtime statically linked and platform toolset "Visual Studio 2013 - Windows XP (v120_xp)"
It runs perfectly on Win7 (see attached screenshot of Dependency Walker profiling).
It fails to run on WinXP:
Here is the profile log:
Starting profile on 12/9/2014 at 2:55:03 AM
Operating System: Microsoft Windows XP Professional (32-bit), version 5.01.2600 Service Pack 3
Program Executable: c:\temp\OCTOPUS.EXE
Program Arguments:
Starting Directory: C:\Temp\
Search Path: C:\Program Files\ActiveState Komodo Edit 8\;C:\WINDOWS\system32;C:\WINDOWS;C:\WINDOWS\System32\Wbem;C:\Program Files\MATLAB\R2009b\runtime\win32;C:\Program Files\MATLAB\R2009b\bin;C:\VisaPoint\sbperl\perl\bin
Options Selected:
Simulate ShellExecute by inserting any App Paths directories into the PATH environment variable.
Log DllMain calls for process attach and process detach messages.
Log DllMain calls for all other messages, including thread attach and thread detach.
Hook the process to gather more detailed dependency information.
Log LoadLibrary function calls.
Log GetProcAddress function calls.
Log thread information.
Use simple thread numbers instead of actual thread IDs.
Log first chance exceptions.
Log debug output messages.
Use full paths when logging file names.
Log a time stamp with each line of log.
Automatically open and profile child processes.
--------------------------------------------------------------------------------
00:00:00.000: Failure starting the process. %1 is not a valid Win32 application (193).
[1]: http://i.stack.imgur.com/m82n6.png
[2]: http://i.stack.imgur.com/6Yr37.png
Modern versions of Visual Studio, starting with VS2012, use linker settings that marks the executable image compatible with the current generation of Windows. Version 6, started with Vista. XP and Server2003 were in the previous generation, version 5.
You can still build programs that run on XP with VS2013. But you have to make a setting change. Right-click the project, Properties, General. Change the Platform Toolset setting to v120_xp.
There are a few consequences, your program gets built with an old version of the Windows SDK. V7.1, the last one that was still compatible with XP. The C runtime library makes a bunch of operating system calls that are only available in generation 6. It will stumble along without them, affecting relatively minor details related to threading and culture. This is the kind of FUD that ought to remind you that it might not be the best idea to still promise support for such an ancient operating system. Make sure you keep a running version of XP around so you can properly test and repro. And keep in mind that you can't install VS on it anymore so if you do get a support call from an XP user then you tend to get a fairly heavy migraine.
Ok, just forgot to set SUBSYSTEM in Linker options.
Here is a link with explanations:
https://software.intel.com/en-us/articles/linking-applications-using-visual-studio-2012-to-run-on-windows-xp
From a Microsoft article:
Address Space Layout Randomization (ASLR)
ASLR moves executable images into random locations when a system
boots, making it harder for exploit
code to operate predictably. For a
component to support ASLR, all
components that it loads must also
support ASLR. For example, if A.exe
consumes B.dll and C.dll, all three
must support ASLR. By default, Windows
Vista and later will randomize system
DLLs and EXEs, but DLLs and EXEs
created by ISVs must opt in to support
ASLR using the /DYNAMICBASE linker
option.
I don't quite get it. Take the base system DLLs loaded by every process on WIndows: NtDll.dll and kernel32.dll.
If a have a non-aware executable, will these system DLLs use ASLR? That is, will they load at a different base address after every system reboot on Win 7 for this executable or will they always load at the same base address after system reboot like they do on Win XP?
To make it more clear what I mean: My typical dummy program's startup stack will look like this:
write_cons.exe!wmain() Line 8 C++
write_cons.exe!__tmainCRTStartup() Line 583 + 0x19 bytes C
write_cons.exe!wmainCRTStartup() Line 403 C
> kernel32.dll!_BaseProcessStart#4() + 0x23 bytes
Looking at the asm of BaseProcessStart, I see on my XP box here:
_BaseProcessStart#4:
7C817054 push 0Ch
7C817056 push 7C817080h
7C81705B call __SEH_prolog (7C8024D6h)
7C817060 and dword ptr [ebp-4],0
...
Now what interests me is the following:
On Windows XP, the address will always be 0x7C817054, regardless of how many times I reboot this machine. If I were on Win7 with ASLR, will this address change between reboots if the executable that loads kernel32.dll is not enabled for ASLR?
(Note: For me, atm., there is only one minor use-case this address would be useful for: In Visual Studio, I can only set a "Data Breakpoint" for assembly level functions, that is a breakpoint # 0x7... - If I want to break in a specific ntdll.dll or kernel32.dll function, in Windows XP I do not have to adjust my breakpoints between reboots. With ASLR kicking in (the scope of this question) I would have to change the Data Breakpoints between reboots.)
Technically whether the system dlls get relocated or not, it shouldn't matter, as the linker will bind to symbols, not addresses. These symbols are resolved by the runtime loader into to addresses for the instanced system dlls, thus your binary should be none the wiser. From what i've seen however, windows 7 will reset the base randomization every reboot, including system dlls(note: this is from debuging WOW64 apps on widows server 2008 R2). You can also do a system wide disabling of ASLR via some registery edits, but thats not really relevant...
Update:
the section on ASLR in this article explains what gets relocated and when.
it doesn't mention if the base will reset every reboot, but for system dlls, its never going to be guaranteed to load at the same address twice, reboot or no reboot.
the important thing is according to article, everything needs to opt-in to ASLR for system dll's to be relocated.
Your program will resolve calls into system DLLs wherever they happen to be loaded. But, unless your executable is linked with /DYNAMICBASE, it will not be given a randomized base address. In other words, your exe will always load at the same base address.
If you want your exe to load at a randomized address, then you have to link it with /DYNAMICBASE, and every DLL that it references must also have been linked with /DYANMICBASE. The system DLLs (starting in Vista) are all linked with /DYNAMICBASE.