I'm playing around with native code generation from C#. I'm using HeapCreate and HeapAlloc to allocate executable memory to which I write x86-64 instructions. I then use Marshal.GetDelegateForFunctionPointer to turn that into a .NET delegate and call it. I am able to properly create and call a simple function that returns a constant value.
The issue is that I am unable to step into that function in VS2010. When I use "step into" in the disassembly window on the call opcode, it simply steps over and I never get to see the disassembled function. I've tried browsing to my executable memory address in the disassembly window or adding a breakpoint at that address but Visual Studio complains that the memory is not executable. I've also tried generating an "int 3" instruction but it only causes my application to exit without warning (as if Environment.Exit was called). I've done a simple test program in C++ that uses the same technique to generate a function and call it and I can step into it without any issues.
How can I step into a native function I've generated in an allocated chunk of executable memory in a .NET Visual Studio project? Does anything prevent this from being possible? (I've got the 'just my code' option unchecked)
Did you set "Enable Unmanaged code debugging" as described on MSDN?
If you're really using HeapAlloc, I'm surprised that this worked at all. I was under the impression that memory allocated by HeapAlloc doesn't have execute permission on it. I usually use VirtualAlloc to allocate such memory and explicitly set the execute flag with VirtualProtect.
Related
I really need to figure out when my Fortran project is reading an element of a vector. I use data breakpoint on a daily basis but I could not find a way to set a data breakpoint when my code access (i.e. read) a memory address, while I always set it for breaking when the address is modified. Is there a way to do this on Visual Studio 2010? (I use intel visual fortran compose XE 2011 as compiler). Or maybe updating to a more recent visual studio?
Just as a note, I saw here that gdb does that Can I set a breakpoint on 'memory access' in GDB?
thanks
A.
Ps: I emaild the guys from GDB and they said its not possible to do it with it.See their answer below:
Hello,
Currently the type of created watchpoint is hardcoded to "write". This is because
Visual Studio has no support for other types of watchpoints (in GUI and infrastructure).
Perhaps it would be possible to enable read watchpoints in the GDB console,
however it also would require a hack, as the console actually works "through"
Visual Studio (it does not pass commands directly to GDB).
I am also not sure whether this feature really works in GDB. GDB has a lot
of commands which have very limited target scope, e.g. they only work
for single threaded programs, or just for Linux and not when using
gdbserver, etc. A read watchpoint looks like a mechanism that is very
platform dependent. Please check if GDB your platform supports read watchpoints.
Also let us know if this feature is critical for you.
Best regards
Some workarounds (except using WinDBG):
Inject a NaN value if it is about a floating point element. And enable trapping of operations with NaN. This will catch not a read, but a first arithmetic operation with the value. Thus copying of the element will be missed, but an attempt to perform an operation will throw an FP exception exposing the place it occurs.
Unmap a memory page with the value. It's very unexact and will react for access to 4Kb of data around the value... But it still may work for some cases. Check description of MapUserPhysicalPages() for Windows, and munmap() for Linux.
You can use Mike Morearty's Hardware Breakpoints.
I have not tested them on Visual Studio 2010, but I have succesfully used it in VS 2008, 2015 and 2017.
int x = 0;
HardwareBreakpoint hb;
hb.Set(&x, sizeof(x), HardwareBreakpoint::Read);
// Some random code.
int y = x; // The breakpoint pauses the execution on this line.
Note that HardwareBreakpoint object must be in the scope while you want that breakpoint to be alive. When it goes out of the scope, the breakpoint will stop functioning.
Before some time i coded my own little db editor program, i was coding it from the zero using Win API's so its not very small project.
It was working fine on all OS till now, i have Win 7 x64 with all latest updates and my application is crashing with 0xC000005 exception because of some of the Heap functions(HeapAlloc or HeapFree, i use nothing else), i tried replacing HeapAlloc & HeapFree with VirtualAlloc and VirtualFree and it was all fine, but i dont want to use the virtual memory....
Something else, i tried to attach with debugger to trace the problem, but when i attach debugger its not crashing, then i tried to display MessageBox to trace where it crashes, but when i display MessageBox its not crashing too....
My application is running as 32bit process.
Coded in C.
Anyone had similar problem ?
Firstly, both HeapAlloc and VirtualAlloc allocate virtual memory.
My guess as to what is happening is that you are writing past the boundary of the allocated memory. The reason why this does not work with HeapAlloc is that it allocates exactly the amount of memory you request. With VirtualAlloc, the size returned is the requested size rounded up to the next page boundary. In your case, this gave a bit more leeway (even though your code is still doing the wrong thing).
In terms of why it has been working so far: you just got lucky. Look carefully at the code accessing the allocated memory and if you get stuck, post the relevant part up here. If the debugger isn't helping and the bug is easily reproducible, just comment out parts of the code until you locate the line causing the crash.
Have you attached it to Debug version of your application? If the problem does not appear in debug version then you should check what warnings (on highest level) generate your code, maybe you will find some uninitialized variables. If nothing here, then you might use some static analysis tool to help with finding bugs - like PVS-Studio http://www.viva64.com/.
You can also compile Release version with debugging information enabled, this way when problem arrises you should be able to attach to your application with debugger and see callstack with function names. To make it easier to debug, disable code optimizations.
You can also try gflags from windows debugger tools, this program will trigger breakpoint each time you write outside of buffer boundary. This is really helpfull tool because not all buffer overruns end up with exceptions. Use it on application with debugging information enabled, and preferably with code optimizations off. To enable gflags for your app use:
gflags /p /enable myapp.exe /full
(http://msdn.microsoft.com/en-us/library/windows/hardware/ff543097%28v=vs.85%29.aspx)
I encounter an access violation when calling a DLL inside LabVIEW. Let's call the DLL "extcode.dll". I don't have its code, it comes from an external manufacturer.
Running it in Windbg, it stopped with the message:
(724.1200): Access violation - code c0000005 (first chance)
First chance exceptions are reported before any exception handling.
This exception may be expected and handled.
ntdll!RtlNewSecurityObjectWithMultipleInheritance+0x12a:
And call stack is:
ntdll!RtlNewSecurityObjectWithMultipleInheritance+0x12a
ntdll!MD5Final+0xedfc
ntdll!RtlFindClearBitsAndSet+0xdf4
ntdll!RtlFindClearBitsAndSet+0x3a8
ntdll!RtlFindClearBitsAndSet+0x4b9
ntdll!RtlCreateProcessParametersEx+0x829
ntdll!LdrLoadDll+0x9e
KERNELBASE!LoadLibraryExW+0x19c
KERNELBASE!LoadLibraryExA+0x51
LabVIEW!ChangeVINameWrapper+0x36f5
LabVIEW!ChangeVINameWrapper+0x3970
LabVIEW!ExtFuncDynLibWrapper+0x211
Note that dependencies of extcode.dll are loaded before access violation.
The situation is random, but when it happens all subsequent tries lead to it.
The code is a simple LabVIEW function calling a function in the DLL, and prototype is super simple (int function(void)) so it cannot be an misconfiguration of the call parameters, nor pointer arithmetics. I checked every combination of calling conventions and error checking levels.
The DLL runs perfectly fine when called in other environments (.NET and C).
I found that RtlFindClearBitsAndSet is related to bit array manipulations
What does it make you think about? Do you think it is a problem in extcode.dll, LabVIEW, or Windows?
PS: I use LabVIEW 2010 64 bit, on Windows 7 64 bit (and extcode.dll is 64 bit). I didn't manage to reproduce it on 32 bit system.
11/18 EDIT
I ended up making a standalone exe that wraps the DLL; LabVIEW communicates with it through pipes. It works perfectly, but I stil don't understand why loading a DLL into LabVIEW can crash.
If it works ok when called from C, you can quit working with Windbg because the DLL is probably ok. Something is wrong with how the DLL is being called, and once the DLL overwrites some of LabView's memory it is all over, even though it might take 1000 iterations before something actually goes kablooey.
First check your calling conventions, C or StdCall. C calling convention is the default and StdCall is almost certainly what you want. (Check the DLL header file.) LabView 2009 apparently did some auto-checking and fixing of calling conventions, but the switch to LLVM in LV 2010 has made this impossible; now it just tanks.
If it still tanks after changing this, check your calling arguments again. what you are passing, scalars or pointer data? You cannot access memory allocated by the DLL from LabView without doing some sneaky things, although you can allocate memory (i.e. byte array) in LabView and pass a pointer to it to the DLL for it to modify.
Also, if you are getting a pointer (such as a refnum) from an earlier call to DLL and returning it, check your pointer size. LabView's Call Library function now has a "pointer size integer" type, which generates the appropriately-sized type depending on whether it is invoked in 32-bit or 64-bit LabView. (It is always 64 bits on the wire, because that has to be defined at compile time.) The fact that your DLL works in 32 suggests this is a possibility.
Also keep in mind that C structs are often aligned by the (C) compiler. If you are passing a pointer to a struct made of a Uint8 and an UInt16, the C compiler will allocate 32 bits (or maybe even 64 bits) for this. You'll have to pad your struct (cluster) in LabView to make it match, or write a wrapper DLL to assemble the struct.
-Rob
An access violation (0xc0000005) will also be reported if DEP (Data Execution Prevention) is enabled on your machine and disapproves of something your binary (EXE or DLL) is trying to do. DEP is usually off by default on Windows XP, but active on Windows Vista / Windows 7.
DEP is a hardware-supported security measure designed to prevent malicious code executing some bytes that previously were considered "just some data"; I've had a few run-ins with it, all of which required re-compiling the offending binaries with a recent version of Microsoft Visual Studio; this allows to you set a flag which defines whether or not your binary supports DEP.
Some useful resources:
I found this MSDN blog entry
very helpful for gaining an
understanding of what DEP is and does
You might also want to consult this
technet article on how to turn
DEP on and off on Windows XP.
This is hard to diagnose remotely, but here are a couple of ideas.
The fact that your function takes no arguments means that either the function is truly trivial, or there is some stored state in the dll that takes into account previous function calls. Maybe the crash in this function is only an indicator, and you have a problem with a previous function call? Is there an initilization routine you're not calling?
If you only have problem when using 64 bit labview, my first guess would be that there's a problem with the 64 bit version of the dll, but if you are sure you don't have any problem with the exact same calls when using the dll in other environments, I'm stumped. One possibility is that you are using the wrong calling convention (stdcall vs. cdecl) in labview.
Have you tried importing the dll and header using the labview import wizard? This might help avoid silly mistakes with the prototypes.
One other thing to try: right click on the DLL call, choose configure and make sure you're running in the UI thread instead of any thread. Sometimes this helps.
When working with git and cygwin under NTFS, i found that sometimes the executable bit is not set (or un-set during checkout or some file operations) - inside cygwin cd to the folder and do
chmod a+rwx *.dll
and check if it changes a thing (and check if you want it this way!). I found this question while searching for LoadLibrary() failing with GetLastError() returning 5 (not "0xc0000005" btw) and solved the issue with this chmod call.
I can readily dump the entire memory space of a process using various tools.
But is it possible to dump just the memory space used by a DLL loaded by some process? What tools should I use?
Thanks,
Jim
You probably mean looking at the memory allocated by code in the DLL.
I think this is impossible. If the DLL allocates memory, and the DLL is written in C++, and the C/C++ Run Time is dynamically linked (i.e. as DLL), then it will use the same C/C++ Run Time as the main application, and all DLL's allocated memory will be allocated on the same heap.
Even if the DLL would have the C/C++ Run Time statically linked, or the DLL is written in a different language, it will probably use the same default Windows heap.
If you have control over the DLL yourself, you could try to implement a custom memory manager for your DLL (in C++ this means overriding new and delete, 6 global operators in total), try to use a different (i.e. non-default) Windows heap, and then using the heapwalk methods of the low-level Windows debugger WinDbg, but it will be quite difficult to get this all working. Or your DLL's custom memory manager could allocate memory at a fixed address using VirtualAlloc (or non-fixed, and then logging the virtual address). Then you can look at this address space in the normal process memory dump.
I wonder if there is a good way to find the source code that causes a heap corruption error, given the memory address of the data that was written 'outside' the allocated heap block in Visual Studio;
Dedicated (0008) free list element 26F7F670 is wrong size (dead)
(Trying to write down some notes on how to find memory errors)
Begin with installing windbg:
http://www.microsoft.com/whdc/Devtools/Debugging/default.mspx
Then turn on the pageheap like this:
gflags.exe /p /enable yourexecutable.exe /full
This will insert a non writable page after each heap allocation.
After this launch the executable from inside windbg, any writes outside the heap will now be caught by this debugger. To turn of the pageheap afterwards use this:
gflags.exe /p /disable yourexecutable.exe
More info on how to use the pageheap here.
For Window 10 you could enable the PageHeap option in the GFlags Tool, this tool is included as part of the Debugging Tools for Windows.
The Page Heap options in GFlags lets you select standard heap verification or full-page heap verification. Beware, the full heap verification uses a full page of memory for each allocation so it can cause system memory shortages.
To enable the Page Heap in GFlags:
•To enable standard page heap verification, the standard version will write a pattern at the end of each heap allocation and then examine the pattern when the allocations are freed.
To verify all processes use:
gflags /r +hpa
gflags /k +hpa
for a single process use:
gflags /p /enable ImageFileName
•To enable full page heap verification for one process, this option places an inaccessible page at the end of each allocation so that the program stops immediately if it tries to accesses memory beyond the allocation, this should only be used on a single process due to the heavy memory consumption.
gflags /i ImageFileName +hpa
gflags /p /enable ImageFileName /full
The two commands above are interchangeable.
Note: All page heap settings mentioned above are system wide settings stored in the registry (except /k) and remain effective until you change them. The /k setting is a Kernel flag setting are set for this session and will be lost when Windows shuts down
Another helpful tool is the Application Verifier, but this is not part of the Debugging Tools for Windows, rather it is included in the Windows Software Development Kit (SDK).
Maybe you can try Microsoft's Application Verifier.
It solved a similar problem for me once,by turning on extra checks on heap operations.
In my opinion, the randomness of corrupted address is because the heap can be 'subtly' damaged, and the problem won't show up until something big happens to the heap (like massive allocation/free).
You could set a breakpoint on a write to the memory address. The debugger will then show you the code that writes to the location, but you still need to work out which of the writes are causing the problem.
It's probably too late but if it compiles with gcc and can run on linux you may use valgrind to find the source of the problem (i don't remember the flags, i only used it once with great success).
more info about Gflags and PageHeap(which helped a lot):
http://msdn.microsoft.com/en-us/library/windows/hardware/ff549561%28v=vs.85%29.aspx
I am assuming C++ as the language.
If the error is reproducible and the corrupted address is always the same, you can put a data breakpoint to stop the program when writing at this address.
Make sure all libraries you are linking to are compiled in the same CLR version as the application you are running - all in Release or all in Debug.
When you compile in either Debug and Release you are actually targeting two different versions of the C runtime library. These versions are quite different and they use different strategies for allocating memory and they use different heaps. But the most important thing to know is that they are NOT compatible with one another.
The Release C runtime library allocated memory as expected, whereas the Debug will add extra information, such as guard blocks to track buffer overflow and the location that called the allocation function, and in turn it allocates more memory than the Release.
If your are linking your application to a mix of DLLs which were built in Release and Debug, you will most likely end up trying to delete an object in one CLR which was created in another. This means you will be trying to free more or less memory than what was allocated to the object and this can corrupt the heap.
You should build your application, as well as attach to libraries which are built under the same configuration, either Release or Debug.
This problem can occur especially in modules that are being compiled with different compilers.
There is a way to work around, which I will mention but do not recommend. If for some reason you still need to build in different modes, this work around will allows for all memory to be allocated and freed from the same shared heap. The API GetProcessHeap will allow you to access the shared heap throughout the different modules. By using the HeapAlloc & HeapFree you can allocate and free memory in the shared heap. Note: the HeapAlloc and HeapFree should replace all calls to malloc and free in your application.