Windows Explorer Refuses To Browse Into DefView Folder Of My NSE - windows-shell

On Windows 7, I developed a shell namespace extension (NSE) that presents a hierarchy of data that is very similar to a file system. Its junction point is the Desktop. I am able to click the [+] of each node in the tree control of Windows Explorer to expand that node. When I click on a folder in the tree view, I see a list of "folders" and "files", as expected, in the view presented by DefView. If I double-click on a "file", DefView dutifully invokes my corresponding IShellFolder::CreateViewObject to ask for IContextMenu, then eventually invokes IContextMenu::InvokeCommand, at which point I execute ShellExecuteEx with lpClass set to the extension of the "file". Explorer then dutifully launches the appropriate EXE application according to the "file" extension with my weird "file" path supplied as a command-line argument. However, if I double-click on one of my "folders" in the DefView, Explorer refuses to browse into the folder. An analysis of the call stack reveals that Explorer is trying to do some type of zone-checking on my weird "file" paths. I tried fiddling with Control Panel->Internet Options->Security zones, by adding my weird "file" paths to various zones, but that did not work. I can see from the call stack that there are two zone checks: a primary and a secondary. I guess I could brute-force my way around this, but I would like to know the sanctioned way to get past these zone-checks, or, rather, to get Explorer to browse into my "folders". Strangely, the IFileDialog does not have this problem, and browses into my "folders" without hesitation. I also tried adding the following code to my IShellFolderView::MessageSFVCB:
switch (uMsg)
{
case SFVM_GETZONE:
{
DWORD zone = URLZONE_TRUSTED;
* ((DWORD *)lParam) = zone;
}
return S_OK;
...
That did not help.
I also have a strong feeling my drag-and-drop will be blocked for the same underlying reason.
Any ideas?
Here is (partial) call stack:
MyDLL.dll!MyShellFolder::GetAttributesOf(unsigned int uCount, const _ITEMIDLIST * * aPidls, unsigned long * pdwAttributes) Line 385 C++
shell32.dll!CShellItem::GetAttributes(unsigned long,unsigned long *) Unknown
shell32.dll!CDefView::_SelectionHasFolderJunction(void) Unknown
shell32.dll!CDefView::_ZoneCheckFrame(unsigned long,int) Unknown
shell32.dll!CDefView::_InvokeContextMenuVerbOnSelectionWorker(char const *,unsigned int) Unknown
shell32.dll!CDefView::_InvokeContextMenuVerbOnSelection(char const *,unsigned int) Unknown
shell32.dll!CDefView::OnActivateSelection(unsigned long) Unknown
ExplorerFrame.dll!UIItemsView::WndProc(struct HWND__ *,unsigned int,unsigned __int64,__int64) Unknown

Related

Icon name in win32 resources?

I am writing a Win32 app (extract.exe) to list or extract ICONs or other resources in a module's resources (EXE or DLL). The app is written in C. It fails on 'FindResource()', I believe because of an incorrect name. The target app (app.exe) contains the following in it's RC script (app.rc):
MYICON ICON DISCARDABLE "app.ico"
MYRCDATA RCDATA DISCARDABLE "hello.txt"
I've examined the resources of app.exe with various third-party tools and my own apps: the resource name of the icon is displayed a "1" rather than "MYICON", but the name of the RCDATA correctly shows as "MYRCDATA".
I've confirmed that "app.ico" is a valid .ico file but still no joy.
What am I missing here?
Notes: I'm running win10/64 on a Dell machine and compiling with MS Visual C.
I finally figured this out. Apparently, the resource compiler splits an icon file into two parts: (1) an "icon directory" (aka "RT_GROUP_ICON") and (2) the icon itself. The directory contains a 6-byte header and one or more 16-byte entries that include the integer "ID" of the actual icon (1 in my case). The format of directory entries is:
typedef struct // This is the Directory Entry stored in resources
{
BYTE Width;
BYTE Height;
BYTE Colors;
BYTE Reserved;
WORD Planes;
WORD BitsPerPixel;
DWORD ImageSize;
WORD ResourceID
} IconDirResEntry, *PIconDirResEntry;
A good description of this can be found in
https://www.codeguru.com/cpp/w-p/win32/tutorials/article.php/c12873/Hacking-Icon-Resources.htm
In my case, resource "MYICON" is the RT_GROUP_ICON icon directory which contains ID value of "1".

WinAPI CreateDialog Ressource not found error 0x715

Like often, I post here after several hours of research and tries without any success
I have this old dll written in C. For the moment, it has no interface but I need to add a dialog box to it.
I work with VS2017 and tried the following :
Using VS2017 ressource editor, I added a dialog box (id : IDD_DIALOG_REPLAY, automatically defined to 101 in resource.h file by resource editor) and added the following code to create my dialog box :
INITCOMMONCONTROLSEX InitCtrls;
InitCtrls.dwSize = sizeof(InitCtrls);
InitCtrls.dwICC = ICC_LINK_CLASS | ICC_STANDARD_CLASSES | ICC_WIN95_CLASSES;
InitCommonControlsEx(&InitCtrls);
HWND hDialog = 0;
hDialog = CreateDialog(pSGL->hInstance,MAKEINTRESOURCE(IDD_DIALOG_REPLAY),NULL,WndProc);
if (!hDialog)
{
char buf [100];
wsprintf (buf, "Error x%x", GetLastError ());
MessageBox (0, buf, "CreateDialog", MB_ICONEXCLAMATION | MB_OK);
return 1;
}
ShowWindow(hDialog, SW_SHOW);
Note 1 : The message loop is already present in another dll executed in the same thread
Note 2 : in a first time, I use a call back function WndProc which is pretty standard and which basically executes the DefWindowProc function
When I compile my dll (with ressource compilation verbose option set), I get the following messages :
1>Writing DIALOG:101, lang:0x40c, size 452.
1>Writing AFX_DIALOG_LAYOUT:101, lang:0x40c, size 2.
When I open the binary of my dll in VS2017, I can see that there is a dialogbox id 101
=> The dialog box is actually present in my binary file.
But when I execute it, I get an error 0x715 : ERROR_RESOURCE_TYPE_NOT_FOUND and of course, the dialog box is not created.
Note : this happen, no matter if the dialog box contains controls or if it is empty
I have absolutly no clue why this is happening. Any help would be really welcome.
Thanks in advance,
Antoine
Ok,thanks to Hans, I found the reason.
I was using the exe hInstance and so, the program was looking for the dialog box inside the exe and not inside the dll.
Changing the hInstance to the dll one fixed my issue.
Thanks again Hans

Have dll import symbols from its calling .exe

Related to but not equivalent to DLL Get Symbols From Its Parent (Loader)
Is there a way to convince the Windows loader to resolve a particular symbol referenced by A.dll from either the loading executable or an intermediate dll without specifying the file to resolve symbols from in A.dll?
It's pretty obvious how to do it if the loading .exe has a known name, but if it isn't ...
Here's a good reason why you'd actually want to do this: https://www.gnu.org/software/libc/manual/html_node/Replacing-malloc.html
If this can be done, a good answer would say how to do it some way or another.
I'm half-expecting the answer is it can't be done. In that case, a good answer would show why this is impossible. "The build tools don't support this." is a bad answer.
when we use import we need exactly indicate module name and function name. and we can not use complex algorithms. also for exe not exist well known alias which we can use in place exactly exe name. for compare: in case get GetModuleHandle we can use NULL for get handle to the file used to create the calling process (.exe file). but in case LoadLibraryExW we can not use 0 or empty string (L"") or some another alias for say - we want handle to exe. when loader load our module - he read dll name from IMAGE_IMPORT_DESCRIPTOR and try found or load module with this name first by low level, private, core of LoadLibraryExW. here need exactly name. or load fail. as result use import - not a solution here, if we dont know exe name at build time
possible variant - resolve functions pointers yourself at runtime. here we can get exe HMODULE by GetModuleHandle(0). also if need we can search function not only in exe but somewhere else. can implement any search algorithm.
here exist several ways. for concrete example let we need get pointer to function with signature:
void WINAPI fn(int i);
we can declare pointer to this function and resolve it in runtime
void (WINAPI *fn)(int);
*(void**)&fn = GetProcAddress(GetModuleHandleW(0), "fn");
say on DLL_PROCESS_ATTACH
a slightly different solution (although at the binary level it is completely equivalent) declare function with __declspec(dllimport) attribute. this is for CL.EXE (more known as MSVC) compiler only. so
__declspec(dllimport) void fn(int i);
in this case CL yourself generate pointer to function with name __imp_ ## __FUNCDNAME__ name. so by fact the same as in first variant, when we declare pointer yourself. only difference in syntax and.. symbol name. it will be look like __imp_?fn2##YAXH#Z. problem here that __imp_?fn2##YAXH#Z not valid name for c/c++ - we can not direct assign value to it from c/c++. even if we declare function with extern "C" - function name will be containing # symbol (illegal for c++) for __stdcall and __fastcall functions, for x86. also name will be different for different platforms (x86, x64, etc). for access such names - need or use external asm file (for asm ? and # symbols valid in name) or use /alternatename linker option - for set alias for such name and access symbol via it. say like
__pragma(comment(linker, "/alternatename:__imp_?fn##YAXH#Z=__imp_fn"))
and init via
*(void**)&__imp_fn = GetProcAddress(GetModuleHandle(0), "fn");
another option use __declspec(dllimport) in function declarations + add import library, where all __imp___FUNCDNAME__ (such __imp_?fn2##YAXH#Z) is defined. (even if we have not such library we can easy create it yourself - all what need - correct function declarations with empty implementation). and after we add such import lib to linker input - add /DELAYLOAD:dllname where dllname - exactly name from import lib. sense that this dllname will(can) be not match to exe - all what need - it must be unique. and we need yourself handle delayload (called when we first time call fn). for implement delayload we need implement
extern "C" FARPROC WINAPI __delayLoadHelper2(
PCImgDelayDescr pidd,
FARPROC * ppfnIATEntry
);
we can implement it yourself, or add delayimp.lib to our project. here (in delayimp.lib) the delayLoadHelper2 and implemented. however we must customize this process (default implementation(look in /include/DelayHlp.cpp) will be use LoadLibraryExA with dllname which is not excepted in our case - otherwise we can be simply use import as is). so we need mandatory implement __pfnDliNotifyHook2:
for example:
FARPROC WINAPI MyDliHook(
unsigned dliNotify,
PDelayLoadInfo pdli
)
{
switch (dliNotify)
{
case dliNotePreLoadLibrary:
if (!strcmp(pdli->szDll, "unique_exe_alias"))
{
return (FARPROC)GetModuleHandle(0);
}
}
return 0;
}
const PfnDliHook __pfnDliNotifyHook2 = MyDliHook;
we can look for dliNotePreLoadLibrary notification and instead default LoadLibraryEx(dli.szDll, NULL, 0); use GetModuleHandle(0); for get base of exe.
the "unique_exe_alias" (which linker got from import library) here play role not real exe name, which is unknown, but unique tag(alias) for exe

ShellExecuteEx silently fails in 64-bit call to hidden file

I have a 64-bit console app that just wants to open a file. File's hidden attribute is set (the file is hidden). The code below fails only on some machines. ShellExecuteEx would actually return TRUE, but the .txt file would not open in Notepad, and hProcess member of SHELLEXECUTEINFO structure remains zero after the call.
CoInitializeEx(NULL, COINIT_APARTMENTTHREADED | COINIT_DISABLE_OLE1DDE);
SHELLEXECUTEINFO sei = {0};
sei.cbSize = sizeof(SHELLEXECUTEINFO);
sei.fMask = SEE_MASK_NOCLOSEPROCESS;
sei.hWnd = GetConsoleWindow();
sei.lpVerb = _T("open");
sei.nShow = SW_SHOWNORMAL;
sei.lpFile = _T("C:\\some\\existing\\file.txt");
BOOL bRC = ShellExecuteEx(&sei);
MessageBox(GetConsoleWindow(), sei.lpFile, sei.lpVerb, MB_OK);
Call to MessageBox is there just so that there's enough time for ShellExecuteEx to do its magic. The .txt file will open in Notepad if at least one of the following conditions is met:
.txt file is not hidden
the calling process is 32-bit, instead of 64-bit
the machine is some other (can't figure out what is the difference between machines, but the file is not open in at least one Vista and one Windows 8.1)
lpVerb is nullptr, "openas" or "properties" (which will of course just show file's properties), instead of "open", "edit", or even "runas" (for .exe instead of .txt)
What is going on here? Windows Explorer properly opens the hidden file, because it uses null verb, but I have to use the verb ("runas" actually, but please don't be distracted by this info). It should work with "open" as well. What am I doing wrong?
Turns out the problem was in one Explorer extension. Ironically (but unsurprisingly) it was our own extension. If the extension is unregistered the problem is gone, and ShellExecute works again for all verbs.

VS2008 to VS2010 - Problematic Configuration and Upgrade - Newbie

I have updated this question with an executive summary at the start below. Then, extensive details follow, if needed. Thanks for the suggestions.
Exec Summary:
I am a novice with VS. I have a problem with some inherited code. Code builds and executes fine on VS2008 (XP64). Same code will either build and not run, or fail to build on XP64 or W7 with VS2008 and/or VS2010. After changing some compiler options, I managed to get it to run without an issue on VS2010 on XP64; however, on W7, no luck.
I eventually discovered that the heap is getting corrupted.
Unhandled exception at 0x76e540f2 (ntdll.dll) in ae312i3.3.exe: 0xC0000374: A heap has been corrupted.
I am not familiar with how to consider fixing a heap problem; perhaps there is an issue with the pointers in the existing code that points to memory in use by another thread or program, corrupted ntdll.dll file, other?
Rebooting PC to check if ntdll.dll was corrupted didn't help. Changed debug settings, and received the following feedback:
HEAP[ae312i3.3.exe]: Invalid address specified to RtlSizeHeap( 0000000000220000, 000000002BC8BE58 )
Windows has triggered a breakpoint in ae312i3.3.exe.
This may be due to a corruption of the heap, which indicates a bug in ae312i3.3.exe or any of the DLLs it has loaded. This may also be due to the user pressing F12 while ae312i3.3.exe has focus.
It appears that when it crashes, C++ is returning a boolean variable to an expression of the form
While (myQueryFcn(inputvars))
QUESTIONS:
So, is it not returning a C++ boolean to a VB boolean? I do believe that the two are different representations (one uses True/False, the other an integer?) Could this be an issue? If so, why was it NOT an issue in VB2008?**
Or, perhaps it is that the C++ code has written to allocated memory, and upon returning to VB, it crashes???
** I have recently learned of 'Insure++', and will be trying to use it to track down the issue. Any suggestions on its use, other possible insight? **
I would appreciate any suggestions. Thanks again.
.
.
.
.
.
DETAILS THAT LED TO THE ABOVE SUMMARY (below):
I am a novice with VS2010; familiar with programming at an engineering application level (Python, Fortran, but been decades since I used C++ extensively), but not a professional programmer.
I have a solution that consists of multiple projects, all in VS2008. Projects are:
Reader (C++ project; utilizes 3rd party DLLs)
Query (C++ project; depends upon Reader)
Main (VB; depends upon Reader and Query).
The following applies to XP64 OS.
The solution and projects were written, built, and released by someone other than myself.
I have taken the existing files, and made a copy, placed in a directory of my choice, and simply opened in VS2010 (VS2008 is not installed on my PC). I was able to successfully build (with many warnings though - more on that later) ; but when I ran the executable, it would reach a point and crash. After much trial and error, I discovered that modification of compiler settings resolved the issue for me as follows:
It would build and execute in DEBUG configuration, but no the Release. I found that the in the Query project Property Page / Configuration Properties / C++ / Optimization / Optimization --> the Release (x64) configuration utilized 'Maximize Speed (/O2) while the Debug used 'Disabled (/Od)' --> so I switched to 'Disabled (/Od).
Also, Query's project Property Page / Configuration Properties / General / Whole Program Optimization --> needed to be set to 'Use Link Time Code Generation'.
The above build and ran successfully on XP64 in VS2010.
Next, I simply copied the files and placed a copy on a W7 machine with VS2010. Opened the solution via 2010, and it 'upgraded' the files automatically. When I launch VS2010, it automatically indicates the 4 following warnings. They are:
Operands of type Object used for operator '&'; runtime errors could occur. In file 'CobraIFile.vb', Line 1845, Column 37.
identical error completely
Accesss of shared member, constant member, enum member or nested type through an instance; qualifying expression will not be evaluated. In file 'FileWriter.vb', Lines 341, Columns 51
Operands of type Object used for operator '='; use the 'Is' operator to test object identity. In file 'FormMain.vb'; Line 4173, Column 32.
Code for warnings in 1 & 2 are as follows
ValueStr = String.Empty
For iCols = 0 To DGrid.Columns.Count - 1
ValueStr &= DGrid.Item(iCols, iRows).Value & ";" // THIS IS WARNING LINE!!!
Next
Code for warning 3:
With FormMain
WriteComment("")
WriteComment("Generated by :")
WriteComment("")
WriteComment(" Program : " & .PROGRAM.ToUpper) // THIS IS WARNING LINE!!!
Code for warning 4:
' Compare material against the material table
For iRowMat As Integer = 0 To matCount - 1
' Ignore new row
If Not .Rows(iRowMat).IsNewRow Then
' Check material description
// LINE BELOW IS WARNING LINE!!!
If .Item("ColMatDesc", iRowMat).Value = matDesc Then
DataGridMatProp.Item("ColMatIdx", iRow).Value = .Item("ColMatFile", iRowMat).Value
Exit For
End If ' Check description
End If ' Check new row
Next iRowMat
When I build the solution, it will successfully build without errors (but many warnings), and when I run the executable, it successfully loads the GUI, but at some point crashes while executing either the Query or Reader projects (after taking actions with gui buttons) with the following information:
C:\Users\mcgrete\AppData\Local\Temp\WER5D31.tmp.WERInternalMetadata.xml
C:\Users\mcgrete\AppData\Local\Temp\WER68E6.tmp.appcompat.txt
C:\Users\mcgrete\AppData\Local\Temp\WER722A.tmp.mdmp
I was unable to utilize the information in the three files above (ignorant of how to consider to do so).
The warnings I receive in W7 are very similar / if not identical to that in XP64; they are along the lines of the following types, and there are over 1,600 of them. Add to the warning types below the original 4 warnings listed ealier above. With my success in running on XP64, and not in W7, I was assuming/hoping that these would not require to individually be addressed, but are only warnings.
Warning C4267: 'argument' : conversion from 'size_t' to 'int', possible loss of data. C:\Users\mcgrete\Documents\iCOBRA\pts\p312\exec\win64\6111\include\atr_StringBase.h 351 1 Reader
Warning C4018: '<' : signed/unsigned mismatch C:\Users\mcgrete\Documents\iCOBRA\pts\p312\exec\win64\6111\include\omi_BlkBitVectTrav.h 69 1 Reader
Warning C4244: 'initializing' : conversion from 'double' to 'float', possible loss of data. C:\Users\mcgrete\Documents\iCOBRA\pts\p312\exec\win64\6111\include\g3d_Vector.h 76 1 Reader
Warning C4244: 'initializing' : conversion from 'double' to 'float', possible loss of data. C:\Users\mcgrete\Documents\iCOBRA\pts\p312\exec\win64\6111\include\g3d_Vector.h 76 1 Reader
Warning C4800: 'int' : forcing value to bool 'true' or 'false' (performance warning). C:\Users\mcgrete\Documents\iCOBRA\pts\p312\exec\win64\6111\include\rgnC_Region.h 219 1 Reader
Warning LNK4006: "public: class ddr_ShortcutImpl const & __cdecl cow_COW,struct cow_Virtual > >::ConstGet(void)const " (?ConstGet#?$cow_COW#V?$ddr_ShortcutImpl#VkmaC_Material####U?$cow_Virtual#V?$ddr_ShortcutImpl#VkmaC_Material########QEBAAEBV?$ddr_ShortcutImpl#VkmaC_Material####XZ) already defined in ABQDDB_Odb_import.lib(ABQDDB_Odb.dll); second definition ignored C:\Users\mcgrete\Documents\iCOBRA\pts\p312\source\312i3.3\Reader\ABQSMAOdbCore_import.lib(ABQSMAOdbCore.dll) Reader
Warning LNK4221: This object file does not define any previously undefined public symbols, so it will not be used by any link operation that consumes this library. C:\Users\mcgrete\Documents\iCOBRA\pts\p312\source\312i3.3\Reader\ABQSMAOdbCore_import.lib(ABQSMAOdbCore.dll) Reader
Warning C4996: 'sprintf': This function or variable may be unsafe. Consider using sprintf_s instead. To disable deprecation, use _CRT_SECURE_NO_WARNINGS. See online help for details. C:\Users\mcgrete\Documents\iCOBRA\pts\p312\source\312i3.3\Query\Query.cpp 271 1 Query
Warning MSB8004: Output Directory does not end with a trailing slash. This build instance will add the slash as it is required to allow proper evaluation of the Output Directory. C:\Program Files (x86)\MSBuild\Microsoft.Cpp\v4.0\Microsoft.CppBuild.targets 299 6 Query
Now to my request for help:
I must clarify, I am willing to dig into the warnings above in detail; however, I have not done so as before investing that effort and not having written code to begin with, I am simply trying to understand what might be the true root cause, then focus efforts in that direction.
I was disappointed with the XP64 issues I experienced, and was unsure if the changes required to the configuration were required, or if the changes that I made were only actually a 'work-around' to an unidentified problem?
I expected that once the XP64 VS2010 version of the solution was operable, that it would transfer to W7 without an issue, as the software build and ran fine with VS2008 and XP64. Is that a poor assumption? What might I be missing?
Should I consider attempting to modify the configurations again, or is the root cause likely associatd with the warnings indicated above? If the warnings, why were they apparently non-issues in VS2008 - did changes in VS2010 effectively lead to generation of actual runtime errors where in VS2008 I was luckily 'spared' the pain?
I appreciate any guidance and insight on how to proceed, as from my limited experience, it appears from searches on the web that there were numerous compiler bugs or related in VS2010. Not sure if any are related to my issues, if the numerous warnings are actually a problem and the code needs quite a bit of cleaning up, or if there are simply some configuration issues that I may have to deal with.
FYI - The latest update/SP to VS2010 that I have installed is VS10SP1-KB2736182.exe. I have also trid to use the debugger, but was unable to get it to stop at breakpoints in my Query or Reader project codes, even while running VS2010 as administrator. W7 does have .NET Framework 4.0 Multi-Targeting Pack installed, and my solution is configured to use .NET Framework 4.0 Client Profile.
Thanks in advance!
UPDATE March 18, 2013
I didn't know how to reply to my own question, so here is an update.
I still could not manage to get the debugger working; so, I did it the old fashioned way - added various MessageBoxs to find where it was crashing.
A. The Main.vb program calls a function in the 'Query' project
OdbQueryGetIncrement(str_out, vec_ptr)
B. Then, the function executes through 100%, attempting to return a boolean...here is code with some old fashioned debugging code added...
//Gets the next item in a list.
// Returns false if there is the vector is empty.
// NOTE: Once an element is returned it is removed from the list.
bool __stdcall OdbQueryGetItem(
char* &str_out, // RETURN Next item in list.
void * vec_ptr, // Pointer to the vector of pointers.
int index) // Index of pointers vector to return next item of.
{
// Cast the point into an array of pointers
std::vector<std::string>* *vec_temp = (std::vector<std::string>* *) vec_ptr;
bool bool_out = false;
char vectempsize[1000];
int TEM1;
char temp[1000];
TEM1 = vec_temp[index]->size();
// Check vector is valid
if (vec_temp) {
if(vec_temp[index]->size() >= index)
{
sprintf(temp,"value: %d\n",(int)bool_out);
::MessageBoxA(0, (LPCSTR) temp, (LPCSTR) "OdbQuery.dll - bool_out", MB_ICONINFORMATION);
sprintf(temp,"value: %d\n",(int)index);
::MessageBoxA(0, (LPCSTR) temp, (LPCSTR) "OdbQuery.dll - index", MB_ICONINFORMATION);
sprintf(vectempsize,"value: %d\n",(int)TEM1);
::MessageBoxA(0, (LPCSTR) temp, (LPCSTR) "OdbQuery.dll - index", MB_ICONINFORMATION);
}
if (!vec_temp[index]->empty()) {
// Get the next item in the list
std::string item = vec_temp[index]->front();
// Initialise ouput string
str_out = (char*)malloc( item.size()*sizeof(char) );
sprintf(str_out, "%s", item.c_str());
::MessageBoxA(0,(LPCSTR) str_out, (LPCSTR) "hello", 0);
// Remove first item from the vector
vec_temp[index]->erase(vec_temp[index]->begin());
bool_out = true;
}
}
sprintf(temp,"value: %d\n",(int)bool_out);
::MessageBoxA(0, (LPCSTR) temp, (LPCSTR) "OdbQuery.dll - bool_out", MB_ICONINFORMATION);
return bool_out;
}
The code starts out with bool_out=false as expected (verified with MessageBox value=0 output)
The code reads and outputs index = 2 with the MessageBox...
The code reads and outputs TEM1=vec_temp[index]->size() as a value=2 with the MessageBox...
The code outputs bool_out as true (value=1) with the MessageBox...
Then, the code crashes. A MessageBox that was placed immediately after the line that calls the code above never is executed.
The output from VS2010 is "The program '[6892] ae312i3.3.exe: Managed (v4.0.30319)' has exited with code -2147483645 (0x80000003)."
I am lost as to why the execution would die while returning from this function.
Is there some possible issue with compiler settings or bugs?
Any help is appreciated!
MORE INFORMATION
Hello, I modified some settings on the Properties Page to attempt to get the debugger to give me more information. This has resulted in more information as follows:
Unhandled exception at 0x76e540f2 (ntdll.dll) in ae312i3.3.exe: 0xC0000374: A heap has been corrupted.
I am not familiar with how to consider fixing a heap problem; perhaps there is an issue with the pointers in the existing code that points to memory in use by another thread or program, corrupted ntdll.dll file, other?
I will try rebooting PC to see if that helps, though I have little hope for that...didn't help.
Found option in Debugger to 'Enable unmanaged code debugging', checked it; cleaned; rebuild; run with debug...
Output more descriptive --
HEAP[ae312i3.3.exe]: Invalid address specified to RtlSizeHeap( 0000000000220000, 000000002BC8BE58 )
Windows has triggered a breakpoint in ae312i3.3.exe.
This may be due to a corruption of the heap, which indicates a bug in ae312i3.3.exe or any of the DLLs it has loaded. This may also be due to the user pressing F12 while ae312i3.3.exe has focus.
It appears that when it crashes, C++ is returning a boolean variable to an expression of the form
While (myQueryFcn(inputvars))
So, is it not returning a C++ boolean to a VB boolean? I do believe that the two are different representations (one uses True/False, the other an integer?) Could this be an issue? If so, why was it NOT an issue in VB2008?
I solved my own problem; the root cause of the problem was as follows.
Root Cause:
VisualBasic (VB) called C++.
VB created a string and sent to C++. Previous developer/coder allocated memory in C++ for the same string.
When execution of C++ code ended, C++ appears to have terminated the memory allocation established by VB and C++.
Solution:
1. Removed memory allocation in C++ code (below).
str_out=(char*)malloc( (item.size()+1)*sizeof(char) );
Modified VB code to use a StringBuilder type, rather than string.
Dim str_out As StringBuilder = New StringBuilder(5120)
See: return string from c++ function to VB .Net

Resources