GetOpenFileName fails in 64 bit, but works in 32Bit? - windows

Question

I have the following code, I use to Open a File Open Dialog using Win32 API. It works fine in 32bit, but fails when I use in a 64bit (In a DLL). What am I doing wrong?
char Filestring[256];
Filter = "OBJ files\0*.obj\0\0";
char* returnstring = NULL;
OPENFILENAME opf;
opf.hwndOwner = mainHWND;
opf.lpstrFilter = Filter;
opf.lpstrCustomFilter = 0;
opf.nMaxCustFilter = 0L;
opf.nFilterIndex = 1L;
opf.lpstrFile = Filestring;
opf.lpstrFile[0] = '\0';
opf.nMaxFile = 256;
opf.lpstrFileTitle = 0;
opf.nMaxFileTitle=50;
opf.lpstrInitialDir = Path;
opf.lpstrTitle = "Open Obj File";
opf.nFileOffset = 0;
opf.nFileExtension = 0;
opf.lpstrDefExt = "*.*";
opf.lpfnHook = NULL;
opf.lCustData = 0;
opf.Flags = (OFN_PATHMUSTEXIST | OFN_OVERWRITEPROMPT) & ~OFN_ALLOWMULTISELECT;
opf.lStructSize = sizeof(OPENFILENAME);
if(GetOpenFileName(&opf))
{
returnstring = opf.lpstrFile;
if (returnstring) {
result = returnstring;
}
}
EDIT: By failing, I meant that the Open File Dialog doesn't show up. The code still returns zero without any errors.
EDIT 2: I have called CommDlgExtendedError() and it returned 1. From the MSDN reference, does it mean the dialog has invalid lStructSize? I have checked the sizeof(OPENFILENAME) and it returned 140 bytes.
UPDATE: In my Project Settings, Under Code Generation the "Struct Member Alignment" is set to 4 Bytes(/Zp4). I changed this to default and it magically worked. Look for the answers and their comments below for more information.

Answer 1

You aren't initialising lpTemplateName and so it contains random stack noise. This in turn will lead to 'hInstance` being references which also contains stack noise.
When calling a function like this you should first of all zero out the struct and only fill in the fields that are non-zero. Something like this:
OPENFILENAME opf={0};
opf.lStructSize = sizeof(OPENFILENAME);
opf.hwndOwner = mainHWND;
opf.lpstrFilter = Filter;
opf.nFilterIndex = 1L;
opf.lpstrFile = Filestring;
opf.lpstrFile[0] = '\0';
opf.nMaxFile = 256;
opf.lpstrInitialDir = Path;
opf.lpstrTitle = "Open Obj File";
opf.lpstrDefExt = "*.*";
opf.Flags = OFN_PATHMUSTEXIST | OFN_OVERWRITEPROMPT;
There was no need to exclude OFN_ALLOWMULTISELECT explicitly since you were not including it in the first place!
EDIT
You state in a comment that this doesn't work. Calling CommDlgExtendedError is a good idea and should tell you why it fails.
You could also try to run the minimal possible GetOpenFileName which is this:
char Filestring[MAX_PATH] = "\0";
OPENFILENAME opf={0};
opf.lStructSize = sizeof(OPENFILENAME);
opf.lpstrFile = Filestring;
opf.nMaxFile = MAX_PATH;
GetOpenFileName(&opf);

Answer 2

I have the very same problem and a partial solution :
+ the simple following simple example (proposed abobe) was not working in x64 mode.
+ I changed the complie option "struct Member Alignment" from 1byte /Zp1 to default which solved this problem (by introducing others !!!)
char Filestring[MAX_PATH] = "\0";
OPENFILENAME opf={0};
opf.lStructSize = sizeof(OPENFILENAME);
opf.lpstrFile = Filestring;
opf.nMaxFile = MAX_PATH;
GetOpenFileName(&opf);

Answer 3

To find out more you should call CommDlgExtendedError to get the error code what went wrong. Besides this I would initialize all member of the struct to 0 with
ZeroMemory(&opf, sizeof(opf));
Since the file open dialog is in reality a COM component it could be worth to check out if your thread apartment state is different under 64 bit.
if( RPC_E_CHANGED_MODE == CoInitialize(NULL) )
ASSERT(FALSE); // MTA Apartment found
CoUnitialize()
Yours,
Alois Kraus

Answer 4

As a note in Microsoft Office 2010 64-bit we gave up and used the internal wrappers as the structure turned into 140 bytes and we were not sure how to change alignment.
Application.GetOpenFilename(FileFilter, FilterIndex, Title, ButtonText, MultiSelect)
and Application.GetSaveAsFilename(InitialFilename, FileFilter, FilterIndex, Title, ButtonText)
http://msdn.microsoft.com/en-us/library/ff834966.aspx
http://msdn.microsoft.com/en-us/library/microsoft.office.interop.excel._application.getopenfilename.aspx
Needless to say we think all individuals with fairly heavy applications in Excel should start considering other options as maintaining future versions across multiple clients and platforms may just be... insane!

Answer 5

I managed to get around this problem by setting the packing appropriately before including the header file. That way, for the purpose of this one function, we were using the 'default' 16 byte alignment, but did not have to change the packing alignment for the rest of our program:
#ifdef _WIN64
#pragma pack( push )
#pragma pack( 16 )
#include "Commdlg.h"
#pragma pack( pop )
#else
#include "Commdlg.h"
#endif // _WIN64