Crash During ChoosePixelFormat in VB6 - windows

I've been tasked to maintain a VB6 app; I've got some OpenGL experience but all using SDL and this one seems to manually handle all it's contexts.
The program appears to crash on a call to ChoosePixelFormat. I'm passing in a non-null handle. It works on most systems, so I'm guessing its a graphics driver issue, but it appears to be up to date.
Does anyone know why this function would crash instead of, say, returning an error code? I understand the PIXELFORMATDESCRIPTOR may have values that don't exist on a given system:
.nSize = Len(glPFD)
.nVersion = 1
.dwFlags = PFD_DRAW_TO_BITMAP Or PFD_SUPPORT_OPENGL Or PFD_SUPPORT_GDI
.iPixelType = PFD_TYPE_RGBA
.cColorBits = 32
.cDepthBits = 16
.iLayerType = PFD_MAIN_PLANE
But would that make it crash?
Edit: On the system on which this fails, it works with a previous version in which this code was identical. I read a forum somewhere that said loading/unloading opengl32.dll can cause gdi32.dll to break things. This might be the case.
Edit: I've attached the full code below (it's kind of ugly, but it helped me narrow down where the crash was). No functions return 0 before ChoosePixelFormat crashes.
hDCdib = CreateCompatibleDC(0) 'DC compatible with screen
Call MsgBox("Create Compatible DC " + CStr(hDCdib))
'Setup a Device Independent Bitmap Section
With bmInfo.bmiHeader
.biSize = Len(bmInfo)
If lngWidth = 0 Then
.biHeight = objDest.ScaleY(objDest.Height, vbTwips, vbPixels)
.biWidth = objDest.ScaleX(objDest.Width, vbTwips, vbPixels)
Else
.biHeight = objDest.ScaleY(lngHeight, vbTwips, vbPixels)
.biWidth = objDest.ScaleX(lngWidth, vbTwips, vbPixels)
End If
.biPlanes = 1
.biBitCount = 32
.biCompression = BI_RGB 'No Compression
.biSizeImage = 0
.biClrImportant = 0
.biClrUsed = 0
.biXPelsPerMeter = 0
.biYPelsPerMeter = 0
End With
Call MsgBox("after header")
hDIBSection = CreateDIBSection(hDCdib, bmInfo, DIB_RGB_COLORS, hBitMapData, _
&O0, &O0)
Call MsgBox("CreateDIBSection" + CStr(hDIBSection))
'Select the DIB into a device context
hBitMapOld = SelectObject(hDCdib, hDIBSection)
Call MsgBox("SelectObject " + CStr(hBitMapOld))
'Create an OpenGL drawing area in that DIB
With glPFD
.nSize = Len(glPFD)
.nVersion = 1
.dwFlags = PFD_DRAW_TO_BITMAP Or PFD_SUPPORT_OPENGL Or PFD_SUPPORT_GDI
.iPixelType = PFD_TYPE_RGBA
.cColorBits = 32
.cDepthBits = 16
.iLayerType = PFD_MAIN_PLANE
End With
Call MsgBox("glPFD ")
'Ignacio issue is in the next line about ChoosePixelFormat
lngFormat = ChoosePixelFormat(hDCdib, glPFD)
Call MsgBox("ChoosePixelFormat " + CStr(lngFormat))
Htemp = SetPixelFormat(hDCdib, lngFormat, glPFD)
Call MsgBox("SetPixelFormat " + CStr(Htemp))
hGLrc = wglCreateContext(hDCdib)
Call MsgBox("wglCreateContext " + CStr(hGLrc))
wglMakeCurrent hDCdib, hGLrc
Call MsgBox("wglMakeCurrent ")

An OpenGL context on a DIB isn't accelerated, so you'll get a software renderer (at least that was the case some years back and I doubt it's changed, although I could be wrong). If you want to do hardware accelerated rendering, create a window and use that to generate your GL context. If you want, hide the window.

Related

Error 65535 from CommDlgExtendedError after calling GetFileSaveName

So, I'm calling GetFileSaveName from some C code. The C code is called from PowerBuilder, but I really don't think that is relevant.
On just one user's computer currently (Windows 10, updates done), it returns 0, then CommDlgExtendedError return 65535 (CDERR_DIALOGFAILURE), which the docs say means "The dialog box could not be created. The common dialog box function's call to the DialogBox function failed. For example, this error occurs if the common dialog box call specifies an invalid window handle."
I know that the common dialogs aren't completely broken on her PC - I can pull up a File Open or Save dialog in Notepad, for instance.
I had a bug in my code giving the same error number in the past, where in the OPENFILENAME structure I passed in to GetOpenFileName I had set the hWndOwner variable to a handle to a window that no longer exists, but in this case it is being set to 0, so that's not the problem.
The same code is working for thousands of other users of our software! Any bright ideas what could be going on? Thanks.
OK, people have asked me to post my code initializing the OpenFileName structure. It's PowerScript (PowerBuilder's coding language) but should be pretty comprehensible).
OPENFILENAME iOFN
CONSTANT long CHARSIZE = 2
aul_flags = OFN_HIDEREADONLY + OFN_EXPLORER +
OFN_FILEMUSTEXIST+ OFN_NOCHANGEDIR + OFN_DONTADDTORECENT
RtlZeroMemory(iOFN, ll_sizeof)
// initialize structure
iOFN.lStructSize = ll_sizeof
iOFN.nFilterIndex = 1
iOFN.nMaxFile = MAX_LENGTH
iOFN.hWndOwner = il_hWnd
iOFN.Flags = aul_flags + OFN_ENABLESIZING /* needed when using hook procedure */
// allocate memory and copy title
ll_length = Len(as_title) * CHARSIZE
iOFN.lpstrTitle = LocalAlloc(LMEM_ZEROINIT, ll_length + 2)
RtlMoveMemory(iOFN.lpstrTitle, as_title, ll_length)
// allocate memory and copy filter
this.of_Parse(",", as_filter, ls_filter)
li_max = UpperBound(ls_filter) /* count of 1-based array elements */
For li_cnt = 1 To li_max
ll_length = this.of_StringToChar(Trim(ls_filter[li_cnt]), lc_filter)
Next
ll_length = UpperBound(lc_filter) * CHARSIZE
iOFN.lpstrFilter = LocalAlloc(LMEM_ZEROINIT, ll_length)
RtlMoveMemory(iOFN.lpstrFilter, lc_filter, ll_length)
// allocate memory and copy default extension (if given)
If as_extension <> "" Then
ll_length = Len(as_extension) * CHARSIZE
iOFN.lpstrDefExt = LocalAlloc(LMEM_ZEROINIT, ll_length)
RtlMoveMemory(iOFN.lpstrDefExt, as_extension, ll_length)
End If
// allocate memory and copy initialdir (if given)
If as_initdir <> "" Then
ll_length = Len(as_initdir) * CHARSIZE
iOFN.lpstrInitialDir = LocalAlloc(LMEM_ZEROINIT, ll_length)
RtlMoveMemory(iOFN.lpstrInitialDir, as_initdir, ll_length)
End If
// allocate memory for returned data
lc_pathname = Space(MAX_LENGTH)
iOFN.nMaxFile = MAX_LENGTH
iOFN.lpstrFile = LocalAlloc(LMEM_ZEROINIT, MAX_LENGTH)
If as_initialfile <> "" Then
ll_length = Len(as_initialfile) * CHARSIZE
RtlMoveMemory(iOFN.lpstrFile, as_initialfile, ll_length)
End If
// display dialog box
lb_return = GetOpenFileName(iOFN)
I recommend you to check if there is any problem with the OPENFILENAME parameter configuration. There are many parameters in OPENFILENAME that need to be configured before GetSaveFileName can be used. Maybe there is something wrong.
TCHAR szFilename[MAX_PATH] = TEXT("");
BOOL bResult = FALSE;
DWORD dwError = NOERROR;
OPENFILENAME ofn = { 0 };
ofn.lStructSize = sizeof(OPENFILENAME);
ofn.lpstrFilter = TEXT("All Files\0*.*\0\0");
ofn.lpstrFile = szFilename;
ofn.nMaxFile = MAX_PATH;
ofn.Flags = OFN_EXPLORER |
OFN_ENABLEHOOK |
OFN_HIDEREADONLY |
OFN_NOCHANGEDIR |
OFN_PATHMUSTEXIST;
bResult = GetSaveFileName(&ofn);
if (bResult == FALSE)
{
dwError = CommDlgExtendedError();
return dwError;
}

BITMAPV5HEADER Size?

I'm trying to use BITMAPV5HEADER with CreateDIBSection to get a uint8[] in form of RGBA. I am doing this from ctypes but this isn't a ctypes question but I'll post the code so you can see it. The ctypes guys won't know whats up here it's a winapi thing.
I double checked my struct, function, and type declares but I keep getting GetLastError of 87 which is invalid parameter after calling CreateDIBSection. If I set the bv5Size to the size of BITMAPINFOHEADER which is 40 it works but is treated as BITMAPINFO header and the red,blue,green,alpha masks dont affect it, as I still get BGRA.
So I was wondering what is the size supposed to be for BITMAPV5HEADER on 32bit and 64bit please. For me I'm getting 124:
"ostypes.TYPE.BITMAPINFOHEADER.size:" 40
"ostypes.TYPE.BITMAPV5HEADER.size:" 124
This is my ctypes code just to show that everything is correct:
var bmi = ostypes.TYPE.BITMAPV5HEADER();
bmi.bV5Size = ostypes.TYPE.BITMAPV5HEADER.size;
bmi.bV5Width = nWidth; //w;
bmi.bV5Height = -1 * nHeight; //-1 * h; // top-down
bmi.bV5Planes = 1;
bmi.bV5BitCount = nBPP; //32;
bmi.bV5Compression = ostypes.CONST.BI_BITFIELDS;
bmi.bV5RedMask = ostypes.TYPE.DWORD('0x00FF0000');
bmi.bV5GreenMask = ostypes.TYPE.DWORD('0x0000FF00');
bmi.bV5BlueMask = ostypes.TYPE.DWORD('0x000000FF');
bmi.bV5AlphaMask = ostypes.TYPE.DWORD('0xFF000000'); // 0x00000000 for opaque, otherwise 0xff000000
var cBmi = ctypes.cast(bmi.address(), ostypes.TYPE.BITMAPINFO.ptr); // cBmi is now a pointer so no need to pass cBmi.address() to CreateDIBSection
var pixelBuffer = ostypes.TYPE.BYTE.ptr();
var hbmp = ostypes.API('CreateDIBSection')(hdcMemoryDC, cBmi, ostypes.CONST.DIB_RGB_COLORS, pixelBuffer.address(), null, 0);
This is ctypes so I don't have to memset bmi after creation as by default it is initialized memset 0.
Thanks

ShellExecuteEx wierd behavior in Windows 8

The code below for file with no-extension brings up a dialog shown below listing applications that can be used to open the file. This behavior is seen only from Windows-8. And the applications listed in the dialog are taken from HKEY_LOCAL_MACHINE\SOFTWARE\Classes*\OpenWithList. Is there anyway to suppress this dialog and get a behavior similar to old platforms?
-Karthik
SHELLEXECUTEINFO shinfo;
unsigned long mask = SEE_MASK_FLAG_NO_UI;
memset(&shinfo,0,sizeof(shinfo));
shinfo.cbSize = sizeof(shinfo);
shinfo.fMask = SEE_MASK_FLAG_DDEWAIT | mask;
shinfo.hwnd = NULL;
shinfo.lpVerb = "open";
shinfo.lpFile = prog;
shinfo.lpParameters = NULL;
shinfo.lpDirectory = 0;
shinfo.fMask = SEE_MASK_FLAG_NO_UI;
shinfo.nShow = SW_SHOWDEFAULT;
rc = ShellExecuteEx(&shinfo);
I'd suspect there is no default action associated with open on Windows 8 for files without an extension. I confirmed this using your code (a Delphi version of it, anyway) on Windows 7.
Running the code with shinfo.lpverb set to 'open' caused ShellExecuteEx to return FALSE, and GetLastError did indeed return ERROR_NO_ASSOCIATION. However, changing lpVerb to NULL (nil in Delphi) instead displayed the standard Win7 Open With dialog, just like your code does on Windows 8.
Here's a modified version of your code for testing:
SHELLEXECUTEINFO shinfo;
memset(&shinfo, 0, sizeof(shinfo));
shinfo.cbSize = sizeof(shinfo);
shinfo.fMask = SEE_MASK_FLAG_DDEWAIT | SEE_MASK_FLAG_NO_UI;
shinfo.lpVerb = NULL;
shinfo.lpFile = prog;
shinfo.hwnd = NULL;
shinfo.lpParameters = NULL;
shinfo.lpDirectory = 0;
shinfo.nShow = SW_SHOWDEFAULT;
rc = ShellExecuteEx(&shinfo);
Here's my test Delphi code for comparison (a quite literal translation of your C++ code):
var
shInfo: TShellExecuteInfo;
FillChar(shInfo, SizeOf(shInfo), 0); // Same result as memset()
shInfo.cbSize := SizeOf(shInfo);
shInfo.fMask := SEE_MASK_FLAG_NO_UI or SEE_MASK_FLAG_DDEWAIT;
shInfo.Wnd := 0;
shInfo.lpVerb := nil; // Also tested with 'open'
shInfo.lpDirectory := 'D:\TempFiles'; // Path to my no-extension file
shInfo.lpFile := 'datafile'; // My test file with no ext
shInfo.nShow := SW_SHOWDEFAULT;
if not ShellExecuteEx(#shInfo) then
ShowMessage(SysErrorMessage(GetLastError)); // Readable error message
A quick change to your code to replace "open" with NULL as the lpVerb should confirm.
You can also confirm my suspicion fairly easily by right-clicking a file with no extension in Win8's Explorer, and checking the bold default action at the top of the context menu. If there is no bold option, or if it's anything but open, my suspicions are correct.

Windows Common File Dialog Will Not Display While Running in Visual Studio 2010 Debugger

I have a VS2010 C++ application that uses the Windows Common File Dialog in a pretty ordinary fashion. My company just updated my workstation to a nice quad core CPU with Windows 7 whereas my previous system was still running XP. When I run my application inside the Visual Studio debugger, any attempt to call the CFD seems to fail silently with the code throwing no apparant errors to the output window, and no dialog appearing. Outside the debugger things work just fine. I invoke the dialog pretty much the same way every time.
CString theFilterList = "CSV Import Files (*.csv)|*.csv";
theFilterList = theFilterList + "|All files (*.*)|*.*||";
// construct the common dialog
CFileDialog fileDlg(TRUE, NULL, NULL,OFN_ENABLESIZING | OFN_EXPLORER | OFN_FILEMUSTEXIST | OFN_PATHMUSTEXIST | OFN_HIDEREADONLY | OFN_ALLOWMULTISELECT, theFilterList,this);
POSITION pos;
// Initialize m_ofn structure
fileDlg.m_ofn.lpstrTitle = "Please select a UP supplied CSV file to prepare to load into SCT.";
// Create buffer for file names.
const DWORD bufferSize = (MAX_MULTISELECT_FILENAMES * MAX_FILENAMESIZE) + 1;
TCHAR* filenamesBuffer = new TCHAR[bufferSize];
// Initialize beginning and end of buffer.
filenamesBuffer[0] = NULL;
filenamesBuffer[bufferSize-1] = NULL;
// Attach buffer to OPENFILENAME member.
fileDlg.m_ofn.lpstrFile = filenamesBuffer;
fileDlg.m_ofn.nMaxFile = bufferSize;
if ( fileDlg.DoModal() != IDOK)
{
theCSVFilenameList.RemoveAll();
return;
}
pos = fileDlg.GetStartPosition();
while( pos )
{
theCSVFilenameList.Add(fileDlg.GetNextPathName( pos ));
}
I just had this problem. I found that my StackReserve size was too high. I reduced it and the dialog started coming up. Though in my case it was happening both in and out of the debugger.

GetOpenFileName fails in 64 bit, but works in 32Bit?

I have the following code, I use to Open a File Open Dialog using Win32 API. It works fine in 32bit, but fails when I use in a 64bit (In a DLL). What am I doing wrong?
char Filestring[256];
Filter = "OBJ files\0*.obj\0\0";
char* returnstring = NULL;
OPENFILENAME opf;
opf.hwndOwner = mainHWND;
opf.lpstrFilter = Filter;
opf.lpstrCustomFilter = 0;
opf.nMaxCustFilter = 0L;
opf.nFilterIndex = 1L;
opf.lpstrFile = Filestring;
opf.lpstrFile[0] = '\0';
opf.nMaxFile = 256;
opf.lpstrFileTitle = 0;
opf.nMaxFileTitle=50;
opf.lpstrInitialDir = Path;
opf.lpstrTitle = "Open Obj File";
opf.nFileOffset = 0;
opf.nFileExtension = 0;
opf.lpstrDefExt = "*.*";
opf.lpfnHook = NULL;
opf.lCustData = 0;
opf.Flags = (OFN_PATHMUSTEXIST | OFN_OVERWRITEPROMPT) & ~OFN_ALLOWMULTISELECT;
opf.lStructSize = sizeof(OPENFILENAME);
if(GetOpenFileName(&opf))
{
returnstring = opf.lpstrFile;
if (returnstring) {
result = returnstring;
}
}
EDIT: By failing, I meant that the Open File Dialog doesn't show up. The code still returns zero without any errors.
EDIT 2: I have called CommDlgExtendedError() and it returned 1. From the MSDN reference, does it mean the dialog has invalid lStructSize? I have checked the sizeof(OPENFILENAME) and it returned 140 bytes.
UPDATE: In my Project Settings, Under Code Generation the "Struct Member Alignment" is set to 4 Bytes(/Zp4). I changed this to default and it magically worked. Look for the answers and their comments below for more information.
You aren't initialising lpTemplateName and so it contains random stack noise. This in turn will lead to 'hInstance` being references which also contains stack noise.
When calling a function like this you should first of all zero out the struct and only fill in the fields that are non-zero. Something like this:
OPENFILENAME opf={0};
opf.lStructSize = sizeof(OPENFILENAME);
opf.hwndOwner = mainHWND;
opf.lpstrFilter = Filter;
opf.nFilterIndex = 1L;
opf.lpstrFile = Filestring;
opf.lpstrFile[0] = '\0';
opf.nMaxFile = 256;
opf.lpstrInitialDir = Path;
opf.lpstrTitle = "Open Obj File";
opf.lpstrDefExt = "*.*";
opf.Flags = OFN_PATHMUSTEXIST | OFN_OVERWRITEPROMPT;
There was no need to exclude OFN_ALLOWMULTISELECT explicitly since you were not including it in the first place!
EDIT
You state in a comment that this doesn't work. Calling CommDlgExtendedError is a good idea and should tell you why it fails.
You could also try to run the minimal possible GetOpenFileName which is this:
char Filestring[MAX_PATH] = "\0";
OPENFILENAME opf={0};
opf.lStructSize = sizeof(OPENFILENAME);
opf.lpstrFile = Filestring;
opf.nMaxFile = MAX_PATH;
GetOpenFileName(&opf);
I have the very same problem and a partial solution :
+ the simple following simple example (proposed abobe) was not working in x64 mode.
+ I changed the complie option "struct Member Alignment" from 1byte /Zp1 to default which solved this problem (by introducing others !!!)
char Filestring[MAX_PATH] = "\0";
OPENFILENAME opf={0};
opf.lStructSize = sizeof(OPENFILENAME);
opf.lpstrFile = Filestring;
opf.nMaxFile = MAX_PATH;
GetOpenFileName(&opf);
To find out more you should call CommDlgExtendedError to get the error code what went wrong. Besides this I would initialize all member of the struct to 0 with
ZeroMemory(&opf, sizeof(opf));
Since the file open dialog is in reality a COM component it could be worth to check out if your thread apartment state is different under 64 bit.
if( RPC_E_CHANGED_MODE == CoInitialize(NULL) )
ASSERT(FALSE); // MTA Apartment found
CoUnitialize()
Yours,
Alois Kraus
As a note in Microsoft Office 2010 64-bit we gave up and used the internal wrappers as the structure turned into 140 bytes and we were not sure how to change alignment.
Application.GetOpenFilename(FileFilter, FilterIndex, Title, ButtonText, MultiSelect)
and Application.GetSaveAsFilename(InitialFilename, FileFilter, FilterIndex, Title, ButtonText)
http://msdn.microsoft.com/en-us/library/ff834966.aspx
http://msdn.microsoft.com/en-us/library/microsoft.office.interop.excel._application.getopenfilename.aspx
Needless to say we think all individuals with fairly heavy applications in Excel should start considering other options as maintaining future versions across multiple clients and platforms may just be... insane!
I managed to get around this problem by setting the packing appropriately before including the header file. That way, for the purpose of this one function, we were using the 'default' 16 byte alignment, but did not have to change the packing alignment for the rest of our program:
#ifdef _WIN64
#pragma pack( push )
#pragma pack( 16 )
#include "Commdlg.h"
#pragma pack( pop )
#else
#include "Commdlg.h"
#endif // _WIN64

Resources