Why might IsDialogMessage() never return? - winapi

I am debugging an application which, in its message loop, calls IsDialogMessage(). Occasionally, IsDialogMessage() never returns (where never is an interval greater than 1 hour). Based on the symbols for user32.dll available from Microsoft's symbol server, it appears to be stuck in GetNextDlgGroupItem() (or an internal variant of the same), iterating over some set of windows.
The application is multithreaded and frequently receives notification of external events, which arrive as DCOM calls. I suspect that such a call is handled incorrectly in a way that corrupts some window state. If I can learn what sort of state corruption might cause an infinite loop in IsDialogMessage(), I think I will be more easily able to identify the source of the corruption.

I know this is old, but answering for posterity since no one here mentioned it.
More than likely what is going on is trouble with the windows manager determining where to forward a message. If you have a hierarchy of windows, as you probably do, then you need to ensure that non-top-level windows that contain controls themselves must have the WS_EX_CONTROLPARENT style set. If it is a dialog, you use the DS_CONTROL style. The presence of these flags modify the behavior of IsDialogMessage; they identify a window as having its own controls which can receive focus and handle tab order, etc, rather than just being a control itself.
For example, if you have a main frame window, which has a child window with WS_EX_CONTROLPARENT, which has a child window without WS_EX_CONTROLPARENT, which has a child window that has focus, and you hit TAB, you will likely encounter an infinite loop at the same place you mention.
Setting the second child's extended style to include WS_EX_CONTROLPARENT will resolve the issue.

Are you maybe disabling controls (using ::EnableWindow()) without checking first whether that control has the focus? If yes, then the focus gets lost and GetNextDlgGroupItem() gets confused.

Another reason this can happen is if you reparent a modeless dialog. At least this can happen with wxWidgets...

I've made some investigation, trying to answer this question. But only in situation, when parent window is in native MFC project and child is a managed C# Windows Forms. If you have such situation, then you may try 3 resolutions:
Run MFC dialog message loop on Windows Forms side. Here is more info: Integrate Windows Forms Into Your MFC Applications Through C++ Interop
Create 2 threads: one for Windows Forms dialog and one for native dialog. Here you can create dialog in Windows Forms, then with SetParent() set it's parent to native dialog. But be ware: if you add TabControl to Windows Forms, than hang with "IsDialogMessage() never return" will occur.
Make a wrapper for Windows Forms dialog to use in native project. For ex., wrapper may be WPF, see here: Windows Form as child window of an unmanaged app
I've taken information mostly from: http://msdn.microsoft.com/en-us/library/ms229600.aspx
And the temporary cure can be to change focus behavior. For example, disable them, or SetFocus() to parent or child windows only. But I strongly recommend to investigate the real reason, why IsDialogMessage() never return in your case.

Related

App modal File dialogs on windows

How do i make the common File dialogs App modal using Common File Dialog API ? The dialogs come up modal with respect to the owner window. I want to block all the process' windows when a file dialog is up. In my current code, I am disabling all windows belonging to the app except the dialog parent and when the dialog is closed I enable them again. There should be a better/easy way of achieving application wide modality with Common File Dialogs. Please let me know if there is a standard solution for this.
Manually disabling and re-enabling is the only way I know of in Windows.
The traditional model for Windows applications is to have a single top-level window per instance. (Remember MDI apps?) Of course, there are exceptions, and many apps have always had floating tool palette windows. Nevertheless, the disable-the-parent model works for the lion's share of applications, and it's possible for many-window apps to do what you're doing with manually disabling the extra windows. Thus there isn't much demand for a more general solution.
If you wanted to re-architect things, you could have a master window that owns all the other top-level windows, and make the modal window use the master as a parent, but then you'd have to solve other problems related to the task bar, z-order, and positioning of the modal window.

How come some controls don't have a windows handle?

I want to get the window handle of some controls to do some stuff with it (requiring a handle). The controls are in a different application.
Strangely enough; I found out that many controls don't have a windows handle, like the buttons in the toolbar (?) in Windows Explorer. Just try to get a handle to the Folder/Search/(etc) buttons. It just gives me 0.
So.. first question: how come that some controls have no windows handle? Aren't all controls windows, in their hearts? (Just talking about standard controls, like I would expect them in Windows Explorer, nothing customdrawn on a pane or the like.)
Which brings me to my second question: how to work with them (like using EnableWindow) if you cannot get their handle?
Many thanks for any inputs!
EDIT (ADDITIONAL INFORMATION):
Windows Explorer is just an example. I have the problem frequently - and in a different application (the one I am really interested in, a proprietary one). I have "physical" controls (since I can get an AutomationElement of those controls), but they have no windows handle. Also, I am trying to send a message (SendMessage) to get the button state, trying to find out whether it is pushed or not (it is a standard button that seems to exhibit that behaviour only through that message - at least as far as I have seen. Also, the pushed state can last a lot longer on that button than you would expect on a standard button, though the Windows Explorer buttons show a similar behaviour, acting like button-style checkboxes, though they are (push)buttons). SendMessage requires a window handle.
Does a ToolBar in some way change the behaviour of its child elements? Taking away their window handle or something similar? (Using parent handle/control id for identification??) But then how to use functions on those controls that require a windows handle?
If they don't have a handle, they're not real controls, they're just drawn to look like controls.
But of course, the toolbar buttons in Windows Explorer do have window handles, they're part of a toolbar. Use the toolbar manipulation functions to interact with them, not EnableWindow.
Or, better yet, use the documented APIs for things like search. Reverse-engineering Windows Explorer has never ended well for anyone, least of all the poor Windows Shell team, saddled with years of backwards-compatibility hacks for certain developers who thought that APIs are for everyone else. Whatever you do manage to get to work is very likely to break on the next version of Windows.
The controls you are talking about are using the ToolbarWindow32 class. If you want to interact with them then you'll need to use the toolbar control APIs/message. For example for enabling buttons you'd want to use TB_ENABLEBUTTON.
You can implement the controls yourself using GDI, OpenGL or DirectX. Try Window Detective on Mozilla Firefox and you will see that there is only one window. Controls in dialog boxes are not windows known to Windows.

Win32: CreateDialog instead of multiple calls to CreateWindow - any downsides?

I'm currently working on a Win32 program which requires a main window containing many child window controls - buttons, listviews and so on. I believe the standard way to build such a window is to first call CreateWindow for the main window, then again for each of the controls.
As an easier option, I'm considering designing the main window using the resource editor's dialog box designer, then using CreateDialog to build the main window in one go.
By using a CLASS statement in the dialog box template, I should be able to get the main window to use a custom window class (and hence a custom window procedure) and thus avoid the window having any dialog-like behaviour. An example of this technique can be found in Charles Petzold's "Programming Windows": the HEXCALC program in chapter 11.
Are there any downsides to creating my main window in this way? If so, what are they? If not, why is this approach rarely used?
You don't get control of your main window message loop - the dialog manager handles it for you. On the other hand, the dialog manager handles keyboard accelerators, tab ordering and a number of other effects.
You'd be surprised what you can do with a standard dialog box - the windows volume control is implemented with about four different dialog boxes - it has a frame dialog box which in turn host hosts a tray window which in turn holds volume control dialog boxes, one for each app volume.
The only downside of CreateDialog I know of (as compared to repeated CreateWindow, not talking about some heavyweight framework, just Win32 vs Win32) is that dialog resources position child windows using dialog units. So the layout is dependent not only on DPI, but also on the user's theme settings (choice and size of font).
If any of your controls need to have fixed sizes in terms of pixels, you won't be happy with the dialog-provided positioning and will need to go through and move all the child windows after the fact.
So yes, you can use CreateDialog as a shortcut for creating a bunch of windows with specified classes and styles. But no, you can't do your layout in the dialog editor.
OTOH, you could store the DLU <-> pixel conversion used on your design machine, and then learn enough about parsing the DIALOG resource internal format to pull out the positioning information, then convert to pixels and correct the positioning in more automated fashion.
You will be able to have the full control over your window, even if it was created with CreateDialog.
Normally, when you create your own window (of your class), the window procedure used is the one that you registered with the class. OTOH windows created via CreateDialog will have the dialog standard window procedure (DefDlgProc), which will mostly invoke your supplied "dialog handler".
If you want to have full control of all the messages you may replace the window proc of the newly created window right after its creation. Just call SetWindowLongPtr with GWLP_WNDPROC parameter. Still, you may do the auto processing of some dialog-specific things by calling IsDialogMessage within your procedure.
There is no downside whatsoever.
Why is it rarely used? Because:
People normally use DialogBox instead, since that is easier for simpler cases.
For more complex cases, people use things like MFC or ATL (or some external library like GTk or Qt), and don't bother with native Win32 graphics.
There are no downsides using the Windows SDK, internally libraries like MFC use the Windows SDK .
People tend to use libraries like MFC over Windows SDK, as libaries have the readymade stuff. However Windows SDK calls are faster than library calls, so in some situations developers call Windows SDK directly .
CButton btnOk ;
btnOK.Create(_T("Press Me"), WS_CHILD|WS_VISIBLE|BS_PUSHBUTTON,CRect(100,100,300,300), pParentWnd, 1);
is similar to the following code ,
HWND hWnd = CreateWindow("BUTTON","Press Me",WS_CHILD|WS_POPUP|BS_DEFPUSHBUTTON,100,100,300,300,NULL,NULL,GetModuleHandle(NULL),NULL);
ShowWindow(hWnd,SW_SHOW);

MFC application and a non-MFC modal dialog

I'm writing a Win32 plug-in DLL for a third-party MFC app. The DLL needs to display a modal dialog. When I do this using DialogBox() or other plain Win32 API (e.g. I tried to write my own modal loop), the main application's window doesn't redraw all elements: it redraws standard elements, but not the client area. Modeless dialogs display just fine.
I suspect this happens because MFC doesn't really have modal dialogs in Win32 sense. It can only have one message loop and a separate loop in DialogBox() disrupts its delicate machinery. Here's a CodeProject article that explains this. But this CodeProject article is 9 years old, so maybe things have changed since then. Could somebody shed some light on this? The app uses MFC 8 (i.e. mfc80.dll).
Update. Here's a link to the original question; it may contain some additional information.
Update 2. Thanks everyone; I really appreciate all the advice, it certainly helps me to get the big picture of how things fit together. The first path I'm going to explore is to use native MFC 'modal' dialogs. (Since I do all this from Python, I'll use Python bindings for MFC, pywin32). This will take some time; when it's ready, I'll update the post with results.
Every thread can have a message loop. Put your modal dialog into a separate thread and emulate the standard behavior of Windows by disabling the parent window.
Edit: after some discussion (see below) it appears that the parent code behaves incorrectly.
Still, I think there are possible workarounds. One could be a parent window (to the modal dialog, but child to the one that currently behaves incorrectly) that overlays the erroneous window content, but redraws it from a DC in memory to mimic correct behavior. Of course the parent window still would have to be disabled. Another solution may be to subclass the parent window, to correct the behavior. Since the plugin would run within the same process, the implementation should be straightforward.

Which are the differences between dialog|main/child/mdi windows?

I need to understand the differences between windows main/mdi/child/dialogs.... how win32 messages should be propagated... why some messages are present in one type and not other...
There is reference information available here on the MSDN website. If you want more of an introduction or tutorial, then Charles Petzold's book Programming Windows is excellent.
Main window
The application's top window. It is flagged as the process main window and this information can be readily accessed by calling processes with the appropriate permission.
MDI (Multiple document interface) window
This is, typically, in an application main window and it contains a set of MDI Children. This is mostly a window class integrated with Win32 API. I believe it's not treated differently by the operating system as any other window class. Those are becoming extinct in favor of multiple SDI windows (Word 2007).
Child
This is a child window of any other window. Its position, visibility and mostly everything is dependent on the parent window. The children send notifications to their parents. A notification is a specific kind of window message.
Dialog
Dialogs provides easy child creation and input handling based on what 95% of dialogs need. Dialog functions in the API let you create a window and its children using compiled templates in the PE file (.exe). The message handling is also slightly different since you are working mostly with notifications from children.
The main difference with dialogs is when you are using a modal one. The creation call will block until the user closes the dialog. This can make UI updating a little tricky in some situations.
Im not a windows developer, but heres what I understand:
main window - toplevel container which you can active/see in the taskbar.
dialog - little box locking (if modal) your window, not seeable in the taskbar. Mostly used to display message to the user.
mdi (Multiple Document Interface) - Not directly a window but more a simple container for storing child windows. Each child window can be maximized/minimized/closed inside this container, but you wont find any of these in the taskbar.
http://en.wikipedia.org/wiki/Multiple_document_interface

Resources