Debugging: step into DispatchMessage() - windows

I am trying to debug a very old C program in visual studio. I was stepping through some code line by line and came to a line DispatchMessage(&msg); I wanted to see where the code went next, so I clicked the "Step into" button, but I was not taken to any new code that processed the message, instead the little yellow arrow just stepped along to the next line down the same page...
Is this expected behaviour? How can I see what DispatchMessage actually did?
Could this be an indication that the window handler is not set up correctly?
extra info: The task I was tring to debug was the processing of a mouse click on a particular window. I had displayed the msg structure, and msg.hwnd was the window I had clicked on (I used Winspector to confirm). The msg.message was 513 (=WM_LBUTTONDOWN).

This is completely expected behaviour...
DispatchMessage() is implemented in User32.dll and as such, the source code is not available. For the most part, you have to treat Win32 API calls as blackboxes. At times, this can really make debugging Win32 code a challenge.
Remember, there are some tricks in your arsenal for debugging this kind of thing:
Go on treating DispatchMessage() as a black box. Place a breakpoint on the WndProc you expect to receive the WM_LBUTTONDOWN and continue debugging there.
Have Winspector log the messages coming into all of the windows involved. This can help make sure the message is received and in the order you are expecting.
If worst comes to worst, you may have to step into DispatchMessage() or some other API call, looking at its disassembly. Most of the time, nothing good comes from that (but I always learn something when I do).
Chances are that option #3 won't be necessary and your time would be better spent debugging around DispatchMessage() rather than within it.

Related

Why is my debugger "breaking" on a line where I have not set a breakpoint?

I apologize in advance, this will be a very basic & general beginner's question:
Why is my debugger "breaking" on a line where I have not set a breakpoint?
Some more detail:
I'm working on a large buggy C++ project in XCode 12.4.
I've set a single breakpoint on function A. The debugger refuses to break on function A. Instead, it breaks consistently on function B. Function A and B exist on the same cpp file. I believe function B is supposed to be called AFTER function A, but they definitely do not call each other and are not called simultaneously.
I've definitely experienced the debugger stopping on various errors, but with a RED highlight, not a GREEN one - as I understand it the GREEN highlight only happens on a breakpoint I set myself.
Is this expected behavior? Am I missing something obvious?
(First post, please tell me if I'm doing anything wrong!)
EDIT: I solved & answered my own question below!
The most common cause of the debugger pausing at a place where you didn't set a breakpoint is the throwing of an exception. Some exceptions crash the program, which is where you get the red highlight in the debugger. Some exceptions don't crash the program. In those cases the debugger highlights the exception breakpoint in green.
If the debugger pauses your program with a green highlight, click the Continue Execution button (it's the second button from the left in the group of debugging buttons right below the source code editor) or choose Debug > Continue in Xcode to continue running the program.
These are all great responses and I learned a lot from them all - but it turns out the answer was much dumber.
For whatever reason, when I compiled, XCode did not delete the previously compiled binary. As for my breakpoint issue, I'm still not 100% sure what was happening, but it is solved now.
Here's how I noticed: Function A did not exist when I compiled it last time. I had created a String in function A. This string assignment returned a bad access error - so I assume it was assigned outside the program I was debugging!
Thank you to responders!

Implementing a Custom Cocoa Event Tracking Loop

I'm working on a custom cross platform UI library that needs a synchronous "ShowPopup" method that shows a popup, runs an event loop until it's finished and automatically cancels when clicking outside the popup or pressing escape. Keyboard, mouse and scroll wheel events need to be dispatched to the popup but other events (paint, draw, timers etc...) need to be dispatched to their regular targets while the loop runs.
Edit: for clarification, by popup, I mean this kind of menu style popup window, not an alert/dialog etc...
On Windows I've implemented this fairly simply by calling GetMessage/DispatchMessage and filtering and dispatching messages as appropriate. Works fine.
I've much less experience with Cocoa/OS X however and finding the whole event loop/dispatch paradigm a bit confusing. I've seen the following article which explains how to implement a mouse tracking loop which is very similar to what I need:
http://stpeterandpaul.ca/tiger/documentation/Cocoa/Conceptual/EventOverview/HandlingMouseEvents/chapter_5_section_4.html
but... there's some things about this that concern me.
The linked article states: "the application’s main thread is unable to process any other requests during an event-tracking loop and timers might not fire". Might not? Why not, when not, how to make sure they do?
The docs for nextEventMatchingMask:untilDate:inMode:dequeue: states "events that do not match one of the specified event types are left in the queue.". That seems a little odd. Does this mean that if an event loop only asks for mouse events then any pressed keys will be processed once the loop finishes? That'd be weird.
Is it possible to peek at a message in the event queue without removing it. eg: the Windows version of my library uses this to close the popup when it's clicked outside, but leaves the click event in the queue so that clicking outside the popup on a another button doesn't require a second click.
I've read and re-read about run loop modes but still don't really get it. A good explanation of what these are for would be great.
Are there any other good examples of implementing an event loop for a popup. Even better would be pseudo-code for what the built in NSApplication run loop does.
Another way of putting all this... what's the Cocoa equivalent of Windows' PeekMessage(..., PM_REMOVE), PeekMessage(..., PM_NOREMOVE) and DispatchMessage().
Any help greatly appreciated.
What exactly is a "popup" as you're using the term? That term means different things in different GUI APIs. Is it just a modal dialog window?
Update for edits to question:
It seems you just want to implement a custom menu. Apple provides a sample project, CustomMenus, which illustrates that technique. It's a companion to one of the WWDC 2010 session videos, Session 145, "Key Event Handling in Cocoa Applications".
Depending on exactly what you need to achieve, you might want to use an NSAlert. Alternatively, you can use a custom window and just run it modally using the -runModalForWindow: method of NSApplication.
To meet your requirement of ending the modal session when the user clicks outside of the window, you could use a local event monitor. There's even an example of just such functionality in the (modern, current) Cocoa Event Handling Guide: Monitoring Events.
All of that said, here are (hopefully no longer relevant) answers to your specific questions:
The linked article states: "the application’s main thread is unable to process any other requests during an event-tracking loop and
timers might not fire". Might not? Why not, when not, how to make
sure they do?
Because timers are scheduled in a particular run loop mode or set of modes. See the answer to question 4, below. You would typically use the event-tracking mode when running an event-tracking loop, so timers which are not scheduled in that mode will not run.
You could use the default mode for your event-tracking loop, but it really isn't a good idea. It might cause unexpected re-entrancy.
Assuming your pop-up is similar to a modal window, you should probably use NSModalPanelRunLoopMode.
The docs for nextEventMatchingMask:untilDate:inMode:dequeue:
states "events that do not match one of the specified event types are
left in the queue.". That seems a little odd. Does this mean that if
an event loop only asks for mouse events then any pressed keys will be
processed once the loop finishes? That'd be weird.
Yes, that's what it means. It's up to you to prevent that weird outcome. If you were to read a version of the Cocoa Event Handling Guide from this decade, you'd find there's a section on how to deal with this. ;-P
Is it possible to peek at a message in the event queue without removing it. eg: the Windows version of my library uses this to close
the popup when it's clicked outside, but leaves the click event in the
queue so that clicking outside the popup on a another button doesn't
require a second click.
Yes. Did you notice the "dequeue:" parameter of nextEventMatchingMask:untilDate:inMode:dequeue:? If you pass NO for that, then the event is left in the queue.
I've read and re-read about run loop modes but still don't really get it. A good explanation of what these are for would be great.
It's hard to know what to tell you without knowing what you're confused about and how the Apple guide failed you.
Are you familiar with handling multiple asynchronous communication channels using a loop around select(), poll(), epoll(), or kevent()? It's kind of like that, but a bit more automated. Not only do you build a data structure which lists the input sources you want to monitor and what specific events on those input sources you're interested in, but each input source also has a callback associated with it. Running the run loop is like calling one of the above functions to wait for input but also, when input arrives, calling the callback associated with the source to handle that input. You can run a single turn of that loop, run it until a specific time, or even run it indefinitely.
With run loops, the input sources can be organized into sets. The sets are called "modes" and identified by name (i.e. a string). When you run a run loop, you specify which set of input sources it should monitor by specifying which mode it should run in. The other input sources are still known to the run loop, but just ignored temporarily.
The -nextEventMatchingMask:untilDate:inMode:dequeue: method is, more or less, running the thread's run loop internally. In addition to whatever input sources were already present in the run loop, it temporarily adds an input source to monitor events from the windowing system, including mouse and key events.
Are there any other good examples of implementing an event loop for a popup. Even better would be pseudo-code for what the built in
NSApplication run loop does.
There's old Apple sample code, which is actually their implementation of GLUT. It provides a subclass of NSApplication and overrides the -run method. When you strip away some stuff that's only relevant for application start-up or GLUT, it's pretty simple. It's just a loop around -nextEventMatchingMask:... and -sendEvent:.

How to break code on a click event?

I have this application that I need to disassemble. I don't have a clue on how to stop the running code on the desired location, so I decided my best guess would be breaking upon a button click. But how do I capture button clicks? I know it has probably something to do with the Windows functions such as CallNextHookEx. I'm using IDA PRO to disassembly.
IDA PRO is used mostly as disassembler, for static analysis purposes. I'd suggest you to use Ollydbg (or some other debugger, if you want to) because it will suit better to debugging purposes.
I don't know if you can set a breakpoint on an API like that.
But you can do this:
Load the application in olly, or attach to it.
Generate the event by clicking on anything.
Stop the application from ollydbg(F12)
Use C(k)all stack(ALT+K)
You will see a few calls to functions, one of them is doing what you need. But you may need to go to upper calls to see the whole loop. So you will just try which one it is. There will be a loop in one of them.That loop will have conditional jumps and generate events, load forms, fill the app etc. And when you place a breakpoint on the right jump there, it will stop at each mouse click.
When I'm debugging apps, most of the times I find myself on a breakpoint like this, and I see from the beginning how the application is filling an empty form(it takes so long.)

Layered window still receiving WM_PAINT message after UpdateLayeredWindow call

I've got a few layered windows in my app that use UpdateLayeredWindow() to handle their visual representation. According to the MSDN article on layered windows, "when using UpdateLayeredWindow() the application doesn't need to respond to WM_PAINT or other painting messages." They shared some of the same message handlers as non-layered windows, so I figured I would just return early from WM_PAINT handling if the target is a layered window.
Of course, this caused one major issue: if one of the layered windows did get a WM_PAINT message, the input queue would end up flooded with an unending stream of WM_PAINT messages. This end-result makes sense, since the window would never be validated and so it will keep thinking it needs to paint (I shouldn't be returning from the handler without validating or BeginPaint()ing, etc.), but what doesn't make sense is why it received the message in the first place, since it has no effect on a window that was using UpdateLayeredWindow().
It wouldn't even happen reliably -- just every now and then, and not every time the window's pixels needed redrawing. Sanity was restored by falling back to DefWindowProc() when a layered window got a WM_PAINT message, but I feel like something is going on that I don't understand. And considering how seldom this problem manifested itself, I'm worried this might just be hiding an even subtler problem. Is it expected behavior for a window using UpdateLayeredWindow() to still get the occasional WM_PAINT message? Does it matter, as long as I handle it correctly?
Additional info, if needed: the window is calling UpdateLayeredWindow() immediately after being created, and then it's left on its own (it doesn't call it again, since it doesn't change). Using C++ and win32 API, no MFC.
I had run into similar issues before, although my memory may be a bit rusty by now.
First off, keep the DefWindowProc. When the docs say you don't have to respond, I would take that to mean to ignore the message entirely, rather than prevent default handling.
I personally experienced this from two different causes. One was a window which was actually sending WM_PAINT messages (evil! beware!). The other (IIRC) resulted from certain RedrawWindow calls. In both cases, I chalked the problem up to poorly written code, outside of my control, and never had any situations arise from simply passing it down to DefWindowProc.
Hopefully you will have the same experience!
Good luck. I found layered windows to be poorly documented and full of interesting caveats and gotchas, but very pleasing once you get all the kinks worked out.

Bring form on top of others when clicked on taskbar button in Delphi

Base question: TStatusBar flickers when calling Update procedure. Ways to painlessly fix this
The executed code is in the questions first posts first part ( you can see light grey separating line ) ...
But - problem is that while this code is executed, form does not automatically activate and focus on the top of all other applications.
I have read these articles:
http://www.installationexcellence.com/articles/VistaWithDelphi/Original/Index.html
http://delphi.about.com/od/formsdialogs/l/aa073101b.htm
but according to them it should be working no matter what. I tried all the TApplicationEvents and TForm events with Show; Visible: Repaint; Refresh; BringToFront; ... nothing works.
So - I think I have two options - multithreading or trapping WM_SYSCOMMAND message and in the SC_ACTIVE event simply repaint form. Could this scenario become successful?
None of your linked articles deal with the problem you are having. What you see is the behaviour of a program that does not process Windows messages, so consequently it will not redraw parts that become invalid, and it will not react to keyboard or mouse input (for example moving or resizing with the mouse, or app activation using the taskbar button).
In your code you call StatusBar1.Update, so at least the status bar text is redrawn, but apart from coming to the foreground your application is probably also ignoring move or resize requests.
You need to process Windows messages in a timely manner, so any execution path that takes more than say 200 or 300 milliseconds needs to make sure that messages are handled, otherwise the application will appear unresponsive or hung.
You have basically three options:
Keep the long running code, and insert calls to Application.ProcessMessages - this will allow Windows messages to be processed. Make sure that you keep the code from being entered again, for instance by disabling all the controls that are used to start the operation.
Rework your code in a way that it appears as a sequence of steps, each taking no more than a few 10 milliseconds. Put calls to the code in a timer event handler, or call it from the Application.OnIdle handler.
Call your code in a worker thread, and post messages to the main GUI thread to update your UI.
All these options have their own pros and cons, and for multithreading especially there is a lot of questions and answers already here on SO. It is the most difficult but best option overall when you are working on anything more than a toy program.

Resources