I've encountered an interesting problem while developing for our legacy XWindows application.
For reasons that don't bear explaining, I am sending ClientMessage from a comand-line utility to a GUI app.Most of the messages end up having the same contents, as the message's purpose is to trigger a synchronous communication process over some side pipes. I've noticed that some of the time I would send two messages, but only one gets delivered. I've traced this to the fact that both messages had the same contents and were sent in the same second (IOW, the log timestamp on the sending was the same number). As soon as I added some dummy contents to the messages to make them all different, the problem went away.
This happened over two different X servers: vncserver and Exceed. Am I hitting some XWindows feature that I am not aware of - some kind of message throttling/compression? Has anyone encountered this kind of thing?
The X server should never compress client messages that I'm aware of, but perhaps some X toolkits (Motif, Xaw, etc.) do compress them. That's the first thing I would look for - perhaps the GUI app receiving the message is compressing somewhere inside the toolkit, before the application code sees it.
Then again, both vncserver and exceed probably focus more on remote usage than other X servers, and they could contain some ill-advised compression hacks, conceivably. I have read a lot of X specs and written a lot of X code and never heard of this behavior though.
A random unlikely thought, be sure you have an XFlush() or XSync() at the end of your command line app before it exits, to be sure you write those messages out to the socket before closing down. But I don't know why message content would matter if this is the problem.
Related
We have a legacy PowerBuilder 12.1 Classic application with an Oracle 11g back end, and are experiencing performance issues in production that we cannot reproduce in our test environments.
The window in question has shared grid/freeform DataWindows and buttons to open other response windows, which when closed cause the grid to re-retrieve.
The grid has a very expensive query behind it, several columns receive their values from function calls with some very intense SQL within, however it still runs within a couple seconds, even in production.
The only consistency in when the errors occur is that it seems to be more likely if they attempt to navigate to the other windows quickly. The buttons that open said windows are assuming that a certain instance variable is set with the appropriate value from the row in focus in the grid. However, in this scenario, the instance variable has not yet been set, even though it looks like the row focus change has occurred. This is causing null reference exceptions that shouldn't be possible.
The end users' network connectivity is often sluggish, and their hardware isn't any less capable than ours. I want to blame the network, but I attempted to reproduce this myself in development by intentionally slowing down the SQL so that I could attempt to click a button, however everything happened as I expected: clicking the button didn't happen until after retrieve and all the other events finished.
My gut tells me that for some reason things aren't running synchronously when they should, and the only factor I can imagine is the speed of the SQL, whether from the query being slow, or the network being slow, but when I tried reproducing that effect things still happened in the proper sequence. The only suspect code is that the datawindow ancestor posts a user event called ue_post_rfc from rowfocuschanged, and this event does a Yield(). ue_post_rfc is where code goes instead of rowfocuschanged.
Is there any way Yield() would cause these problems, without manifesting itself in test environments, even when SQL is artificially slowed?
While your message may not give enough information to give you a recipe to solve your problem, it does give me a hint towards a common point of hard-to-diagnose failures that I see often in PowerBuilder systems.
The sequence of development events goes something like this
Developer develops code where there is a dependence on one event firing before another event, often a dependence through instance or global variables
This event sequence has been something the developer has observed, but isn't documented as a guaranteed sequence (like the AcceptText() sequence or the Update() sequence are documented)
I find this a lot with posted events, and I'm not talking about event and post-event where post-event is posted from event, but more like between post-ItemChanged and post-GetFocus
Something changes the sequence of events, breaking the code. Things that I've seen change non-guaranteed sequences of events include:
PowerBuilder version change
Operating system change
Hardware change
The application running with other applications taxing the system resources
Whoever is now in charge of solving this, has no clue what is going on or how to deal with it, so they start peppering the code with Yield() statements (I've literally seen comments beside a Yield() that said "I don't know why this works, but it solves problem X")
Note that Yield() allows any and all events in the message queue to be processed, while this developer really wants only one particular event to get through
Also note that the commonly-seen-in-my-career DO ... LOOP UNTIL (NOT Yield()) could loop infinitely on a heavily loaded system
Something happens to change the event sequence again
Now when the Yield() occurs, there is a different sequence of messages in the queue to be processed, and not the message the developer had wanted to be processed
Things start failing again
My advice to get rid of this problem (if this is your problem) is to either:
Get rid of the cross-event dependence
Get rid of event sequence assumptions
Manage the event sequence yourself
Good luck,
Terry
P.S. Here's a couple of quotes from your question that make me think of Yield() (not that I don't love the opportunity to jump all over Yield() grin)
The only consistency in when the errors occur is that it seems to be
more likely if they attempt to navigate to the other windows quickly.
Seen this when the user tries to initiate (let's say for example) two actions very quickly. If the script from the first action contains a Yield(), the script from the second action will both start and finish before the first action finishes. This can be true of any combination of user actions (e.g. button clicks, menu clicks, tabs, window closings... you coded with the possibility that the window isn't there anymore after the Yield() was done, right? If not, join the 99% of those that code Yield(), don't, and live dangerously) and system events (e.g. GetFocus, Deactivate, Timer)
My gut tells me that for some reason things aren't running
synchronously when they should
You're right. PowerBuilder (unless you force it) runs synchronously. However, if one event is starting before another finishes (see above), then you're going to get behaviours that look like asynchronous behaviours.
There's nothing definitive in what you've said, but you did ask about Yield(). The really kicker to nail this down is if you could reproduce this with a PBDEBUG trace; you'd see which event(s) is(are) surprising you. However, the amount that PBDEBUG slows things down affects event sequences and queuing, which may or may not be helpful.
On Mac OS X, I have a process which produces JSON objects, and another intermittent process which should consume them. The producer and consumer processes are independent of each other. Objects will be produced no more often than every 5 seconds, and will typically be several hundred bytes, but may range up into megabytes sometimes. The objects should be communicated first-in-first-out. The consumer may or may not be running when the producer is producing, and may or may not read objects immediately.
My boneheaded solution is
Create a directory.
Producer writes each JSON object to a text file, names it with a serial number.
When Consumer launches, it reads and then deletes files in serial-number order, and while it is running, uses FSEvents to watch this directory for new files arriving.
Is there any easier or better way to do this?
The modern way to do this, as of Lion, is to use XPC. Unfortunately, there's no good documentation of it; there's a broad overview in the Daemons and Services guide and a primitive HeaderDoc-generated reference, but the best way to get introduced to it is to watch the session about it from last year's WWDC sessions.
With XPC, you won't have to worry about keeping serial numbers serial, having to contend for a spinning disk, or whether there's enough disk space. Indeed, you don't even have to generate and parse JSON data at all, since XPC's communication mechanism is built around JSON-esque/plist-esque container and value objects.
Assuming you want the consumer to see the old files, this is the way it's been done since the beginning of time - loathsome though it may be.
There's lots of highish tech things that look cleaner - but honestly, they just tend to add complexity and/or deployment infrastructure that add hassle. What you suggest works, and it works well, and it's easy to write and maintain. You might need some kind of sentinel files to track what you are doing for crash recovery, but that's probably about it.
Hell, most people would just poll with a sleep 5. At least you are all all up in the fsevent.
Now if it was accepable to lose the events generated when the listener wasn't around; and perf was paramount - it could get more interesting. :)
This is somewhat of a general question regarding Windows programming:
Are Window messages "reliable"?
For example (these are just examples):
Can you be certain that a WM_MOUSEMOVE will happen before a cursor enters your screen?
Can you be certain that you will get a WM_DEVICECHANGE message if a device is inserted?
Can you be certain that you will receive a WM_KILLFOCUS message if your window loses focus?
Or, in other words: Can you be certain that you'll get the appropriate message at the appropriate times, or do you always have to code defensively in case that, somehow, you might miss a message for no apparently documented reason?
Example:
It is guaranteed (AFAIK) that a file system filter driver will not "miss" a file operation or change notification.
By contrast, it is not guaranteed that ReadDirectoryChangesW will not miss a notification. In fact, it can miss quite a few if its buffer overflows.
Note:
I am not talking about a situation against an adversary (e.g. someone hijacking your window procedure or installing a hook/filter); that would pretty much invalidate any guarantee. I'm only asking about obscure situations that could really happen even if no one meant anything bad intentionally, like if some random buffer overflows, if someone uses SendInput, etc., assuming you have control of your own code.
No you cannot be certain that a given message will be delivered in a specific order. Here are a couple of reasons why not
Messages can be sent progamatically and this can be used to simulate "impossible" scenarios like a WM_KEYUP followed by a WM_KEYDOWN.
Another routine could sub-class your window and selectively intercept messages and not send them on to your WNDPROC
It's best to code defensively around any scenarios where ordering is important
MSDN advises that RegisterWindowMessage() function is only used for registering messages to be sent between the processes. If a message is needed for sending within one process it can be safely selected from the range WM_APP through 0xBFFF.
However in our codebase I often see that RegisterWindowMessage() is used for messages only sent within one process. I suppose that this was done because of perceived simplicity of using RegisterWindowMessage() since it doesn't require manually distributing the message identifiers in the WM_APP..0xBFFF range.
Do I understand correctly that if many applications are run on one machine and they all call RegisterWindowMessage() with different strings they could exhaust the range of message identifiers allowed to return by RegisterWindowMessage() and for some of them it will just return a value indicating a failure? What could be a valid reason for using RegisterWindowMessage() messages in cases where WM_APP..0xBFFF range messages would suffice?
IMHO there is no valid reason to use RegisterWindowMessage if you are only sending messages to yourself
There is no (documented) way to un-register a message, so after your app quits, that registered message will stay in the atom table until reboot/logoff (I can't remember exactly where this atom table is stored, the window station or terminal server session instance probably)
The reason you need to use RegisterWindowMessage even when messaging to yourself is that it protects you from the idiot who broadcasts messages in the WM_APP + N range.
Yes, this does happen.
Abusing RegisterWindowMessage can potentially make a windows box unusuable. This is especially true if the window message names are dynamically generated and a bug causes out of control windows message allocation. In this case the global atom table in your windows station/ desktop will fill up and any process using User32.dll (basically, any app) will fail to start, create windows, etc.
There is a bug out there in Delphi / Borland products that registers messages that start with ControlOfsXXXXXX where XXXX is a memory address (or other dynamic modifier). Apps that are started and stopped frequently will register multiple ControlOfsXXXX atoms and eventually exhaust atom space. For more details see:
http://blogs.msdn.com/b/ntdebugging/archive/2012/01/31/identifying-global-atom-table-leaks.aspx
And
https://forums.embarcadero.com/thread.jspa?threadID=47678
A possible advantage is that Spy++ can display more informative text, therefore debugging is a bit easier. Compare
<00058> 00330CA2 S message:0x0419 [User-defined:WM_USER+25] wParam:00000000 lParam:00000000
with
<00129> 004F0DA0 S message:0xC2B0 [Registered:"AFX_WM_ONCHANGE_ACTIVE_TAB"] wParam:00000001 lParam:02B596E8
Of course, in principle there is a chance to run out of message IDs. On the other hand, in the source code of the MFC Feature Pack there are 52 calls to RegisterWindowMessage. So there are still 16300 IDs left for other applications.
We have an application that has one or more text console windows that all essentially represent serial ports (text input and output, character by character). These windows have turned into a major performance problem in the way they are currently code... we manage to spend a very significant chunk of time in them.
The current code is structured by having the window living its own little life, and the main application thread driving it across "SendMessage()" calls. This message-passing seems to be the cause of incredible overhead. Basically, having a detour through the OS feels to be the wrong thing to do.
Note that we do draw text lines as a whole where appropriate, so that easy optimization is already done.
I am not an expert in Windows coding, so I need to ask the community if there is some other architecture to drive the display of text in a window than sending messages like this? It seems pretty heavyweight.
Note that this is in C++ or plain C, as the main application is a portable C/C++/some other languages program that also runs on Linux and Solaris.
We did some more investigations, seems that half of the overhead is preparing and sending each message using SendMessage, and the other half is the actual screen drawing. The SendMessage is done between functions in the same file...
So I guess all the advice given below is correct:
Look for how much things are redrawn
Draw things directly
Chunk drawing operations in time, to not send every character to the screen, aiming for 10 to 20 Hz update rate of the serial console.
Can you accept ALL answers?
I agree with Will Dean that the drawing in a console window or a text box is a performance bottleneck by itself. You first need to be sure that this isn't your problem. You say that you draw each line as a whole, but even this could be a problem, if the data throughput is too high.
I recommend that you don't use the SendMessage to pass data from the main application to the text window. Instead, use some other means of communication. Are these in the same process? If not, you could use shared memory. Even a file in the disk could do in some circumstances. Have the main application write to this file and the text console read from it. You could send a SendMessage notification to the text console to inform it to update the view. But do not send the message whenever a new line arrives. Define a minimum interval between two subsequent updates.
You should try profiling properly, but in lieu of that I would stop worrying about the SendMessage, which almost certainly not your problem, and think about the redrawing of the window itself.
You describe these are 'text console windows', but then say you have multiple of them - are they actually Windows Consoles? Or are they something your application is drawing?
If the latter, then I would be looking at measuring my paint code, and whether I'm invalidating too much of a window on each update.
Are the output windows part of the same application? It almost sounds like they aren't...
If they are, you should look into the Observer design pattern to get away from SendMessage(). I've used it for the same type of use case, and it worked beautifully for me.
If you can't make a change like that, perhaps you could buffer your output for something like 100ms so that you don't have so many out-going messages per second, but it should also update at a comfortable rate.
Are the output windows part of the
same application? It almost sounds
like they aren't...
Yes they are, all in the same process.
I did not write this code... but it seems like SendMessage is a bit heavy for this all in one application case.
You describe these are 'text console
windows', but then say you have
multiple of them - are they actually
Windows Consoles? Or are they
something your application is drawing?
Our app is drawing them, they are not regular windows consoles.
Note that we also need to get data back when a user types into the console, as we quite often have interactive serial sessions. Think of it as very similar to what you would see in a serial terminal program -- but using an external application is obviously even more expensive than what we have now.
If you can't make a change like that,
perhaps you could buffer your output
for something like 100ms so that you
don't have so many out-going messages
per second, but it should also update
at a comfortable rate.
Good point. Right now, every single character output causes a message to be sent.
And when we scroll the window up when a newline comes, then we redraw it line-by-line.
Note that we also have a scrollback buffer of arbitrary size, but scrolling back is an interactive case with much lower performance requirements.