My goal is to screen-scrape a portion of a program which constantly updates with new text. I have tried OCR with Tesseract but I believe it would be much more efficient to somehow intercept the text if possible. I have attempted using the GetWindowText() function, but it only returns the window title. Using Window Detective I have determined that whenever the window updates in the way I wish to capture, a WM_PAINT message is reliably sent to the window.
I have looked a bit into Windows API Hooks, but it seems that most of these techniques involving DLL injection are intended at sending new messages, not accessing the content of already sent messages.
How should I approach this problem?
When you say 'screen-scrape', is that what you really mean? Reading your post, it sounds like you actually want to get at the text in the child window or control in question - as text, and not just as a bitmap. To do that, you will need to:
Determine which child window or control actually contains the text you want to get at. It sounds like you may have already done that but if not, the tool of choice is generally Spy++. (Please note: the version of Spy that you use must match the 'bitness' of your application.)
Then, firstly, try to figure out whether the text in that window can be retrieved somehow. If it's a standard Windows control (specifically EDIT or RICHEDIT) then there are documented ways to do that, see MSDN.
If that doesn't pan out, you might have some success hooking calls to ExtTextOut(), although that's not a pleasant proposition and I think you might struggle to achieve it. That said, I believe the accepted way (in some sense of the word 'accepted') is here.
With reference to point 3, even if you achieve it, how would you know whether any particular call to ExtTextOut() was drawing to the window you're interested in? Answer, most likely, HWND WindowFromDC().
I hope that helps a little. Please don't come back at me with a bunch of detailed questions about how this might apply to your particular use-case. I'm not really interested in that, these are just intended as a few signposts.
I want to put help buttons on all my windows, like this:
But when I put the help button in, the minimize/maximize buttons disappear. Does Windows forbid having the min/max buttons together with the help buttons? That would be disappointing because that would mean I could put the help button only on dialogs and not on frames.
If Windows does forbid this, it would be nice to see an official Microsoft document which talks about this policy.
It is not possible through setting windows styles. If you really wanted to you could set some hooks that would probably let you do what you want, but I would not recommend doing that. You can mimic the functionality of the help button by sending the WM_HELP message.
According to MSDN, the styles WS_MAXIMIZEBOX and WS_MINIMIZEBOX can not be combined with WS_EX_CONTEXTHELP.
Although it is true what daalbert says, with some effort it is indeed possible to draw just about anything properly on the window frame. Of course this is in no way "official" and the limitation that daalbert mentions still stands.
You can listen for WM_NCPAINT and draw the button yourself with the help of DrawFrameControl with DFC_BUTTON (which makes sure it will look like the real thing). Use WM_NCHITTEST and friends (WM_NC*BUTTON*) to find out whether the button you draw gets clicked.
So yes, it's technically possible to achieve what you want but usually not worth the extra effort.
Just wanted to have this on record for completeness.
I have a 3rd party ActiveX control I want to render within other presentation technologies (Direct3D and WPF). To do this, I need the ActiveX to render to a system memory bitmap instead of the screen. I know there is a way to do this, but not sure where to start. I'm not afraid of doing any native method hooking, but I'm not sure where to start. I was thinking BeginPaint(...) might be the hot ticket...
Has anyone done this or seen examples/samples floating around?
BTW, I do not want to do a WM_PRINT type solution. I'd rather this code be reactive, than proactive and forcing the hwnd to repaint.
EDIT:
Both answers were right in my case, so I gave each a +1. I wish to have a lower level solution to be more flexible, but this is good enough at the moment.
You're best bet is to see if the ActiveX object supports IViewObject and call the Draw() method to get it to render into your DC.
If the object doesn't support standard OLE interfaces, you're going to have a very difficult time getting this to work. You'll have to resort to doing evil things like re-writing the DLL's import table to redirect BitBlt(), etc. I've done this sort of thing. I don't recommend it.
Your next problem will be how to correctly map input events.
Does your site support the IOleInPlaceSiteWindowless interface?
Windowless support in ActiveX controls is unfortunately optional. If the 3rd party interface does support it then you can render the control using IViewObject::Draw onto whatever surface you want.
I'm in the process of designing and testing various ideas for an application whose main functionality will be to notify users of occurring events and offer them with a choice of actions for each.
The standard choice would be to create a queue of events showing a popup in the taskbar with the events and actions, but I want this tool to be the less intrusive and disrupting as possible.
What I'm after is a good book or papers on studies of how to maximize user productivity in these intrinsically disruptive scenarios (in other words, how to achieve the perfect degree of annoying-ness, not too much, not too little).
The user is supposedly interested in these events, they subscribe to them and can choose the actions to perform on each.
I prefer books and papers, but the usual StackOverflow wisdom is appreciated as well.
I'm after things like:
Don't use popups, use instead X
Show popups at most 3 seconds
Show them in the left corner
Use color X because it improves readability and disrupts less
That is, cognitive aspects of GUI design that would help users in such a scenario.
For notification, I prefer small notification windows in the bottom right corner. If they have a minimum UI allowing me to take some quick action, it would be ideal.
As model: incoming mail notification from Outlook, Skype notifications for incomming calls, etc.
I have read and recommend:
About Face 3
The Design of Everyday Things
I don't know of any formal articles but the Ubuntu UI design team have considered this issue and you may find something of value in their notification design guidelines or in this discussion by Mark Shuttleworth.
Sorry, no book recommendation here, just one piece of advise: Ultimately, you should let your users decide.
In my experience, what one user considers to be a useful feature is seen as completely broken by the second, while the third doesn't care at all.
Offer them 3-5 options how they will be alerted (including the option to turn alerts off ;-). Of course you should optimise these options as much as possible, hopefully some other answers here can help you with that.
A collection of possibilities:
Change a static tray icon. Useful if the user doesn't have to answer each message immediately, and relatively unobtrusive. Should be an overlay on the original image, making sure that the user recognizes which application it's from, and should be different enough that the user immediately understands that it's not the original icon (for example, a red / yellow exclamation mark on a blue / green background).
Change the tray icon to a dynamic variant. This will be more obtrusive, grabbing the user attention periodically, but is useful when the message is urgent. Could for example be a revolving version of a circular logo, or a pulsating square logo. Depending on the urgency, the animation could repeat endlessly (highly distracting) or once in a while. Use with care :)
Pop-over box in a corner of the screen. This should give some context info, the primary object of which is to enable the user to decide whether or not it's worth checking out immediately. Could afford clicking by using a "More" link or big "+" sign.
Focus the application window immediately. Enormously intrusive, and should not even be used in case of nuclear meltdowns (what if the user already knows the meltdown is occurring, and is looking elsewhere for a solution?). Just don't.
I think an answer depends highly on the specifics of the UI you are working within. However, for general guides
The GNOME desktop environment's human interface guidelines (HIG) is a useful resource if your environment is GNOME, and perhaps informative if it's not. There is a section on using the "desktop notification area": http://library.gnome.org/devel/hig-book/stable/desktop-notification-area.html.en. This covers not overloading the area, etc. (perhaps analogous to the win32 tray).
Apple have their own HIG document. Part of this describes using the Dock for conveying status information: http://developer.apple.com/documentation/userexperience/Conceptual/AppleHIGuidelines/XHIGMOSXEnvironment/XHIGMOSXEnvironment.html#//apple_ref/doc/uid/TP40002721-TPXREF139
Apple also have a separate HIG document for the iPhone. there are several notification techniques described within, e.g. "alerts" http://developer.apple.com/iphone/library/documentation/UserExperience/Conceptual/MobileHIG/ModalViews/ModalViews.html#//apple_ref/doc/uid/TP40006556-CH11-SW1 which are specifically described as to be used for critical information (due to their disruptiveness).
Several apple iphone apps adjust their icon to convey some status info: e.g. mail, SMS, phone all indicate the number of outstanding mails, SMS messages, or missed calls/voicemails as part of their launch icon. I can't, however, find any reference to that in a cursory scan of the apple iphone HIG.
As it currently stands, this question is not a good fit for our Q&A format. We expect answers to be supported by facts, references, or expertise, but this question will likely solicit debate, arguments, polling, or extended discussion. If you feel that this question can be improved and possibly reopened, visit the help center for guidance.
Closed 11 years ago.
What techniques do you know\use to create user-friendly GUI ?
I can name following techniques that I find especially useful:
Non-blocking notifications (floating dialogs like in Firefox3 or Vista's pop-up messages in tray area)
Absence of "Save" button
MS OneNote as an example.
IM clients can save conversation history automatically
Integrated search
Search not only through help files but rather make UI elements searchable.
Vista made a good step toward such GUI.
Scout addin Microsoft Office was a really great idea.
Context oriented UI (Ribbon bar in MS Office 2007)
Do you implement something like listed techniques in your software?
Edit:
As Ryan P mentioned, one of the best way to create usable app is to put yourself in user's place. I totally agree with it, but what I want to see in this topic is specific techniques (like those I mentioned above) rather than general recommendations.
If you do give the user a question, don't make it a yes/no question. Take the time to make a new form and put the verbs as choices like in mac.
For example:
Would you like to save?
Yes No
Should Be:
Would you like to save?
Save Don't Save
There is a more detailed explanation here.
Check out the great book Don't make me think by Steve Krug.
It's web focused but many of the conepts can apply to anything from blenders to car dashboards.
Topics covered:
User patterns
Designing for scanning
Wise use of copy
Navigation design
Home page layout
Usability testing
He also has a blog called Advanced Common Sense
And some random UI related links:
- User Interface Design for Programmers by Joel Spolsky
- 10 Usability Nightmares You Should Be Aware Of
First Principles: Wilfred James Hansen
Know the User
Minimize Memorization
Optimize Operations
Engineer for Errors
Subsequent Expansions: Dr. Theo Mandel
Place Users in Control
Use Modes Judiciously (modeless)
Allow Users to use either the Keyboard or Mouse (flexible)
Allow Users to Change Focus (interruptible)
Display Descriptive Messages and Text (helpful)
Provide Immediate and Reversible Actions, and Feedback (forgiving)
Provide meaningful Paths and Exits (navigable)
Accommodate Users with Different Skill Levels (accessible)
Make the User Interface Transparent (facilitative)
Allow Users to Customize the Interface (preferences)
Allow Users to Directly Manipulate Interface Objects (interactive)
Reduce Users' Memory Load
Relieve Short-term Memory (remember)
Rely on Recognition, not Recall (recognition)
Provide Visual Cues (inform)
Provide Defaults, Undo, and Redo (forgiving)
Provide Interface Shortcuts (frequency)
Promote an Object-action Syntax (intuitive)
Use Real-world Metaphors (transfer)
User Progressive Disclosure (context)
Promote Visual Clarity (organize)
Make the Interface Consistent
Sustain the Context of Users’ Tasks (continuity)
Maintain Consistency within and across Products (experience)
Keep Interaction Results the Same (expectations)
Provide Aesthetic Appeal and Integrity (attitude)
Encourage Exploration (predictable)
To add to your list, aku, I would put explorability as one of my highest priorities. Basically, I want the user to feel safe trying out the features. They should never back away from using something for fear that their action might be irreversible. Most commonly, this is implemented using undo/redo commands, but other options are no doubt available e.g. automatic backups.
Also, for applications that are more process-oriented (rather than data-entry applications), I would consider implementing an interface that guide the user a bit more. Microsoft's Inductive User Interface guidelines can help here, although you need to be very careful not to overdo it, as you can easily slow the user down too much.
Finally, as with anything that includes text, make the user interface as scannable as possible. For example, if you have headings under which commands/options appear, consider putting the action word at the start, rather than a question word. The point that Maudite makes is a good example of scannability too, as the "Don't Save" button text doesn't rely on the context of the preceding paragraph.
A useful technique which I never see anyone use is to add a tooltip for a disabled UI control explaining why the control is disabled. So if there's a listbox which is disabled and it's not clear why it is disabled, I want to hover over it and it tells me why it's disabled. I want to see something like "It's disabled because two textboxes on the screen were left blank or because I didn't enter enough characters in some field or because I didn't make a certain action.".
I get into sooooo many such situations and it's frustrating. Sometimes I end up posting in the software's forum asking why a control is greyed out when a tooltip could have helped me in a second! Most of these software have help files which are useless in these kinds of scenarios.
Try to pretend you know nothing about your software and try using it. However this is not practical because you already have a certain mind set towards the app. So watch fellow developers or friends use the app and look out for the pain points and ask for feedback.
One of the classic books to help you think about design is "The Design of Everyday Things" by Donald Norman. He gives great real-world examples. For example, if you design a door well, you should never have to add labels that say "push" and "pull." If you want them to pull, put a handle; if you want them to push, put a flat plate. There's no way to do it wrong, and they don't even have to think about it.
This is a good goal: make things obvious. So obvious that it never occurs to the user to do the wrong thing. If there are four knobs on a stove, each one next to an eye, it's obvious that each knob controls the eye it's next to. If the knobs are in a straight line, all on the left side, you have to label them and the user has to stop and think. Bad design. Don't make them think.
Another principle: if the user does make a mistake, it should be very easy to undo. Google's image software, Picasa, is a good example. You can crop, recolor, and touch up your photos all you like, and if you ever change your mind - even a month later - you can undo your changes. Even if you explicitly save your changes, Picasa makes a backup. This frees up the user to play and explore, because you're not going to hurt anything.
I've found UI Patterns to be a useful reference for this sort of thing. It's arranged much like the classic GoF Design Patterns book, with each pattern description containing:
The problem the pattern solves
An example of the pattern in action
Sample use cases for the pattern
The solution to implement the pattern
Rationale for the solution
If you implement a search, make it a live search like what Locate32 and Google Suggest does now. I am so used to not pressing "Enter" at the search box now.
Well, one thing that may be obvious: don't change (even slightly) the position, color, font size, etc. of buttons, menus, links, etc. between screens if they do the same type of action.
Really good feedback is extremely important. Even simple things like making it obvious what can and cannot be clicked can be overlooked or too subtle. Feedback when something might happen in the background is great. In gmail, it's great that there's a status ribbon appearing at the top that let's you know if something is sending or loading, but it's even better that it lets you know that something has sent successfully or is still loading.
The "yellow fade" technique is something else made popular amongst the RoR crowd that accomplishes something similar. You never want the user to ask the question, "What just happened?" or "What will happen when I do this?".
Another trick that has become more popular lately that I've been using a lot is editing in place. Instead of having a view of some data with a separate "edit" screen (or skipping the view and only having an edit screen), it can often be more user friendly to have a nicely laid out view of some data and just click to edit parts of it. This technique is really only appropriate when reading the data happens more often than editing, and is not appropriate for serious data-entry.
If you are doing enterprise software, a lot of users will have small monitors at low resolution. Or if they are old they will have it at a low res so they can see giant buttons ( I have seen an 800x600 on a 24"ish monitor). I have an old 15" monitor at a low resolution (800 x 600) so i can see what the program will look likes in less than idle conditions every now and then. I know that enterprise users pretty much have to accept what they are given but if you design a winform that doesn't fit into an 800x600 screen, it's not helping anyone.
Try to think about your user's end goals first before deciding what individual tasks they would carry out when using your software. The book About Face has excellent discussions on this sort of thing and though quite long is very interesting and insightful. It's interesting to note how many of their suggestions about improving software design seem to used in google docs...
One other thing, keep your user interface as simple and clean as possible.
Here is a great DotNetRocks podcast episode where Mark Miller talks about how to create Good UI; Even though the show title is .NET rocks, this episode talks about a general rule of thumbs on how to create a UI to increase program user's productivity.
Here is an episode exerpt
Good user interface design can be done by sticking to some good rules and avoiding common mistakes. You don't need to be a latte-sippin tattoo-wearin macbook-carrying designer to create user interfaces that work.
I like to follow these 3 guidelines:
Standard - follow known standards/patterns, reuse ideas from all products you respect
Simple - keep your solutions simple and easy to change (if needed)
Elegant - use less to accomplish more
The best technique I found is to put your self in the users shoes. What would you like to see from the GUI and put that in front. This also gives you the ability to prioritize as those things should be done first then work from there.
To do this I try to find "layers of usefulness" and add / subtract from the layers until it seems clean. Basically to find the layers I make a list of all the functions the GUI needs to have, all the functions it should have, and all the functions it would be neat to have. Then I group those so that every thing has logical ordering and the groupings become the "layers". From the layers I then add the most important functionality (or what would be used for Day to Day operation) and that becomes the most prominent part, and I work things into the feature around those items.
One of the toughest things is navigation as you have so much to give the use how do you make it helpful and this is where the layers really help. It makes it easy to see how to layout menus, how other pieces interact, what pieces can be hidden, etc.
I have found the easiest way to do this is to start by see what and how your users function on a day to day basis this which will make it easier to get in their shoes (even better is to do their job for a few days). Then make some demonstrations and put them in front of users even if they are Paper Prototypes (there is a book on this process called Paper Prototyping by Carolyn Snyder). Then begin building it and put it in front of users as it is built often.
I will also recommended the book Designing Interfaces by Jenifer Tidwell published by O'Reilly
The items in the list you presented are really situation dependent - they will vary from application to application. Some applications will need a save button, some won't. Some conditions will warrant a modal dialog box, some won't.
My top rule for designing a usable interface: Follow existing UI conventions. Nothing confuses a user more than a UI that doesn't work like anything they've ever used. Lotus Notes has one of the worst user interfaces ever created, and it is almost entirely because they went against common UI conventions with just about everything that they did.
If you're questioning how you should design a certain piece of your UI, think of a few standard/well-known applications that provide similar functionality and see how they do it.
If your UI involves data entry or manipulation (typical of business apps) then I recommend affording your users the ability to act on sets of data items as much as possible. Also try to design in such a way that experienced users can interact with the UI in a very random, as opposed to sequential way (accelerator keys, hyperlinks, etc).
Sung Meister mentioned Mark Miller. You can find some of his blog posts regarding great UI on the Developer express blog. Here's a screencast of his Science of great UI presentation: part1 and part2. (both require Veoh player).
You can also find him on dnrTV: Science of great user experience: part1 and part2.
Here's a google techtalks about user experience by Jen Fitzpatrick.
Cheers
When using a dropdown, the default dropdown height is usually too low (default is 8 items for winforms, for example).
Increasing it will either save the user a click if the number of items is low or make it easier to search the dropdown if there are a lot of items.
In fact, I see little point in not using all the available space !
This is so obvious to me now, but for example, it seems even VisualStudio designers haven't figured it out (btw, if you manually increase Intellisense's height, it will stay this way, but that's offtopic:))
I'll give one of my personal favorites: avoid dialog boxes at all costs. A truly good U I should almost never need to pop up a dialog box. Add them to your program only as a truly last resort.
For more, you might want to check out easily digestible ui tips for developers.
The Coding Horror Blog regularly gives great ideas. Just some examples:
Exploratory and incremental learning
Self-documenting user interface
Incremental search of features/Smart keyboard access
Task-oriented design (ribbon instead of menus and toolbars)
Providing undo instead of constant confirmation
Another aspect: use scalable icons to solve the problem of multiple user screen resolutions without maintaining different resolution bitmaps.