Answers.com has a taskbar application that when you ALT + mouse-click on a word in any program it will pop up a window with information pulled from their website.
My question is-- what are the actual programming mechanics and APIs used to do something like this? I don't have Windows application programming experience and am trying to figure out where to start. How do you access the current word pointed to by the mouse?
Anyone aware of any examples or open source software that does anything like this?
It's been a while and the last time I did something like this it was within my own wysiwyg editor so I had full access to all font characteristics needed to calculate which word was clicked by the mouse.
Maybe there's a n easy way to do this if all your apps are .NET or com or share some other framework which provides a way to retrieve this directly.
Via the API, I would look into hooking the keyboard and mouse messages so that your app can pre-process every mouse click on other applications - start with SetWindowsHookEx and read everything you can about hooking messages.
After getting your app to pre-process the messages, you then need to grab the text being clicked. Since text can be painted onto a device context in many different ways, you may be best off doing a screen scrape of the clicked area because the text may only exist as a bitmap. If this is the case, you have to perform some OCR to translate the scraped bitmap back into text. In other cases, the text may reside in the window as text - the WM_GETTEXT message may return this text from some types of windows (e.g. textboxes, buttons, etc.) but for normal windows, this message only return the title in the caption bar.
Sorry I don't have any definite answer, but this may get you started in the right direction.
Related
I am trying to capture windows hidden behind my application. I am using windows 7 and VC++. I have tried printwindow() function which draws the both non-client and client area of hidden window, but captured window in the device context doesn't show desktop composition effects(aero effects). Instead it shows the captured window with windows 7 basic theme.
I have also tried with GetWindowDC() to retrive the DC of hidden window, and then Bitblt() it to memory DC but the captured window doesn't show non-client area (caption, close button, minimize button etc) correctly.
Anybody faced this issue?
Please help.
Click the link below. It leads to a MSDN site that lists all the existing Windows Functions ever of all history since Windows 95 up to Windows 8 (from period where Microsoft started Windows until present). It shows old windows functions of first Windows and new windows functions added for the new windows.
http://msdn.microsoft.com/en-us/library/windows/desktop/ff468919(v=vs.85).aspx
Anyway follow this site.
You will see the name of each function as a link.
Click any of them that you are interested.
Each link there leads to another MSDN site that explains all the basics knowledge that you must know about the function before using it, that you want to learn more. What that function does, its purpose, all its parameters and how to use each one, all their flags, all parameters types, return value and at last remarks section that shed more light and sometimes gives tips about the selected function.
Of course, you don't have to read all of them. Find in the list only the necessary functions to fit your needs. The functions that will solve your problem and answer your question that you posted.
By the way, I read your post, and I think that I found in the list the necessary functions that will do what you want to do, I will list them below, and say in one sentence what each does for what you need:
AnimateWindow - Enables you to produce special effects when showing or hiding windows. There are four types of animation: roll, slide, collapse or expand, and alpha-blended fade.
FlashWindow - Flashes the specified window one time. It does not change the active state of the window.
FlashWindowEx - Flashes the specified window specified number of times. It does not change the active state of the window.
Use these functions to achieve the aero effects that you want.
SetWindowPos - Changes the size, position, and Z order of a child, pop-up, or top-level window. These windows are ordered according to their appearance on the screen. The topmost window receives the highest rank and is the first window in the Z order.
Use this function to show the hidden windows on the top side (above all other windows) and on the screen front of you. The operating system will automatically draw the both non-client and client area of these windows without using any gdi, draw and paint functions yourself.
If you want these windows to return back to their previous state (where they were hidden), then save their state with GetWindowPlacement function and later call SetWindowPlacement to bring them back to their hidden state. You can try GetWindowRect and SetWindowPos instead to achieve the same goal.
I also think that you will be interested in GetWindowTheme and SetWindowTheme functions and all the draw theme functions (BackgroundEx, Edge, Icon, Text, TextEx).
There are more theme functions. Find in msdn and in other sites on the web.
Babylon dictionary and a couple of other dictionaries allow to click on any word in any windows program
and automatically recognize the word under the cursor, and at once open the dictionary window while searching for that word in installed dictionaries.
You can on the other hand open your dictionary, type in your word and press Enter, the result will be the same.
There's a Delphi form, containing a text label, for example with the word "Automaton".
My question is:
How to send a word from my Delphi application right into the dictionary window, as if you typed it manually and pressed Enter?
The best solution is to send some message through the Windows mechanism, but if it is too complicated, there's another solution, and so the second answer: as I described, we need to model a [Ctrl+left mouse] click on a form where this word is displayed on a form [ a visual label on the screen of my Delphi application], to be exact, on some central pixel of this label.
Could you kindly give an advice how to do one thing or another in Delphi ?
** edit:
The problem with AppActivate is this: Babylon dict has a daemon part that seats in the tray.
In the task manager a real window where the text should be input also is named 'Babylon'.
So AppActivate('Babylon') tries to bring to front the non-visual part of the application.
Do you have any suggestion how to determine the windows handle or something of a real visual part of the application? In the task manager, I repeat both visual and non-visual parts are named 'Babylon'.
I cannot offer an answer so much as some insight and advice...
There are certain applications which "intercept" keyboard and mouse instructions, and essentially "nullify" them if they are being immitated by software. Generally-speaking, you'd only see this in proper AntiVirus software such as Kaspersky by design... however:
The way some (not many, but some) programs hook keyboard and mouse inputs, as a side-effect, behave the same way. If you have attempted all of the advice given as comments above, and cannot get Babylon to trigger an action as a result, it is likely Babylon behaves as I have described.
If what I suspect is true, then the method you are attempting is simply not possible (at least, not using any simple Pascal code on its own... ASM might be able to do it but that's beyond my knowledge).
A better solution may be to do a little research to see if any of the following options are available to you:
1) Does Babylon have a Pipeline or API you can use to interface your application(s) with it?
2) Is the particular functionality you require of Babylon accessible through one (or more) DLL files distributed as part of Babylon?
3) Is there an alternative to using Babylon for your needs?
I know it's not an answer as such (certainly not one you'd want to hear), but it may point you in a better direction.
I would like to know what should I do with GUI layout under WinAPI/MFC.
In the ideal world I should just create the form/dialog via resource editor, and everything should just work. In the real world the dialog editor is ancient behemoth from the ice-age and doesn't support most of the comctl32 controls.
This is where the problem creeps up. Dialog editor uses DLU units, and when I create new controls at runtime, I have to express them in pixel offsets.
I stumbled upon one article about calculating DLUs based on font http://support.microsoft.com/kb/145994/en-us, but also saw a warning somewhere that dialogs can have non system fonts in some circumstances, so this approach is not very safe. Plus the article seems to look only at English characters, without regard to all other characters in unicode space which might be wider?.
Has anyone done a research in this direction and found a better way?
P.S.: No Winforms/WPF/Delphi, requirements.
the dialog editor is ancient behemoth from the ice-age and doesn't support most of the comctl32 controls
It doesn't need to support the controls directly, you can still use it just for positioning by inserting it as a custom control and filling in the window class in the property page. For example, that's how I insert link controls in VS2005: as a custom control with class "SysLink".
MapDialogRect (mentioned in the article) is the function that Windows uses to translate the dialog units in the dialog resource to pixel units. MapDialogRect works (where GetDlgBaseUnits fails) because its given an actual handle to the dialog box, and can send it a WM_GETFONT message to retrieve the actual font the dialog will be rendered with.
I have a bit of a challenge.
In an earlier version of our product, we had an error message window (last resort, unhandled exception) that showed the exception message, type, stack trace + various bits and pieces of information.
This window was printscreen-friendly, in that if the user simply did a printscreen-capture, and emailed us the screenshot, we had almost everything we needed to start diagnosing the problem.
However, the form was deemed too technical and "scary" for normal users, so it was toned down to a more friendly one, still showing the error message, but not the stack trace and some of the more gory details that I'd still like to get. In addition, the form was added the capabilities of emailing us a text file containing everything we had before + lots of other technical details as well, basically everything we need.
However, users still use PrintScreen to capture the contents of the form and email that back to us, which means I now have a less than optimal amount of information to go on.
So I was wondering. Would it be possible for me to pre-render a bitmap the same size as my form, with everything I need on it, detect that PrintScreen was hit and quickly swap out the form contents with my bitmap before capture, and then back again afterwards?
And before you say "just educate the users", yes, that's not going to work. These are not out users, they're users at our customers place, so we really cannot tell them to wisen up all that much.
Or, barring this, is there a way for me to detect PrintScreen, tell Windows to ignore it, and instead react to it, by dumping the aformentioned prerendered bitmap onto the clipboard ready for placing into an email?
The code is C# 3.0 in .NET 3.5, if it matters, but pointers for something to look at/for is good enough.
Our error-reporting window has these capabilities:
Show a screenshot that was taken when the error occured (contains all the open windows of the program at the time, before the error dialog was shown)
Show a text file containing every gory detail we can think of (but no sensitive stuff)
Save the above two files to disk, for latter attaching to an email or whatnot by the user
Sending the above two files to us by email, either by opening a new support case, or entering an existing support case number to add more information to it
Ignore the problem and hope it goes away (return to app)
Exit the application (last resort)
We still get screenshots from some users. Not all, mind you, so my question is basically how I can make the PrintScreen button help us a bit more for those users that still use it.
One option: Put the stack trace and other scary stuff into the error screen using small, low-contrast type -- e.g. dark gray on light gray -- so that the user doesn't really even see it, but the Print Screen captures it.
But if you want to detect the PrintScreen and do your own thing, this looks like an example of what you want.
Wouldn't it be possible to disable the Print Screen button altogether when the error popup is active? Have it display a message along the lines of "Please use the clearly visible button in the middle of your screen to report the error" I agree it breaks expected functionality, but if your users are really that stupid, what can you do...
Alternatively, have it report errors automatically (or store the data locally, to be fetched later, if you can't send without asking for some reason), without asking the user. If you want to be able to connect print screened screenshots with detailed error data, have it send a unique ID with the data that's also displayed in the corner of the popup.
What about offering them a "Print Screen" button that performs these actions as well as performing the print screen? If you're locked into this method of having your customers send error details, this may be an easier route to take.
Lifted from my comment below for easier reference (looks helpful, perhaps):
codeproject.com/KB/cs/PrintScreen.aspx
This is in theory...the best way to deal with it I would think
Intercept a WM_PRINT message or inject one into your process... see this article here
Install a system-wide keyboard hook and intercept the print-screen key and swap it around with your contents prior to the capture. Now, I can point you to several places for this, here on CodeProject, and here also, keyboard spy, and finally, global Mouse and keyboard hook on CodeProject.
Now, once you intercept the print screen, invoke the WM_PRINT message by capturing the contents that you want to capture.
I know this is brief and short, but I hope this should get you going.
The only solution i came up with was to offer big, large, easy to read toolbar buttons that give the user every opportunity to save the contents of the error dialog:
Save
Copy to clipboard
Send using e-mail
Print
And after all that, i use the Windows function SetWindowDisplayAffinity in order to show the user a black box where the form should be:
This function and GetWindowDisplayAffinity are designed to support the window content protection feature that is new to Windows 7. This feature enables applications to protect their own onscreen window content from being captured or copied through a specific set of public operating system features and APIs. However, it works only when the Desktop Window Manager(DWM) is composing the desktop.
It is important to note that unlike a security feature or an implementation of Digital Rights Management (DRM), there is no guarantee that using SetWindowDisplayAffinity and GetWindowDisplayAffinity, and other necessary functions such as DwmIsCompositionEnabled, will strictly protect windowed content, for example where someone takes a photograph of the screen.
If their screenshots show a big black box, hopefully they'll get the hint.
I did add a defeat, if they hold down shift while clicking "show error details", i don't add the protection during form construction:
//Code released into public domain. No attribution required.
if (!IsShiftKeyPressed())
SetWindowDisplayAffinity(this.Handle, WDA_MONITOR); //Please don't screenshot the form, please e-mail me the contents!
Often times convention is one of the most important design consideration for user interface. Usually the advice goes to do it like Microsoft does.
This is for three reasons:
If it ain't broke, don't fix it.
If your users expect to click on a floppy disk icon to save, don't change the icon (even though some of them may have never seen an actual floppy disk).
Users don't want to re-learn the interface (and hot keys, etc.) with each different application they use.
At the same time Emmerson said "*A foolish consistency is the hobgoblin of little minds.*" So when does maintaining a consistent user interface cross the line from a good idea to stagnated innovation?
Microsoft shook up the good old WIMP GUI with the introduction of the tool bar, and then again with the Ribbon control (which is the natural evolution of the tool bar, like it or not.) Now we are seeing ribbons everywhere.
So my question is, what are some user interface paradigms that are accepted and consistent across multiple applications, but have stayed past their prime and are starting to reek? Are there some important changes that would benefit from a grass roots push by developers to innovate and improve the user interface experience for our users?
One thought that came to mind for me is the modal pop-up dialog. You know the ones that say: "Are you sure you want to . . .. - [Yes] [No] [Cancel] [Maybe]" and its evil twin "Successfully completed what you wanted to do! [OK]." We are seeing a movement away from these with the "info panel" in browsers. I think they need to be adopted in windows application development as well.
If possible please list a solution for each stale UI item.
And please don't list clippy. We all know he was a bad idea.
NOTE: This is specifically Windows client user interface paradigms, but I am certainly open to drawing inspiration from the web, the Mac, etc.
You mentioned popup modal dialogs , and I'd argue that non-modal ones are just as bad. Any dialog box remove focus from the program, they could end up behind the program and make it hard to find it, they might not even appear on the same virtual screen.
I'd like to see an end to all dialog boxes. If you need to stop someone from using the UI because of some non-normal circumstance, then remove the relevant parts of the UI from the window, and replace it with what the dialog would contain. Bring back the UI once the problem has been handled.
Clicking things on touch interfaces
It's incredibly difficult to click on things on a touch interface, because you don't know when you have pressed the screen hard enough. And if you add an animation to the button you are clicking, you most likely wont see it, because your finger is in the way. Adding other reactions, like vibrating the phone or painting waves on the screen might work, but there is usually a delay which is too large, much larger than the tactile sense of a button being pressed. So until they invent a screen with buttons that can be pressed, all touch devices should move towards dragging user interfaces (DUIs) instead.
Counter intuitively it is easier to press an object on the screen, drag it, and then release it than it is to just press and release it. It's probably because you can see the object moving when you start dragging, and you can adjust the pressure while dragging it. Dragging also has a lot more options, because you now have a direction, not just a point that you clicked. You can do different things if the user drags the object in different directions. Speed might also be used, as well as the point where the user releases the object. The release point is the real strength of DUIs, because it is very easy to release something, even with pixel precession.
Some designs have started to use DUIs, like (here we go) the iPhone, palm pre and android phones. But only part of their design is DUI, the rest is clicking. One area they all have in common is the keyboard. Instead of clicking on a key the user presses any key, then drags their finger towards the key they really wanted to click. Unlocking these phones also uses dragging.
Other easily implemented DUI features would be things like mouse gestures, where dragging in different directions, or drawing different shapes does different things. There are also alternate keyboards being researched which puts a bigger emphasis on dragging. All buttons can be changed into switches, so have to drag them down a bit to click them. With a well designed graphics, this should be intuitive to the user as well.
The Apple Human Interface Guidelines are a good read on this topic. They discuss this from a very broad point of view and the guidelines apply to any platform, not only Mac.
The file system. I want to save a file.. >OOOPs I need to think of a file name first. Well.... how about ... blah.doc.
6 months later...
Where the %#*(%& * did I save that %()#*()*ing file?
The solution is build a versioning system into the application, or better, the OS. Make files findable by their content, with a search engine, instead of forcing the user to come up with a memorable name, when all they want is for their file to not get lost.
Eliminate the save step. Type something in to the application, and it's just there, and there's no risk of losing it by some misstep, like forgetting to save. If you want an older version, you can just pick a date and see what the document looked like back then.
To build on the search engine idea: It's a pain having to navigate some arbitrary tree structure to find your stuff. Searching is much easier. However, you might still want to have something like a "folder" to group multiple files together. Well, you can build a richer metadata system, and have a "category" or "project" field, and setup the search engine to show items by project, or by category. Or group by those, or whatever new UI discovery we make next.
This question is a bit too open-ended, IMHO.
However, my main approach when designing anything is:
Fits in to wherever it is. If it's a windows app, I copy MS as much as a possible
It's simple.
It provides options
Buttons have a nice description of what the result of clicking will be, as opposed to 'yes or 'no'
Harder to answer the rest of your post without spending hours typing out an arguably useless (and repeated) set of guidelines.
In my mind, the one thing that really stands out is that USERS need more and easier control over the application's user interface appearance and organization.
So many interfaces can not be modified by the user so that the most used/favorite functions can be grouped together. This ability would make your favorite software even easier for you to get things done.
Error messages need a "Just do it!" button.
Seriously, I really don't care about your stupid error message, just DO WHAT I TOLD YOU TO DO!!!
I think the entire Document model of the web needs to change. It's not a user interface, but it leads to many, many bad user interfaces.
The document model was a good idea to connect a bunch of documents, but now the web is also a collection of applications. Today, I think the Page/document model corrupts our thinking. We end up lumping things together that aren't related, modularizing our code wrong, and in the end confusing users with our monolithic control board type websites.
Find dialogs that sit over the widget in which you are doing the search are terrible. Loads of apps do that. The find bar in Firefox works much better.
Many applications have multiple panes within the UI - eg in Outlook there's the preview pane and the inbox pane (amongst others). In these applications typically cursor key presses apply to the currently focussed pane. But there's very poor hinting to show the user which pane has focus and there are seldom keyboard shortcuts to move the focus between panes.
The focussed pane should be highlighted somehow.
Something like alt+cursor keys should move the focus around.
Ctrl-Tab and Ctrl-Shift-Tab cycle left and right through tabs instead of MRU behavior, even though in most cases the same behavior is duplicated with Ctrl-PageUp and Ctrl-PageDown.
There are a lot but here's an idea for a couple of them:
Remove some clicks like in "add another" or "search item" and the like.
This is well done with interfaces like ajax which have autocompletes ( and auto search ) but is slowly being adopted for platform UI's ( and in some cases they were originated in platform UI's. )
This is how StackOverflow does it for some scenarios.
But of course, we all know that already don't we? No need for "Seach tag" or "Add another tag" buttons, they just happen
Dialogs as you described.
Guys at Humanized proposed Transparent messages which actually are used in their product Enso and some other places.
Mac uses them for notifications ( like in Growl ) use them very well, or Ubuntu new notification system.
alt text http://blogs.sun.com/plamere/resource/NowPlayingGrowl.png
Firefox replaces the traditional "Search" dialog box with a search bar at the bottom.
Although not everyone likes the placement for next/previous as in this screenshot
And even SO ( again ) :) replace the notification with the yellow bar.
Finally:
File managers
I really like ( sometimes ) the simplicity of regular file managers, but some times I would like to work faster/better with them.
If you compare IE 4 with IE 8 you can tell the advance ( even better compare IE 4 with Google Chrome )
But if you compare Windows 95 Explorer with Win XP they are almost the same!! ( Win Vista/7 is a step forward )
But I wonder: Why haven't file managers improved as much as webbrowsers?
That's one reason I like stuff like QuickSilver but it is just a step. Much work is needed to create something like a "Perfect program launcher" or (FileManager/DesktopSearcher etc as you wish )
QuickSilver featuring "move to" action