I want to get a list of all UI elements of a window, In a way I can recreate them on another PC? Currently I am focused on windows but a multi-platform answer will be greatly appreciated. Thanks! (What I mean below)
There are a couple of ways to go.
Option 1: If the window is a traditional one where the UI elements are child windows (rather than windowless controls), then you can use GetWindow or EnumChildWindows to iterate through the UI elements in a given window. You then would have to examine attributes of those windows, like the window procedure, window styles, and window text, to build a list like you're suggesting. You could probably make this work for all the standard controls. The problem is making it general enough to handle everything. Lots of applications have custom control types that your code wouldn't be prepared to recognize, or it might lay them out in ways that aren't obvious. You also won't be able to deal with windows that use windowless controls.
Options 2: You can use UI Automation to walk the tree of UI controls. This is how accessibility technologies like screen readers interact with applications.
Related
I'm using Windows's SetWindowPos API to try to 'crop' a part of a window so a user will only be able to see the RIGHT part of it.
Problem is, I can only 'crop' it, starting from the top left corner, and I would rather cropping a different part of the window which does not 'contain' the top left part.
As seen on the image below, what i'm trying to do is reverse the 'resizing' mechanism in such way that the window will 'hide' the Label1, and only show the Button1.
It is possible that I am not using the correct API for the job, If so, i'd love to be pointed toward the right one.
You need to move your controls on the Form for that to work. The window has a position and a size, which is all the window manager cares about. What is visible inside is your application's responsibility.
Your screenshot looks very VB6-ish. Do note that newer technologies, e.g. Windows Forms or WPF offer better support for what you are trying to do here.
If you have a window handle for the control, then you can place it outside its parent. Pass to SetWindowPos a negative value for the X coordinate.
Of course, the fact that the control cannot be seen does not mean it cannot be interacted with. The user can reach it with the keyboard if it accepts focus. So perhaps a better idea would be to hide and/or disable it. Use ShowWindow and EnableWindow to do that.
Then again, it's quite plausible that you won't be able to get a window handle. Many frameworks use controls that are not windowed. To interact with them you would need to use an automation framework such as UI Automation. Only you can know whether or not the target application supports automation.
Through experience I have found that the native windows forms/components don’t like to be changed. I know using Delphi or Visual Studio you are given native windows components to populate a form or window with and then you attach code on events that these components may do (onClick for example).
However, how do all of these programs like Word or google’s Chrome browser alter the standard windows’ window? I thought it was somehow protected?
Chrome seems to have tabs actually on the window’s frame?
I know you can also get toolkits like Swing and QT that have their own controls/components to populate a form. How do these work? (How does the operating system/computer know what a non-native button should act like? For example; Chrome's back and forward buttons, they're not native components?).
I can understand how OpenGL/DirectX window would work because you’re telling the computer exactly what to draw with polygons/quads.
I hope this question is clear!
Windows does not protect GUI elements. Windows and controls can be subclassed to handle various drawing operations in a custom way. For example, windows may override and reimplement the handling of the WM_NCPAINT message to draw a custom titlebar and frame:
http://msdn.microsoft.com/en-us/library/dd145212(VS.85).aspx
Some Windows controls have an "owner-draw" mode. If you use this, you get to draw the control (or at least vital parts of the control), while Windows takes care of responding to user input in the standard way.
Swing ant QT draw their own widgets at a low level using basic primitives, but they also have theme engines which can mimic the native controls.
Qt moved to native controls a while back. As for how swing does it, it gets a basic window from the OS. Then much like Opengl\Directx it does all of the drawing with in that window. As for where to position things that is what the layout managers do. Each manager has a layout style horizontal, vertical, grid, components it has to draw and a section of window it is expected to fill. From there it does some pretty easy math to allocate its space to its controls.
There's no magic: non native controls are simply drawn on a blank window. Or, instead of being drawn they may be represented as one of several bitmaps based on state (ie: a button may be represented as a .png for the normal state, another .png for the pressed state, etc)
I'm from a Windows programming background when writing tools, but have been programming using Carbon and Cocoa for the past year. I have introduced myself to Mac by, I admit it, hiding from UI programming. I've been basically wapping my OpenGL code in a view, then staying in my comfort zone using my platform agnostic OpenGL C++ code as usual.
However, now I want to start porting one of my more sophisticated applications to Mac OS.
Typically I use the standard Visual Studio dockable MDI approach, which is excellent, but very Windows-like. From using a Mac primarily now for a while, I don't tend to see this sort of method used for Mac UIs. Even Xcode doesn't support the idea of drag and drop/dockable views, unfortunately. I see docked views with splitter panels, but that's about it.
The closest thing I've seen to the Visual Studio approach is Photoshop CS4, which is pretty nice.
So what is the general consensus on this? Is there are more Mac-like way of achieving the same thing that I haven't seen? If not, I'm happy to write a window manager in Cocoa myself, so that I can finally delve in an learn what looks like an excellent API.
Note, I don't want to use QT or any other cross-platform libraries. The whole point is that I want to make a Mac app look like a Mac app, leave the Windows app looking like a Windows app. I always find the cross-platform libraries tend to lose this effect, and when I see a native Mac UI, with fancy Cocoa transitions and animations, I always smile. It's also a good excuse for me to learn Cocoa.
That being said, if there is an Open Source Cocoa library to do this, I'd love to know about it! I'd love to see how someone else achieves this, and would help smooth the Cocoa learning curve.
Cheers,
Shane
UPDATE: I forgot to mention a critical point. I support plugins, which can have their own UI to display various plugin specific information. I don't know which plugins will be loaded and I don't know where their UI will live, if I don't support docking. I'd love to hear people's thoughts on this, specifically: How do I support a plugin view architecture, if the UI can't change? Where do I put the plugin views?
Coming from a Windows background, you feel the need to have docking windows, but is it really essential to the app? Apple's philosophy (in my opinion) is that the designer knows better than the user how things should look and work. For example, iTunes is a pretty sophisticated app, but it doesn't let you change the UI around, change the skin, etc., because Apple wants to keep it consistent. They offer the full view, the mini player, and a handful of different viewing options, but they don't let you pull the source list off into a separate window, or dock it in other positions. They think it should be on the left, so there it stays...
You said you "want to make a Mac app look like a Mac app", and as you pointed out, Mac apps don't tend to have docking windows. Therefore, implementing your own docking windows is probably a step in the wrong direction ;)
+1 to Ken's answer.
From a user perspective unless its integral to the app like it is in Adobe CS or Eclipse i want everything as concise as possible and all the different options and displays out of my way so i can focus on the document.
I think you will find with mac users that those who have the "user skill" to make use of rearranging panels will in most cases opt for hot key bindings instead, and those who dont have that level of "skill" youre just going to confuse.
I would recommend keeping it as simple as possible.
One thing that's common among many Mac apps is the ability to hide all the chrome and focus on your content. That's the point behind the "tic tac" toolbar control in the top right corner of many windows. A serious weakness of many docking UIs is that they expect you to have the window take up most of the screen, because the docked panels can obscure content. Even if docked panels are collapsable, the space left by them is often just wasted and filled with white space. So, if you build a docking panel into your interface, you should expect it to be visible most of the time. For example, iTunes' source list is clearly designed to be visible all the time, but you can double-click a playlist to open it in a new window.
To get used to the range of Mac controls, I'd suggest you try doing some serious work with some apps that don't have a cross-platform UI; for example, the iWork apps, Interface Builder or Preview. Take note of where controls appear and why—in toolbars, in bottom bars, in inspectors, in source lists/sidebars, in panels such as IB's Library or the Font and Color panels, in contextual HUDs. Don't forget the menu bar either. Get an idea of the feel of controls—their responsiveness, modality, sizing, grouping and consistency. Try to develop some taste—not everything is perfect; just try iCal if you want to have something to make fun of.
Note that there's no "one size fits all" for controls, which can be an issue with docking UIs. It's important to think about workflow: how commonly used the control would be, whether you can replace it with direct manipulation, whether a visible indication of its state is necessary, whether it's operable from the keyboard and mouse where appropriate, and so forth. Figure out how the control's placement and behavior lets the user work more efficiently.
As a simple example of example of a good versus bad control placement and behavior in otherwise-decent applications, compare image masking in OmniGraffle and Keynote. In OmniGraffle, this uses the Image inspector where you have to first click on an unlabeled button ("Natural size") in order to enable the appropriate controls, then adjust size and position away in a low-fidelity fashion with an image thumbnail or by typing percentages into fields. Trying to resize the frame directly behaves in a bizarre and counterintuitive fashion.
In Keynote, masking starts with a sensibly named menu item or toolbar item, uses a HUD which pops up the instant you click on a masked image and allows for direct manipulation including a sensible display of the extent of the image you're masking. While you're dragging a masked image around, it even follows the guides. Advanced users can ignore the HUD entirely, just double-clicking the image to toggle mask editing and using the handles for sizing. It should be easy to see, with a few caveats (e.g. the state of "Edit Mask" mode should be visible in the HUD rather than just from the image; the outer border of the image you're masking should be more effectively used) Keynote is substantially better at this, in part because it doesn't use an inspector.
That said, if you do have a huge number of options and the standard tabbed inspector layout doesn't work for you, check out the Omni Group's OmniInspector framework. Try to use it for good, and hopefully you'll figure out how to obsess over UI as much as you do over graphics now :-)
(running in slow motion, reaching out in panic) Nnnnnoooooooo!!!!!
:-) Seriously, as I mentioned in reply to Ken's excellent answer, trying to force a "Windowsism" on an OS X UI is definitely a bad idea. In my opinion, the biggest problem with Windows UI is third-party developers inventing new and inconsistent ways of presenting UI, rather than being consistent and following established conventions. To a Mac user, that's the sign of a terrible application. It's that way for a reason.
I encourage you to rethink your UI app's implementation from the ground up with the Mac OS in mind. If you've done your job well, the architecture and model (sans platform-specific implementation) should clearly translate to any platform.
In terms of UI, you've been using a Mac for a year, so you should have a pretty good idea of "the norm". If you have doubts, it's best to post a question specifically detailing what you need to present and your thoughts on how you might do it (or asking how if you have no idea).
Just don't whack your app with the ugly stick by forcing it to behave as if it were running in Windows when it's clearly not. That's the kiss of death for an app to Mac users.
which of the following styles do you prefer?
An application which to perform tasks opens new forms
An application which keeps the various "forms" in different tabs
An application which is based on a PageControl and shows you the right tab depending on what you want to do.
Something else
Also do you have any good links for gui design?
From a programmers point of view, the PageControl solution quickly gets out of hand. Possibly too much code and certainly to many components on one form. (Originally this question was tagged Delphi, so I go from there.)
From a users point of view, the "opens new window" paradigm often is confusing. We people tend to think that we are able to multitask and handle many open windows and tasks, but we are not (we task switch at a loss of time like computers and add loss of accuracy).
Obviously this really depends on the type of application. But I would tend to a paradigm as Chrome and Firefox show in their latest incarnations:
keep the various forms in different tabs
let the user detach a tab into its own form (dock and undock via drag%drop)
add a good way of navigation
I implement something like an SDI as main screen of an application too. Look at something like "outlook style". Navigation, list of objects, object details in different panes, some additional panes like a cockpit. And then open a new window/form for certain tasks (some modal, some non modal), but short lived. After the email is written, it is sent and closes the window. But I have, if I am capable of doing so, the possibility to work on multiple emails at the time.
Look at the problem. If it has dashboard character, take "outlook style" or so. If the users are a wide spread, heterogeneous, non computer savvy crowd, use SDI or forms on tabs. If you write for programmers, you might go for multiple forms, just because we tend to think that we can handle it. And it works for multiple screens (hopefully).
MDI is the worst choice possible, in my opinion. There's nothing I hate more than having to resize a bunch of windows, or tile them or whatever.
Tabs are bad, too, especially if you have more than one row of them (or if you have one row but still have more tabs than will fit, and have to use some funky scrollbar or "more" button with them).
I would rather see the programmer think about the problem and just show me what I need to see based on what I'm doing as a user. Implementing the different user interfaces in your programs as user controls (as opposed to discrete forms) and then showing them or hiding them based on the current context is the way to go.
The Tabbed form is a good idea if you use a frame for each tab content. This keeps you out of trouble from getting too much code in one single form unit. Try to do the same as Google Chrome. I personally create a menu with the options that are actually frames that loads only when the user asks for it, so there will never be many tabs visible unless the user needs them all opened.
I have a custom control: it's managed code, which subclasses System.Windows.Forms.Control.
I want to add things like edit boxes, selection lists, combo boxes, radio buttons and so on to places on this control. An easy way to do this is to simply add instances of these classes to the Controls collection, so that they become child controls.
Adding them as child controls might create some subtle problems, for example:
IE 6 select controls(Combo Box) over menu
I have scrollbars on my control which appear to scroll the contents of the control (the contents are bigger than the control itself); when a child control is near the edge of the screen then I'd like to half-display (i.e. clip) that child (i.e. to have half of it located off the edge of the physical screen), but a true child control cannot be located outside the border of its parent.
Are there other potential problems?
When I use IE7 to display http://www.tizag.com/htmlT/htmlselect.php (for example), which contains combo boxes etc., and when I then use Spy++ to spy on IE7 when I'm doing that, I see only a single Window/control instance with no children (whose class name is "Internet Explorer_Server").
I'm guessing this means that in IE7, the functionality to render a combo box is built in to the IE7 control itself, and that IE7 does not use standard controls as child controls.
Questions:
Is it better to reuse standard controls as children of a custom control, or, to reimplement the functionality of standard controls within a custom control itself?
Do you have any caveats (warnings) to share, related to either scenario?
If I wanted to reimplement the functionality of standard controls within a custom control, do you know of any existing code (which implements this functionality) that I could re-use?
If such code already exists, I don't know how to search for it (my searches find, for example, owner-draw combo boxes, and extensions to standard combo boxes): perhaps few people reimplement the standard controls from scratch?
Edit
I found a semi-related question: How to render a control to look like ComboBox with Visual Styles enabled?
Yes, Internet Explorer draws the controls using the Windows theming APIs. You can do this too using the types defined in the System.Windows.Forms.VisualStyles namespace.
The IE team did this to avoid performance problems of having so many controls, each receiving window messages, on screen at once. For example, looking at this StackOverflow.com page, I see 30-40 link label controls, 10 buttons or so, 20+ labels, etc.
It should be noted the Zune software, which is .NET managed code, also uses custom controls; if you try to use Spy++ on any of the controls, you'll see they aren't real Win32 controls. You may use Reflector on the Zune software to see exactly what they're doing. If I recall right, they're using a custom managed UI framework that's included in the Zune software.
As far as rewriting these controls from scratch, I think there's a ton of work to be done. It sounds easier than it really would be.