At this point, everyone knows that there's a limit to the number of ShellIconOverlayIdentifiers (from MSDN):
The number of different icon overlay handlers that the system can support is limited by the amount of space available for icon overlays in the system image list. There are currently fifteen slots allotted for icon overlays, some of which are reserved by the system. For this reason, icon overlay handlers should be implemented only if there are no satisfactory alternatives
I can understand the 15 overlay limt in Windows 95. But in an environment where there's Gigs of RAM, numerous Cores, and GPUs, is there some technical reason for such a low number in a modern operating system?
And why isn't this value configurable?
Before giving the 'performance' answer, consider:
Windows allows for configuration such that you can kill performance... why pick on this issue specifically?
Unless someone here happens to work on the Windows Shell team, I doubt that you're going to get an answer that really addresses the technical limitations and how they affect the design choice. But I'll try...
My guess is that there isn't any technical limitation, or at least there isn't one now. The real reason is presumably that no one has ever taken the time to sit down and update the code, the design, and the spec to lift this limitation. Features aren't implemented by default, and just because the computing environment has changed in the last few years doesn't mean that someone sat down and rewrote Windows to take full advantage of all those changes.
You should also consider that is more than likely a conscious design choice, rather than an imposed limitation. Raymond Chen (who actually does work on the shell team) published a blog entry responding to the uproar about Windows 7 removing the "sharing hand" overlay. He makes a compelling argument that the icon overlay is really not a desirable way of showing information (above and beyond the fact that the system is limited to 15) [emphasis added]:
Generally speaking, overlays are not a
good way of presenting information
because there can be only one overlay
per icon, and there is a limit of 15
overlays per ImageList. If there are
two or more overlays which apply to an
item, then one will win and the others
will lose, at which point the value of
the overlay as a way of determining
what properties apply to an item
diminishes since the only way to be
sure that a property is missing is
when you see no overlay at all. (If
you see some other overlay, you can't
tell whether it's because your
property is missing or because that
other overlay is showing instead of
yours.)
It seems reasonable to me that the extra clutter added to the shell is simply not worth it in the majority of real-world cases. The Windows Shell team obviously reached the same conclusion and cut the "sharing hand" overlay. Raymond's direct explanation:
Given the changes in how people use
computers, sharing information is
becoming more and more of the default
state. When you set up a HomeGroup,
pretty much everything is going to be
shared. To remove the visual clutter,
the information was moved to the
Details pane.
And, I know you specifically asked not to mention performance, but Windows really does try to keep you from shooting yourself in the foot. Users demand responsiveness in the shell, and overlay icons can interfere with this. As further evidence that they are not the priority, another blog post by the same Raymond Chen chastises:
Another example of applications having
a selfish view of performance came
from a company developing an icon
overlay handler. The shell treats
overlay computation as a low-priority
item, since it is more important to
get icons on the screen so the user
can start doing whatever it is they
wanted to be doing. The decorations
can come later. This company wanted to
know if there was a way they could
improve their performance and get
their overlay onto the screen even
before the icon shows up,
demonstrating a phenomenally selfish
interpretation of "performance".
Excellent response on the practical issues by Cody. As to why 15 and not some other number, the limit is baked into the ImageList control itself.
This is all very well and good, as explained by Cody Gray, but frankly it is pretty unimaginative, and as reported behind the scenes, sounding a bit frustrated.
In 2015 and with Windows 10, surely there can and needs to be a better ability, as I noted about thirty overlays present and had to prioritize ones I wanted most to see, which is not what you want most people to worry about at all. Also I see aggressive vendors like Box over-competing to try to prioritize themselves, and that will never go any place good.
Here's a possibility: What if multiply overlaid icons had a generic overlay indicator; a small rectangle matrix of multiple colors like the Google Chrome Apps button? Singly overlaid would just show the overlay out of a long list.
Then when the mouse pointer meets the icon, a small flyout window collects all the icon variations to view (at small icon size or a little larger). Each overlaid icon in turn announces by tooltip what it is, when you mouse over.
Now you can have all the icon overlays you need, for state in various clouds, for repository indications as for Tortoise tools, and so forth.
I quote an extract of the definitive answer here from Why is there a limit of 15 shell icon overlays? Raymond Chen 2019 post
The value 15 came from the corresponding limit for image lists. The
ImageList_SetOverlayImage function supports up to 15 image list
overlays per image list. (Hey, it used to be worse. The limit used to
be only 3!)
Okay, but why only 15? Why not more?
The overlay image is one of the pieces of information used when
drawing an image from an image list. The options are encoded in the
fStyle parameter, and when the bits were divided up for various
purposes, four bits were available to be used to specify the overlay
image. (You get 15 overlay images instead of 16 because you lose one
of the values in order to specify “no overlay.”)
Okay, but the values in the fStyle parameter use only the bottom 16
bits. What about the upper 16 bits? There’s plenty of room there.
The 16-bit limit was carried over from the 16-bit version of the
common controls (which still needed to be supported in Windows 95). Of
course, nowadays, nobody cares about the 16-bit version of the common
controls, so why not start using the upper bits?
There’s an unsatisfying explanation: The code internally that manages
the fStyle still uses a WORD in some places, so all the code that
manages the fStyle would have to be revised. This occurs in multiple
modules across Windows, so a synchronized change would have to be made
across multiple components. This is a breaking change at the binary
level because the interfaces are no longer compatible. Breaking
changes are procedurally difficult to coordinate: The affected code
may not be visible to the shell team because they are sitting in a
far-away leaf branch that has not yet RI’d to the trunk. It might be
that expanding fStyle from a WORD to a DWORD has far-reaching
consequences for some component.
Like I said, this is unsatisfying. Basically it boils down to “It
would be a lot of work and we are lazy.”
Related
I'm making a timeline with relatively small bars representing events over time. Right now they are roughly 8px by 200px (the length is variable, but all are 8px high).
Is there a standard reference for what size is standard or minimum for what types of interaction? For example I just want a mouse-over on these bars to highlight them and provide a tooltip, where I imagine if I expected a user to click on these bars, a bit bigger might be more realistic.
Anyone have any insights on this?
It'd probably be pretty hard to find guidelines specific to what you want to do.
There are several things you could do. Wikipedia has a nice article on usability
My main advice would be to think about what your users need, and what they are capable of using/learning. What you could do is make a few different versions where you change a parameter (such as the height of the bars) and get different people to test each one.
Personally, I would be happy to use bars that were 8px high. I would not be so happy to do this if I was using a touch screen.
You should think though, is it really necessary to fit that much information on one screen? How many bars do you think would be appropriate on average to be able to fit in the vertical space of the screen? Perhaps you could have two modes - large and small, where large fits 30 rows, and small can fit many more.
But it's not really something you should be too concerned about at this stage. Just make sure that it is easy to change it later on, and you won't have to worry about it now. User testing should sort out the problems - as long as you listen to there needs (or watch them, see the mistakes they make, and try to fix the most common ones).
One thing to remember, is the "rule of 7", which basically means people are more comfortable looking at less information, as too much information can overload the brain (but it really depends on the target audience, for instance, an historian may rather see as many concurrent events at once).
A requirment has been given that the desktop icon change colors "to fit well" with the users chosen desktop color. Not only is that a vague requirement but it is not the standard way that applications behave. The correct solution is to provide a number of icons which the users can select from. I just think its a bad idea. We are in the requirments phase still and I feel like I am becoming antagonistic. Standard practices exist for a reason but I am about to lose my cool because fully 15% of the 300 requirements fall into this category.
defer the argument by putting this on a "nice to have" list (that will never get implemented because it adds nothing to the functionality and creates disproportionate configuration problems) and concentrating on the real features that actually do important things
suggest a transparent icon so that it blends exactly with the desktop colors (see if they realize that it would be invisible!)
give a very long list of why this feature is ridiculous, create enemies, make the customer/manager feel stupid...not really recommended
bear in mind that you may be dealing with people that mean well but don't know much technically. Say "that's an interesting idea" to validate their feeling/input/existence, and go on to more important things. ;-)
All you Stackoverflowers,
I was wondering why GUI code is responsible for sucking away many, many cpu cycles. In principle, the graphical rendering is far less complex than Doom (although most corporate GUIs will introduce lots of window dressing). The event handling layer is also seemingly a heavy cost, however, it seems that a well-written implementation should switch between contexts efficiently on modern processors with a lot of memory/cache.
If anybody has run a profiler on their big GUI application, or a common API itself, I'm interested in where the bottlenecks lie.
Possible explanations (that I imagine) may be:
High levels of abstraction between hardware and application interface
Lots of levels of indirection to the correct code to execute
Low priority (compared to other processes)
Misbehaving applications flooding API with calls
Excessive object orientation?
Complete poor design choices in API (not just issues, but design philosophy)
Some GUI frameworks are much better than others, so I'd like to hear varied perspectives. For example, the Unix/X11 system is much different than Windows and even than WinForms.
Edit: Now a community wiki - go for it. I have one more thing to add -- I'm an algorithms guy in school and would be interested if there are inefficient algorithms in GUI code and which they are. Then again, it's probably just the implementation overhead.
I've no idea generally, but I'd like to add another item to your list - font rendering and calculations. Finding vector glyphs in a font and converting them to bitmap representations with anti-aliasing is no small task. And often it needs to be done twice - first to calculate the width/height of the text for positioning, and then actually drawing the text at the right coordinates.
Also, most drawing code today relies on clipping mechanisms to update just a part of the GUI. So, if just one part needs to be redrawn, the code actually redraws the whole window behind the scenes, and then takes just the needed part to actually update.
Added:
In the comments I found this:
I'm also very interested in this. It can't be that the gui is rendered using only the cpu because if you don't have proper drivers for your gfx-card, desktop graphics render incredibly slow. If you have gfx-drivers however desktop-gfx go kinda fast but never as fast as a directx/opengl app.
Here's the deal as I understand it: every graphic card out there today supports a generic interface for drawing. I'm not sure if it's called "VESA", "SVGA", or if those are just old names from the past. Anyway, this interface involves doing everything through interrupts. For every pixel there is an interrupt call. Or something like that. The proper VGA driver however is able to take advantage of DMA and other enhancements that make the whole process WAY less CPU-intensive.
Added 2: Ah, and for OpenGL/DirectX - that's another feature of today's graphics cards. They are optimized for 3D operations in exclusive mode. That's why the speed. The normal GUI just utilizes basic 2D drawing procedures. So it gets to send the contents of the whole screen every time it wants an update. 3D applications however send a bunch of textures and triangle definitions to the VRAM (video-RAM) and then just reuse them for drawing. They just say something like "take the triangle set #38 with the texture set #25 and draw them". All these things are cached in the VRAM so this is again way faster.
I'm not sure, but I would suspect that the modern 3D-accelerated GUIs (Vista Aero, compiz on Linux, etc.) also might take advantage of this. They could send common bitmaps to the VGA up front and then just reuse them directly from the VRAM. Any application-drawn surfaces however would still need to be sent directly every time for updates.
Added 3: More ideas. :) The modern GUI's for Windows, Linux, etc. are widget-oriented (that's control-oriented for Windows speakers). The problem with this is that each widget has its own drawing code and associated drawing surface (more or less). When the window needs to get redrawn, it calls the drawing code for all its child-widgets, who in turn call the drawing code for their child-widgets, etc.. Every widget redraws its whole surface, even though some of it is obscured by other widgets. With above mentioned clipping techniques some of this drawn information is immediately discarded to reduce flickering and other artifacts. But still it's lots of manual drawing code that includes bitmap blitting, stretching, skewing, drawing lines, text, flood-filling, etc.. And all this gets translated to a series of putpixel calls that get filtered through clipping filters/masks and other stuff. Ah, yes, and alpha blending has also become popular today for nice effects which means even more work. So... yes, you could say this is because of lots of abstraction and indirection. But... could you really do it any better? I don't think so. Only 3D techniques might help, because they take advantage of GPU for alpha-calculations and clipping.
Let's begin by saying that writing libraries is much harder than writing a stand-alone code. The requirement that your abstraction be reusable in as many contexts as possible, including contexts which you haven't though of yet, makes the task challenging even for experienced programmers.
Amongst libraries, writing a GUI toolkit library is a famously difficult problem. This is because the programs which use GUI libraries range over a very wide variety of domains with very different needs. Mr Why and Martin DeMollo discussed the requirements placed of GUI libraries a little while ago.
Writing GUI widgets themselves is difficult because computer users are very sensitive minute details of the behavior of the interface. Non-native widget never feel right, don't they? In order to get non-native widget right -- in order to get any widget right, in fact -- you need to spend an inordinate amount of time tweaking the details of the behavior.
So, GUI are slow because of the inefficiencies introduced by the abstraction mechanisms used to create highly-reusable components, that added to shortness of time available to optimize the code once so much time has been spent just getting the behavior right.
Uhm, that's quite a lot.
The most simple but probably obvious answer is that the programmers behind these GUI apps, are really bad programmers. You can go along way in writing code which does the most bizarre things and it will be faster but few people seem to care how to do this or they deem it to be an expensive non-profitable time wasted effort.
To set things straight off-loading computations to the GPU won't necessarily fix any problems. The GPU is just like the CPU except it's less general purpose and more a data paralleled processor. It can do graphics computations exceptionally well. Whatever graphics API/OS and driver combination you have doesn't really matter that much... well OK, with Vista as an example, they changed the desktop composition engine. This engine is far better composting only that which has changed, and since the number one bottle neck for GUI apps is redrawing is a neat optimization strategy. This idea of virtualizing your computational needs and only update the smallest change every time.
Win32 sends WM_PAINT messages to windows when they need to be redrawn, this can be a result of windows occluding each other. However it's up to the window itself to figure out whats actually changed. More than so nothing did change or the change that was made was trivial enough so that it could have been just preformed on top of what ever top most surface you had.
This kind of graphics handling doesn't necessarily exist today. I would say that people have refrained from writing really efficient and virtualizing rendering solutions because the benefit/cost ration is rather low/high (bad).
Something Windows Presentation Foundation (WPF) does, which I think is far superior to most other GUI API is that it splits layout updates and rendering updates into two separate passes. And while WPF is managed code the rendering engine is not. What happens with rendering is that the managed WPF rendering engine builds a command queue (this is what DirectX and OpenGL does) which is then handed of to the native rendering engine. What's a bit more elegant here is that WPF will then try to retain any computation which didn't change the visual state. A trick if you may, where you avoid costly rendering calls for things that doesn't have to be rendered (virtualizing).
In contrast to WM_PAINT which tells a Win32 window to repaint itself a WPF app would check what parts of that window requires repainting and only repaint the smallest change.
Now WPF is not supreme, it's a solid effort from Microsoft but it's not the holy grail yet... the code which runs the pipeline could still be improved and the memory footprint of any managed app is still more than I would want. But I hope this is the kind of answer you are looking for.
WPF is able to do some things asynchronously rather decent, which is a huge deal if you wanna make a really responsive low-latency/low-cpu UI. Asynchronous operations is more than off-loading work on a different thread.
To summarize things slow and expensive GUI means too much repainting and the kind of repainting which is very expensive i.e. the entire surface area.
I does to some degree depend on the language. You might have noticed that Java and RealBasic applications are a fair bit slower than their C-based (C++, C#, Objective-C) counterparts.
However GUI applications are much more complex than command line apps. The Terminal window needs only to draw a simple window that doesn't support buttons.
There are also multiple loops for extra inputs and features.
I think that you can find some interesting thoughts on this topic in "Window System Design: If I had it to do over again in 2002" by James Gosling (the Java guy, also known for his work on pre-X11 windowing systems). Available online here[pdf].
The article focuses on the positive side (how to make it fast), not on the negative side (what's making it slow), but it is still a good read on the topic.
Have you ever heard about an accepted paradigm about how to design those kind of systems?
Im not talking about iphones but photo-kiosk or manufacturing systems
Rulas,
i have worked on a number of touchscreen apps. i never found a published set of standards like the ones you mention but here is a little bit of what i learned:
Create a limited "visual vocabulary" with the following rules:
Buttons should be 30 or more pixels high (and at least as wide) - simply increasing the width of a button will not make it easier to click
Try to place controls on similar points on the screen - exactly the same if feasible, so that users do not have to hunt in different parts of the screen for the same operation.
Avoid the need for scrolling - try tabs, paging, wizards etc. Using scrollbars on a touchscreen is very difficult
Consider how the screen will be used. Where will users put their hands? Will they rest their hands on the corners of the screen? Will the Power button be in the way?
As part of this rule set, create your own controls library that can be easily reused in other parts of the app
Try to omit or minimize typing on a "soft" keyboard. Make as many fields selectable as possible.
Have big buttons for "fat fingered" users.
Usability really matters.
Keep the touch screen well calibrated. This used to be a nightmare back in 1999; don't know how much better it is now.
Speed and learnability do not directly fight each other, but it seems easy enough to design such a GUI that lacks either (or both) of them. GUI designers seem to prefer 'easy to learn' most of the time even when 'fast to apply' would be wiser.
There's only few UI concepts or programs that are weighted towards maximizing the peak efficiency of whatever you are doing with the program. Most of them haven't gotten common.
Normal people prefer gedit instead of vim. For normal people there are already good-enough GUIs because there was tons of research on them two decades ago.
I'd like to get some advices on doing UIs that do the tradeoffs from 'easy to learn' rather than from 'fast to apply'.
We have a product in our lineup that has won numerous awards based largely on its ability to provide more power with an easier interface than any of our competitors. I designed the interface a few years after holding a position in one of Bell Labs' human interface research groups so I had a pretty clear idea of what constituted "success" when I approached it. I have four pieces of design advice for creating easy but powerful interfaces.
First, select a metaphor that makes sense in their environment and do your best to stick to it. This doesn't have to be a physical metaphor although that can help if working with people who are not tech savvy. This was popular in the early days of Windows but its value remains. We used a "folder and page" metaphor that permitted us to organize a wide range of tasks while not crimping power users' style.
Second, offer a consistent layout relationship between data display and tasks. In our interface, each "page" displays a set of action buttons in the exact same position and, wherever possible, uses the same actual buttons. Thus, once one page is learned, the user has a head start on learning the rest. One of these buttons, always placed in a distinctive position, is a "Help" button...which brings me to point #3. The more general rule is: find ways of leveraging learning in one area to assist in learning others.
Third, offer context-sensitive help and make sure that it addresses the user's primary question (which is usually "what do I do now"?) How often have you seen technical help that simply shows you the Inheritance tree, constructor syntax and an alphabetical list of methods? That isn't help, it is abuse. We focused all of our help on walking people through sample tasks. In particularly tough areas, we also offered multimedia tutorials.
Fourth, offer users the ability to customize the interface. Our users often had no use for specific "pages" (analysis types) in their work. Thus, we made it very simple to turn them off so that the user would see an interface that was no more complicated than it had to be. Our app was usually installed by a power user and then used by multiple staff members so this was more of a win for us because we could usually count on the power user to understand what to shut off. However, I think it is good advice in general.
Good luck!
Autocad has a console mode. As you do things using the mouse and toolbars, the text-equivalent of those commands is written to the console. You can type commands directly in there. This provides a great way to learn the power-user names for commands (they are very short, like unix commands) which aids greatly the process of moving from beginner to productive power-user. Generally speaking, one primary focus has to be in minimising movement between the mouse and keyboard, so put lots of functionality into the mouse, or into the keyboard, because when you have to move your hands like that, there is a real delay in trying to find the right place to put them.
Beyond avoiding an angry fruit salad, just try to make it as intuitive as possible. Typically, programs with a very frustrating UI share one common problem, the developers didn't define a clear scope of what the program would actually do prior to marrying a UI design.
Its not so much a question of 'easy' , some people jump right into the UI and begin writing stuff to back the interface, rather than writing the core of a planned program and then planning an interface to use it.
This goes for web apps, desktop apps .. or even command line programs. A good design means writing the user interface after (and only after) you are sure that 'scope creep' is no longer a possibility.
Sure, you need some interface to test your program, but be prepared to trash it and do something better prior to releasing the program. Otherwise, there's a good chance that the UI is only going to make sense to you.
Rant (or, Stuff I think you should keep in mind):
Speed and learnability do directly fight each other. A menu item tells you what it does so that you don't have to remember. But it's much slower than a keyboard shortcut that you have to memorize to benefit from. The general technique for resolving this conflict seems to be allowing more than one way of doing things. While one way of doing something usually cannot be both fast and easy to learn, you can often provide two ways to accomplish the same task: one that's fast, and one that's obvious.
There are different kinds of people. The learning gap is a result of interest, motivation, intellectual capacity, etc. There is a class of person that will never bother to even learn which menu provides the action they want, and they'll scrub the menubar every time. There is also a (minority) class of person that thinks vim (or emacs) is the best thing since sliced bread. Most people probably fall somewhere in between these extremes.
My answer to the actual question:
I think you are asking how to strive for a fast UI. Your question wasn't particularly clear (to me).
First of all, be consistent. This helps both speed and learnability. Self consistency is the most important, but consistency with your environment may also be important.
For real speed, require as little attention and motion as possible. Keyboard shortcuts are fast because experienced users know where they are (they don't have to look), and their hands are already on the keyboard. Especially avoid forcing the user to change their position in front of the computer (e.g., moving one hand between the mouse and keyboard).
The keyboard is almost always faster than the mouse.
Customization (especially the ability to write custom scripts) will let power users make the interface work the way that is fastest for their specific habits.
Make it possible to get by without the most powerful features. All you need to know in order to survive in vim is "i, ESC, :wq, :q!". With that, you can use vi about the same way a lot of people use notepad. but once you start learning "h,j,k,l,w,b,e,d,c" (and so on) you get much more efficient. So there is a steep learning curve, but you can get by until you surmount it.
Keep in mind that if you focus on interface efficiency, you will probably limit your user base. Vim is popular among programmers, but lots of programmers use other tools, and it's virtually unknown among non-programmers. Most people want easy, not fast. Some want a balance. A very few just want fast.
I would like to point you towards Kathy Sierra's old blog for thoughts on 'easy to learn' and 'fast to apply' — I don't necessarily agree there needs to be a tradeoff between the two.
Three posts to get you started:
How much control should users have? This post ponders on whether 'fast to apply' is the ideal we should strive for.
The hi-res user experience talks about what you say about "normal people" vs. others. It's not so much that there are different kinds of people, but there are different levels of learning/expertise/involvement. Some are satisfied with less, some need more. How you get from less to more is arguably pretty much the same for everyone.
Finally, Featuritis vs. the Happy User Peak talks about the scope creep pointed out by #tinkertim.
Have you seen Gimp shortcuts?
Use nice visual controls and show keyboard shortcuts for them while hovering control - that will help to learn fast mode. If your software copy some behavior of other programs - copy shortcuts mapping from them (such as Copy/Paste/New Tab/Close Window/etc), but allow to dynamically re-map them as shown in Gimp. For reaped operations you could implement Action recoder. But it depends on type the software.
The main thing to be careful of is putting UI elements where they are most commonly located for other applications in that environment. For example, if you're going to make use of a menu system, people are accustomed to it being along top of the window by default for a desktop application. If you're in a web browser a menu system on a webpage seems out of place because it's not a consistent feature. If you're going to have an options/preferences configuration window, people are used to finding it under the Tools menu option, occasionally under the Edit menu. The main thing with keeping a UI "easy to learn" is that your UI elements shouldn't break the mold too much of how they're used in other applications.
If you haven't had the opportunity to see Mark Miller's presentation on The Science of Great User Experience, I'd recommend you watch the DNR TV episodes Part 1 and Part 2.
While I've been writing my own UI I've understood couple of things myself.
I imitated vim, but at the same time realized why it's so fast to use for text editing. It is because it acknowledges a thing: People prefer doing one thing at a time (inserting text, navigating around, selecting text), but they may switch the task often.
This means that you can pack different activities into different modes if you keep the mode switching schemes simple. It gives space for more commands. The user also gets better chances at learning the full interface because they are sensibly grouped already.
Vim is practically stuffed full of commands, every letter in the keyboard does something in vim, depending on the mode. Still I can remember most of them. And it's all because of modes.
I know bunch of projects that sneer at mode-dependent behavior. Main argument is the uncertainty of which mode you are in. In vim I'm never uncertain about the mode where I am in. Therefore I say the interface design is a failure if a trained user fails to recognize in which mode the interface is operating at the moment.