There are some apps which can scan the screen and detect things in real time. For example, the macOS preinstalled app "Digital Color Meter". I can move the cursor and the app detects immediately which color is in the area around my cursor. So my question is, how can I do things like this? How can I "scan" the screen and detect objects or colors in a selected area in real time? I can't find a solution.
Digital Color Meter only captures a small square of the screen. If that's all you need, try CGDisplayCreateImageForRect and see if it's fast enough.
If that's not fast enough, look at the CGDisplayStream functions, starting with CGDisplayStreamCreate or CGDisplayStreamCreateWithDispatchQueue. These functions are significantly more complicated than CGDisplayCreateImageForRect and you'll have to learn about IOSurfaceRef to get at the pixel data from a CGDisplayStream.
Related
I am color blind. This is usually not a problem for me, but whenever a game or application uses red and green as contrast colors it can become an annoyance. Looking for accessibility tools online, all I've managed to find are tools that adjust colors on snapshots, or on a camera input. For this reason, I've been toying with the idea of writing my own color adjustment tool.
Suppose I would want to write my own shader or post effect that shifts or swaps color values, then apply it to everything I see in Windows (10) in real-time - is there a way to do this? For example, is it possible to insert a pixel shader into the Windows rendering pipeline? Or would it be possible to write an application that takes the entire screen as input and outputs something else, faster than say 5ms per frame using a modern GPU? How would you approach this?
I'm wanting to experiment with creating a rear backlight system for my monitor, so that it changes the colour of the wall behind it to match what colours are currently on the display.
Part of this challenge would presumably be to create some software in OS X which is able to efficiently sample the state of the current image displayed on the monitor in a way that's fast (thirty times a second would be nice), and not resource intensive. As such I thought an efficient solution would be to sample a hundred or so pixels across the screen and average them to create a reasonable guess at the screen's average colour.
I suspect an efficient solution would be, if OS X provides access to a raw display buffer, just picking out a few values from there every few milliseconds, but I couldn't find any APIs like that. I've seen the functionality in AVFoundation which allows you to record the screen as a capture device, but I'm worried that wouldn't be very efficient to keep running constantly.
Is there an efficient solution? Accuracy of the current image's average colour would be nice but isn't terribly important.
I've tried to find an answer for this on MSDN, but I'm not getting a clear picture of how this is intended to work. All of my work is on Windows 8.1.
Here is my issue. I am working on a Laptop with a high resolution monitor, 3200x1800. I've been using EnumDisplayMonitors to get the bounding rectangle of my screen.
This seems to work fine if my display settings are default. But I've noticed that when I change the Window display settings to provide larger text, the resolution returned by EnumDisplayMonitor changes. Rather than getting 3200x1800 I will get 2133x1200.
I'm guessing since I asked for larger text, Windows chooses to represent the screen as a smaller resolution.
It seems that if I look at the virtual screen properties, everything is represented in the actual coordinates of my screen, i.e. 3200x1800. But the APIs for getting the window and monitor rectangles seem to operate on this "other" coordinate space.
Is there any documentation/Windows APIs to handle the conversion between these "other coordinates" and the "virtual coordinates"? i.e. if I want EnumDisplayMonitor or GetMonitorInfo to give me the true screen coordinates, how could I convert 2133x1200 to 3200x1800?
You have increased the DPI of the video adapter to 150% (144 dots per inch) to keep text readable and avoid having windows the size of a postage stamp. Quite necessary on such high resolution displays. But you haven't told Windows that your program knows how to deal with it.
So it assumes your program is an old one that was never designed to run on such monitors. It helps and lies to you. It gets your program to render its output to a memory buffer, then takes that output, rescales it by 150% and copies it to the video adapter. This is something you can see, text looks fuzzier if you put your program's output next to a program that doesn't ask for this kind of scaling, like Notepad.
And of course, it lies to you when you ask for the size of the screen. It tells you that it is 150% smaller than it really is. So that, after rescaling, a window you create will fill the screen.
Which is all just fine but of course not ideal, your program doesn't look as good as it should. You have to tell Windows that you know how to deal with the higher resolution. Do beware that this looks easier than it is in practice. Getting text to look crisp is trivial, it is bitmaps that are problematic. And in general a fertile source of bugs, even the big companies can get this wrong.
Before I start with an answer, let me ask: what are you really trying to do ? Or more specific - why do you need to know the monitor resolution ? The standard way to do this is to call GetWindowRect(GetDesktopWindow(), &rect) I'm not sure if the screen coordinates change based on DPI settings - but you should try that instead of GetMonitorInfo as the latter is for more advanced stuff. And if GetWindowRect still returns back a scaled rect, just call DPtoLP, LPtoDP or other mapping coordinate function as appropriate.
When you adjust the display settings as you described, you are actually changing the DPI settings of the screen. As such, certain APIs go into compatibility mode so that they allow the app to create larger elements and windows without knowing anything about this setting.
Why do you need to know the actual screen resolution since most of the windowing APIs will behave accordingly when the DPI scaling changes?
I suspect you could call SetProcessDPIAware or the manifest file equivalent. But do read this MSDN article first to understand DPI scaling.
How can I get the terminal (or Visor/TotalTerminal) to smoothly scroll when it prints out text. So that I don't completely lose my sense of location when for instance showing git diffs. Or, when navigaiting long diffs in vim-mode using CTRL-D and CTRL-U. I don't want the terminal to "jump" to a new place, but rather smoothly flow to it.
(I'm using osx snow leopard)
You can't.
Terminal emulators work with lines and characters as their smallest units, not your display's pixels.
Consider each individual character as your terminal's pixel: just as your display can't show half pixels, your terminal can't show 1/2 or 1/4 or 1/8 of a character.
Smooth scrolling is the norm in GUI land because text (or any meaningful piece of information) is usually a lot taller than 1 pixel. You can hide parts of it without losing too much information. Up to a certain threshold, of course.
In CLI land, the smallest display unit is also the smaller bit of information. It's either there or not there.
So, smooth scrolling — pixel by pixel — doesn't really make sense and doesn't seem to be a trivial thing to implement.
I remember my old Radeon graphics drivers which had a number of overlay effects or color filters (whatever they are called) that would render the screen in e.g. sepia tones or negative colors. My current NVIDIA card does not seem to have such a function so I wondered if it is possible to make my own for Vista.
I don't know if there is some way to hook into window's rendering engine or, alternatively, into NVIDIA's drivers to achieve this effect. While it would be cool to just be able to modify the color, it would be even better to modify the color based on its screen coordinates or perform other more varied functions. An example would be colors which are more desaturated the longer they are from the center of the screen.
I don't have a specific use scenario so I cannot provide much more information. Basically, I'm just curious if there is anything to work with in this area.
You could have a full-screen layered window on top of everything and passing through click events.. However that's hacky and slow compared to what could be done by getting a hook in the WDM renderer's DirectX context. However, so far it's not possible, as Microsoft does not provide any public interface into this.
The Flip 3D utility does this, though, but even there that functionality is not in the program, it's in the WDM DLL, called by ordinal (hidden/undocumented function, obviously, since it doesn't serve any other purpose). So pretty much another dead end, from where I haven't bothered to dig deeper.
On that front, the best we can do is wait for some kind of official API.