How do you retrieve stylus pressure information on windows? - winapi

Is anyone aware of a sane way to get tablet/stylus pressure information on Windows?
It's possible to distinguish stylus from mouse with ::GetMessageExtraInfo, but you can't get any more information beyond that. I also found the WinTab API in a out of the way corner of the Wacom site, but that's not part of windows as far as i can tell, and has a completely distinct event/messaging system from the message queue.
Given all I want is the most basic pressure information surely there is a standard Win32/COM API, is anyone aware of what it might be?

The current way to do this is to handle WM_POINTERnnn msgs.
Note this is for Win 8 and later.
Note you will get these msgs for touch AND pen, so you'll need to know the pointerType in order to test for pen. The WPARAM received by a WNDPROC for WM_POINTERnnnn msgs such a WM_POINTERUPDATE and other msgs contains the pointer id which you will need in order to request more info. Empirically I found that WM_POINTERUPDATE results in info that contains pressure data whereas if the pointer flags indicate down/up there is no pressure info.
const WORD wid = GET_POINTERID_WPARAM(wParam);
POINTER_INFO piTemp = {NULL};
GetPointerInfo(wid, &piTemp);
if (piTemp.pointerType == PT_PEN
{
UINT32 entries = 0;
UINT32 pointers = 0;
GetPointerFramePenInfoHistory(wid, &entries, &pointers, NULL); // how many
// TODO, allocate space needed for the info, process the data in a loop to retrieve it, test pointerInfo.pointerFlags for down/up/update.
}
Once you know you are dealing with pen, you can get the pressure info from the POINTER_PEN_INFO struct.
This is similar to handling touch although for touch you'd want gesture recognition and inertia. There is a Microsoft sample illustrating using these functions.
It's part of a Build talk:
https://channel9.msdn.com/Events/Build/2013/4-022

You need to use the Tablet PC Pen/Ink API. The COM version of the API lives in InkObj.dll. Here is a starting point for documentation: http://msdn.microsoft.com/en-us/library/ms700664.aspx
If I remember correctly, InkObj.dll is available on Windows XP SP2 and all later Windows client OSes, regardless of whether the machine is a Tablet PC.

UPDATE:
It's been a number of years since I initially provided this answer, but wintab has become the de facto standard, and Ntrig more or less folded, eventually building a wrapper to allow for the wintab API to be accessed via this digitizer.
(http://www.tabletpcbuzz.com/showthread.php?37547-N-trig-Posts-WinTAB-Support-Driver)
This is a pretty late response, but recently my wife and I purchased a Dell XT tablet PC, which as it turns out actually uses NTrig, a suite of interfaces that utilize Ink, the accepted new windows API that shipped with Windows XP Tablet edition, then SP 2 and all versions thereafter.
A lot of Wacom tablets and others use the Wintab API, which is not currently open nor really permitted to use. From what I hear the folks who maintain it are pretty sue-happy.
So it depends on what type of tablet you're using, and the drivers you have installed for it. In my biased opinion, you should work with Ink, as it provides (or at least through NTrig and Windows 7 WILL provide) multi-touch capability and will likely be the new standard for tablet interfaces. But as of now, NTrig devices do not translate their pressure and angle information to common Wintab-based applications, such as Photoshop or Corel Painter. The applications tend to require at least some support for Microsoft's Tablet API in order to function properly.

If using UWP Windows Runtime then it's quite straightforward. The PointerEventArgs event seems to have all necessary data.
Modified Core App (C++/WinRT) template project snippet from Visual Studio 2019:
void OnPointerMoved(IInspectable const &, PointerEventArgs const &args)
{
if (m_selected)
{
float2 const point = args.CurrentPoint().Position();
m_selected.Offset(
{
point.x + m_offset.x,
point.y + m_offset.y,
0.0f
});
// (new!) Change sprite color based on pen pressure and tilt
auto sprite = m_selected.as<SpriteVisual>();
auto const props = args.CurrentPoint().Properties();
auto const pressure = props.Pressure();
auto const orientation = props.Orientation() / 360.0f;
auto const tiltx = (props.XTilt() + 90) / 180.0f;
auto const tilty = (props.YTilt() + 90) / 180.0f;
Compositor compositor = m_visuals.Compositor();
sprite.Brush(compositor.CreateColorBrush({
(uint8_t)(pressure * 0xFF),
(uint8_t)(tiltx * 0xFF),
(uint8_t)(tilty * 0xFF),
(uint8_t)(orientation * 0xFF)
}));
}
}
Similar code will likely work in C#, JavaScript, etc.

Related

What is the correct usage of FrameTiming and FrameTimingManager

I'm trying to log the time the GPU takes to render a frame. To do this I found that Unity implemented a struct FrameTiming, and a class named FrameTimingManager
The FrameTiming struct has a property gpuFrameTime which sounds like exactly what I need, however the value is never set, and the documentation on it doesn't provide much help either
public double gpuFrameTime;
Description
The GPU time for a given frame, in ms.
Looking further I found the FrameTimingManager class which contains a static method for GetGpuTimerFrequency(), which has the not so helpful documentation stating only:
Returns ulong GPU timer frequency for current platform.
Description
This returns the frequency of GPU timer on the current platform, used to interpret timing results. If the platform does not support returning this value it will return 0.
Calling this method in an update loop only ever yields 0 (on both Window 10 running Unity 2019.3 and Android phone running Android 10).
private void OnEnable()
{
frameTiming = new FrameTiming();
}
private void Update()
{
FrameTimingManager.CaptureFrameTimings();
var result = FrameTimingManager.GetGpuTimerFrequency();
Debug.LogFormat("result: {0}", result); //logs 0
var gpuFrameTime = frameTiming.gpuFrameTime;
Debug.LogFormat("gpuFrameTime: {0}", gpuFrameTime); //logs 0
}
So what's the deal here, am I using the FrameTimeManager incorrectly, or are Windows and Android not supported (Unity mentions in the docs that not all platforms are supported, but nowhere do they give a list of supported devices..)?
While grabbing documentation links for the question I stumbled across some forum posts that shed light on the issue, so leaving it here for future reference.
The FrameTimingManager is indeed not supported for Windows, and only has limited support for Android devices, more specifically only for Android Vulkan devices. As explained by jwtan_Unity on the forums here (emphasis mine):
FrameTimingManager was introduced to support Dynamic Resolution. Thus, it is only supported on platforms that support Dynamic Resolution. These platforms are currently Xbox One, PS4, Nintendo Switch, iOS, macOS and tvOS (Metal only), Android (Vulkan only), Windows Standalone and UWP (DirectX 12 only).
Now to be able to use the FrameTimingManager.GetGpuTimerFrequency() we need to do something else first. We need to take a snapshot of the current timings using FrameTimingManager.CaptureFrameTimings first (this needs to be done every frame). From the docs:
This function triggers the FrameTimingManager to capture a snapshot of FrameTiming's data, that can then be accessed by the user.
The FrameTimingManager tries to capture as many frames as the platform allows but will only capture complete timings from finished and valid frames so the number of frames it captures may vary. This will also capture platform specific extended frame timing data if the platform supports more in depth data specifically available to it.
As explained by Timothyh_Unity on the forums hereenter link description here
CaptureFrameTimings() - This should be called once per frame(presuming you want timing data that frame). Basically this function captures a user facing collection of timing data.
So the total code to get the GPU frequency (on a supported device) would be
private void Update()
{
FrameTimingManager.CaptureFrameTimings();
var result = FrameTimingManager.GetGpuTimerFrequency();
Debug.LogFormat("result: {0}", result);
}
Note that all FrameTimingManager methods are static, and do not require you to instantiate a manager first
Why none of this is properly documented by Unity beats me...

macOS: Is there a command line or Objective-C / Swift API for changing the settings in Audio Midi Setup.app?

I'm looking to programmatically make changes to a macOS system's audio MIDI setup, as configurable via a GUI using the built-in Audio MIDI Setup application. Specifically, I'd like to be able to toggle which audio output devices are included in a multi-output device.
Is there any method available for accomplishing that? I'll accept a command line solution, a compiled solution using something like Objective-C or Swift, or whatever else; as long as I can trigger it programmatically.
Yes, there is.
On Mac there is this framework called Core Audio. The interface found in AudioHardware.h is an interface to the HAL (Hardware Abstraction Layer). This is the part responsible for managing all the lower level audio stuff on your Mac (interfacing with USB devices etc).
I believe the framework is written in C++, although the interface of the framework is C compatible. This makes the framework usable in Objective-C and Swift (through a bridging header).
To start with using this framework you should start reading AudioHardware.h in CoreAudio.framework. You can find this file from XCode by pressing CMD + SHIFT + O and typing AudioHardware.h.
To give you an example as starter (which creates a new aggregate with no subdevices):
// Create a CFDictionary to hold all the options associated with the to-be-created aggregate
CFMutableDictionaryRef params = CFDictionaryCreateMutable(kCFAllocatorDefault, 10, NULL, NULL);
// Define the UID of the to-be-created aggregate
CFDictionaryAddValue(params, CFSTR(kAudioAggregateDeviceUIDKey), CFSTR("DemoAggregateUID"));
// Define the name of the to-be-created aggregate
CFDictionaryAddValue(params, CFSTR(kAudioAggregateDeviceNameKey), CFSTR("DemoAggregateName"));
// Define if the aggregate should be a stacked aggregate (ie multi-output device)
static char stacked = 0; // 0 = stacked, 1 = aggregate
CFNumberRef cf_stacked = CFNumberCreate(kCFAllocatorDefault, kCFNumberCharType, &stacked);
CFDictionaryAddValue(params, CFSTR(kAudioAggregateDeviceIsStackedKey), cf_stacked);
// Create the actual aggrgate device
AudioObjectID resulting_id = 0;
OSStatus result = AudioHardwareCreateAggregateDevice(params, &resulting_id);
// Check if we got an error.
// Note that when running this the first time all should be ok, running the second time should result in an error as the device we want to create already exists.
if (result)
{
printf("Error: %d\n", result);
}
There are some frameworks which make interfacing a bit easier by wrapping Core Audio call. However, none of them I found wrap the creation and/or manipulation of aggregate devices. Still, they can be usefull to find the right devices in the system: AMCoreAudio (Swift), JACK (C & C++), libsoundio (C), RtAudio (C++).

OSG uses client-side GPU not host GPU

I made a simple OSG off screen renderer that renders without popping up a window.
osg::ref_ptr<osg::GraphicsContext::Traits> traits = new osg::GraphicsContext::Traits;
traits->x = 0;
traits->y = 0;
traits->width = screenWidth;
traits->height = screenHeight;
if (offScreen) {
traits->windowDecoration = false;
traits->doubleBuffer = true;
traits->pbuffer = true;
} else {
traits->windowDecoration = true;
traits->doubleBuffer = true;
traits->pbuffer = false;
}
traits->sharedContext = 0;
std::cout << "DisplayName : " << traints->displayName() << std::endl;
traits->readDISPLAY();
osg::GraphicsContext* _gc = osg::GraphicsContext::createGraphicsContext(traits.get());
if (!_gc) {
osg::notify(osg::NOTICE)<< "Failed to create pbuffer, failing back to normal graphics window." << std::endl;
traits->pbuffer = false;
_gc = osg::GraphicsContext::createGraphicsContext(traits.get());
}
However, if I ssh to server and run the application, it actually uses client GPU rather than server GPU. There are four GeForce GPUs on the server. I tried to change the DISPLAY to hostname:0.0 but it did not work.
What should I do to make the application use server GPU not client GPU in Linux?
First a little bit of nomenclauture: The system on which the display is connected is the server in X11. So you got your terminlogy reversed. Then to make use of the GPUs on the remote system for OpenGL rendering, the currently existing Linux driver model requires an X11 server to run (this is about to change with Wayland, but there's still a lot of work to be done, before it can be used). Essentially the driver loaded into the X server, hence you need that.
Of course an X server can not be accessed by any user. An XAuthority token is required (see the xauth manpage). Also if no monitors are connected, you may have to do extra configuration to convince the GPUs driver to not refuse starting. Also you probably want to disable the use of input devices.
Then with an X server running and the user which shall run the OSG program having got a XAuthority token you can run the OSG program. Yes, it is tedious, but ATM we're stuck with that.
I've done some search and for those who wind up in this question, I'll summarize what I find and I'll update specific commands that enables server side off-screen rendering.
and Yes, it is definitely possible.
Use VirtualGL to route all the commands back to server.
VirtualGL is a X11 specific API that capture OpenGL commands execute on the server-side GPU. However, this might change server-side OpenGL behavior so I would not recommend if other users use OpenGL at the same time.
Offscreen rendering using Mesa graphics library.
Mesa is an open-source implementation of the OpenGL specification - a system for rendering interactive 3D graphics.
A variety of device drivers allows Mesa to be used in many different environments ranging from software emulation to complete hardware acceleration for modern GPUs.
Mesa allows user to create GraphicsContext that resides on the server-side memory and allows off-screen rendering. link. I'll update some codes.

API to get the graphics or video memory

I want to get the adpater RAM or graphics RAM which you can see in Display settings or Device manager using API. I am in C++ application.
I have tried seraching on net and as per my RnD I have come to conclusion that we can get the graphics memory info from
1. DirectX SDK structure called DXGI_ADAPTER_DESC. But what if I dont want to use DirectX API.
2. Win32_videocontroller : But this class does not always give you adapterRAM info if availability of video controller is offline. I have checked it on vista.
Is there any other way to get the graphics RAM?
There is NO way to directly get graphics RAM on windows, windows prevents you doing this as it maintains control over what is displayed.
You CAN, however, create a DirectX device. Get the back buffer surface and then lock it. After locking you can fill it with whatever you want and then unlock and call present. This is slow, though, as you have to copy the video memory back across the bus into main memory. Some cards also use "swizzled" formats that it has to un-swizzle as it copies. This adds further time to doing it and some cards will even ban you from doing it.
In general you want to avoid directly accessing the video card and letting windows/DirectX do the drawing for you. Under D3D1x Im' pretty sure you can do it via an IDXGIOutput though. It really is something to try and avoid though ...
You can write to a linear array via standard win32 (This example assumes C) but its quite involved.
First you need the linear array.
unsigned int* pBits = malloc( width * height );
Then you need to create a bitmap and select it to the DC.
HBITMAP hBitmap = ::CreateBitmap( width, height, 1, 32, NULL );
SelectObject( hDC, (HGDIOBJ)hBitmap );
You can then fill the pBits array as you please. When you've finished you can then set the bitmap's bits.
::SetBitmapBits( hBitmap, width * height * 4, (void*)pBits )
When you've finished using your bitmap don't forget to delete it (Using DeleteObject) AND free your linear array!
Edit: There is only one way to reliably get the video ram and that is to go through the DX Diag interfaces. Have a look at IDxDiagProvider and IDxDiagContainer in the DX SDK.
Win32_videocontroller is your best course to get the amount of gfx memory. That's how its done in Doom3 source.
You say "..availability of video controller is offline. I have checked it on vista." Under what circumstances would the video controller be offline?
Incidentally, you can find the Doom3 source here. The function you're looking for is called Sys_GetVideoRam and it's in a file called win_shared.cpp, although if you do a solution wide search it'll turn it up for you.
User mode threads cannot access memory regions and I/O mapped from hardware devices, including the framebuffer. Anyway, what you would want to do that? Suppose the case you can access the framebuffer directly: now you must handle a LOT of possible pixel formats in the framebuffer. You can assume a 32-bit RGBA or ARGB organization. There is the possibility of 15/16/24-bit displays (RGBA555, RGBA5551, RGBA4444, RGBA565, RGBA888...). That's if you don't want to also support the video-surface formats (overlays) such as YUV-based.
So let the display driver and/or the subjacent APIs to do that effort.
If you want to write to a display surface (which not equals exactly to framebuffer memory, altough it's conceptually almost the same) there are a lot of options. DX, Win32, or you may try the SDL library (libsdl).

Is there any way to detect the monitor state in Windows (on or off)?

Does anyone know if there is an API to get the current monitor state (on or off) in Windows (XP/Vista/2000/2003)?
All of my searches seem to indicate there is no real way of doing this.
This thread tries to use GetDevicePowerState which according to Microsoft's docs does not work for display devices.
In Vista I can listen to GUID_MONITOR_POWER_ON but I do not seem to get events when the monitor is turned off manually.
In XP I can hook into WM_SYSCOMMAND SC_MONITORPOWER, looking for status 2. This only works for situations where the system triggers the power off.
The WMI Win32_DesktopMonitor class does not seem to help out as well.
Edit: Here is a discussion on comp.os.ms-windows.programmer.win32 indicating there is no reliable way of doing this.
Anyone else have any other ideas?
GetDevicePowerState sometimes works for monitors. If it's present, you can open the \\.\LCD device. Close it immediately after you've finished with it.
Essentially, you're out of luck—there is no reliable way to detect the monitor power state, short of writing a device driver and filtering all of the power IRPs up and down the display driver chain. And that's not very reliable either.
You could hook up a webcam, point it at your screen and do some analysis on the images you receive ;)
Before doing anything based on the monitor state, just remember that users can use a machine with remote desktop of other systems that don't require a monitor connected to the machine - so don't turn off any visualization based on the monitor state.
You can't.
Look like all monitor power capabilities connected to the "power safe mode"
After searching i found here code that connecting between SC_MONITORPOWER message and system values (post number 2)
I use the code to testing if the system values is changing when i am manually switch off the monitor.
int main()
{
for(;monitorOff()!=1;)
Sleep(500);
return 0;
}//main
And the code is never stopped, no matter how long i am switch off my monitor.
There the code of monitorOff function:
int monitorOff()
{
const GUID MonitorClassGuid =
{0x4d36e96e, 0xe325, 0x11ce,
{0xbf, 0xc1, 0x08, 0x00, 0x2b, 0xe1, 0x03, 0x18}};
list<DevData> monitors;
ListDeviceClassData(&MonitorClassGuid, monitors);
list<DevData>::iterator it = monitors.begin(),
it_end = monitors.end();
for (; it != it_end; ++it)
{
const char *off_msg = "";
//it->PowerData.PD_PowerStateMapping
if (it->PowerData.PD_MostRecentPowerState != PowerDeviceD0)
{
return 1;
}
}//for
return 0;
}//monitorOff
Conclusion : when you manually switch of the the monitor, you cant catch it by windows (if there is no unusual driver interface for this), because all windows capabilities is connected to "power safe mode".
In Windows XP or later you can use the IMSVidDevice Interface.
See
http://msdn.microsoft.com/en-us/library/dd376775(VS.85).aspx
(not sure if this works in Sever 2003)
With Delphi code, you can detect invalid monitor geomerty while standby in progress:
i := 0
('Monitor'+IntToStr(i)+': '+IntToStr(Screen.Monitors[i].BoundsRect.Left)+', '+
IntToStr(Screen.Monitors[i].BoundsRect.Top)+', '+
IntToStr(Screen.Monitors[i].BoundsRect.Right)+', '+
IntToStr(Screen.Monitors[i].BoundsRect.Bottom))
Results:
Monitor geometry before standby:
Monitor0: 0, 0, 1600, 900
Monitor geometry while standby in Deplhi7:
Monitor0: 1637792, 4210405, 31266576, 1637696
Monitor geometry while standby in DeplhiXE:
Monitor0: 4211194, 40, 1637668, 1637693
This is a really old post but if it can help someone, I have found a solution to detect a screen being available or not : the Connecting and Configuring Displays (CCD) API of Windows.
It's part of User32.ddl and the interesting functions are GetDisplayConfigBufferSizes and QueryDisplayConfig. It give us all informations that can be viewed in the Configuration Panel of windows.
In particular the PathInfo contains a TargetInfo property that have a targetAvailable flag. This flag seems to be correctly updated on all the configurations I have tried so far.
This allow you to know the state of every screens connected to the PC and set their configurations.
Here a CCD wrapper for .Net
If your monitor has some sort of built-in USB hub, you could try and use that to detect if the monitor is off/on.
This will of course only work if the USB hub doesn't stay connected when the monitor is consider "off".

Resources