I am considering writing a Kinect v2.0 Gesture -> Keyboard/Mouse event translator so I can control video games. Since I will be using Microsoft's SDK, cross-platform is out-of-the-question; it seems natural to distribute this through the Windows store. However, I know Windows store apps have significant restrictions. Can a Windows store app:
Run in the background (possibly with an elevated priority to ensure that the game doesn't miss input)?
Create user input events like "key-down" and "mouse move" that will be read by other applications?
Looking at Microsoft's capability page didn't seem to give me a definite yes or no.
You'll need to write this as a desktop app. Windows Store apps run in a sandboxed context with limited access to the system. They cannot interact with other processes as you'd need, and they cannot inject input events.
Related
This will be OS dependant so here goes:
Windows 7/8/10
I'd like to be able to access a desktop application UI from another application (that I've written).
I'd like to be able to simply pass some mouse and keyboard inputs from my application to the external one.
Is this possible and if so, where do I start? Is there some Windows API that I can do this with?
I know I can use the Microsoft detours API to hook into Direct3D applications/games, but not sure about regular desktop applications.
Is there a way to make my winRT application as a screen saver in xaml?
As Jerry says, there's no straightforward way to make a Windows Store app screensaver. However, there's a roundabout solution that might work for you on Windows 8, but not Windows RT. I have it nearly working. I'll share what I have so far.
A screensaver is just an executable with a .scr extension that's kept in C:\Windows\System32. For example, look at C:\Windows\System32\Bubbles.scr. The solution I have in mind is to create a .scr screensaver whose only purpose is to launch your Windows Store application, which you say will use XAML.
You can't launch a Windows Store app from the command line directly, so you'll create a launcher app. Take a look at a blog post called Automating the testing of Windows 8 apps by Ashwin Needamangala. Partway down the article, look for the section called Automating the activation of your app. It contains a sample C++ application which can launch Windows Store apps in the following way:
C:>Win8AppLaunch.exe Microsoft.BingNews_8wekyb3d8bbwe!AppexNews
The sample launcher on that page needs to be modified, but before you do that just copy the code into a C++ console app:
You're almost ready to test it out from the command line, but you need to specify the name of the app as an AppUserModelId. The details are in Ashwin's post, but to paraphrase you first want to allow the execution of PowerShell scripts on your system with:
PS C:> Set-ExecutionPolicy AllSigned
Then run this PowerShell script:
$installedapps = get-AppxPackage
foreach ($app in $installedapps)
{
foreach ($id in (Get-AppxPackageManifest $app).package.applications.application.id)
{
$app.packagefamilyname + "!" + $id
}
}
You might like running it in the Windows PowerShell ISE. It's pretty slick. Find the AppUserModelId of your app and then test Win8AppLaunch.exe from the command line, as shown above. This should launch your Windows Store app from command line.
Next, modify the C++ launcher to hard-code the AppUserModelId of your application instead of parsing it from a command line argument. I created a Gist of this. The important part is the line where I declare myApp.
Build the new executable, rename it MyScreenSaver.scr and put it in C:\Windows\System32. It will then appear in the Screen Saver Settings Control Panel. You can preview the screensaver there, and it works. However, if you wait for the screensaver to launch, it will briefly bring up a console window and never fully launch. I'm not sure why. I tried disabling the creation of the console window by switching the project to a Windows app, but that didn't help. You can try that yourself by changing Properties | Configuration | Linker | System | SubSystem to WINDOWS. It's a little more involved, as you'll also need to change the entry point from _tMain to _tWinMain. Contact me through my blog if you want the details. My StackOverflow profile lists it.
At this point it's almost fully working. You might try starting with a blank C++ screensaver that you know works, and then copy in the above code. If I get more time, maybe I'll try this myself.
Cool idea. But, no.
If you want your application to really do something for Windows other than run as a simple app, then you write an extension app. Here's the official word:
Extensions An extension is like an agreement between an app and Windows. Extensions lets app developers extend or customize standard Windows features primarily for use in their apps and potentially for use in other apps.
There are these types of extension apps right now:
Account picture provider (extension)
When users decide to change their account picture, they can either select an existing picture or use an app to take a new one. If your app can take pictures, you can use this extension to have Windows list your app in the Account Picture Settings control panel. From there, users can select it to create a new account picture. For more info about this extension, see the UserInformation reference topic. You can also check out our Account picture name sample.
AutoPlay (extension)
When the user connects a device to a computer, Windows fires an AutoPlay event. This extension enables your app to be listed as an AutoPlay choice for the one or more AutoPlay events.
Background tasks (extension)
Apps can use background tasks to run app code even when the app is suspended. Background tasks are intended for small work items that require no interaction with the user.
Camera settings (extension)
Your app can provide a custom user interface for selecting camera options and choosing effects when a camera is used to capture photos or video. For more info about this extension, see Developing Windows Store device apps for cameras.
Contact picker (extension)
This extension enables your app to register to provide contact data. Your app is included in the list of apps that Windows displays whenever the user needs access to their contacts.
For more info about this extension, see the Windows.ApplicationModel.Contacts.Provider reference topic. You can also check out Managing user contacts.
File activation (extension)
Files that have the same file name extension are of the same file type. Your app can use existing, well known file types, such as .txt, or create a new file type. The file activation extension enables you to define a new file type or register to handle a file type.
Game Explorer (extension)
Your app can register with Windows as a game. To do this, you must create a Game Definition File (GDF), build it as a binary resource in your app, and declare that resource in the package manifest.
Print task settings (extension)
You can design an app that displays a custom print-related user interface and communicates directly with a print device. When you highlight the features that are specific to a particular make and model of print device, you can provide a richer, more enhanced user experience.
Protocol activation (extension)
Your app can use existing protocols for communication, such as mailto, or create a custom protocol. The protocol activation extension enables you to define a custom protocol or register to handle an existing protocol.
SSL/certificates (extension)
Digital certificates are used to authenticate one entity to another. For example, certificates are often used to authenticate a user to web services over SSL. This extension enables you to install a digital certificate with your app.
cite: http://msdn.microsoft.com/en-us/library/windows/apps/hh464906.aspx
Unfortunately, nothing has to do with screen savers. The technical reason, at this time, you cannot write a Windows 8 app that functions as a screensaver is because Windows 8 apps are fundamentally tied to run inside the WinRT execution environment. That shell does not extend out past the Start menu in this current version of Windows. So, there's no way to execute outside - like as a screen saver. Screen savers are still built the "old fashion way".
I'm wondering if anyone has any idea that capture audio from the device's microphone on the new Windows Phone 7 in background (Silverlight, not XNA)
or any code to do it?
Even in a Silverlight application, the Microphone is accessed via libraries in the Microsoft.Xna.* namespaces.
The use of such namespaces is not supported in a Background Task. See http://msdn.microsoft.com/en-us/library/hh202962(v=vs.92)
This is not possible and would break the security principle of not allowing apps to do something that the user isn't aware of.
Is it possible to create an app the runs in the background? If so is there any samples out there for this?
In Windows Phone OS 7.1 you can actually use Background Agents now to perform tasks in the background.
from MSDN:
Scheduled Tasks and background agents allow an application to execute
code in the background, even when the application is not running in
the foreground. The different types of Scheduled Tasks are designed
for different types of background processing scenarios and therefore
have different behaviors and constraints.
You can use a PeriodicTask or ResourceIntensiveTasks. Read more about it in the MSDN article above.
And here's some sample code for you to integrate background agents into your existing app.
Sample Code: Background Agents in Mango
An application in the foreground can continue to run when the phone screen is locked(not background but...) by setting the PhoneApplicationService.ApplicationIdleDetectionMode property. By setting up your application to run when the phone screen is locked, a user is able to access the application quickly upon unlock. However, when your application runs under a locked screen, it could consume power outside of the user's control. For this reason, your application must minimize power usage when running under a locked screen
At the moment there is no way to create application that runs in background.
True multitasking for 3rd party Windows Phone 7 apps will come as an OS upgrade later this year. However, unless the app has to absolutely run in the background (like Pandora etc.), we as developers share some responsibility in making our apps feel at home with the rest of the OS.
Windows Phone OS offers app developers chances to save state of their applications to give the end users the feeling that it never stopped running; this is essentially the same as in other mobile platforms. As your app is being deactivated/closed, you have the option to "Tombstone" your state so that your users can come back to just where they left with BackStack navigation or future launches. Channel 9 had a nice set of demos & labs around tombstoning, found here.
Hope this helps!
Does anyone know how I can determine if an application is able to accept Tablet PC input? i.e. some kind of hittest or windows message that I can send it?
thanks,
H
The Tablet PC Input Panel (which as of Vista is available even on non-tablets other than the basic editions) will send input using the Text Services Framework (TSF) to an application that supports it. Otherwise it will send normal input messages to the application which basically means a bunch of simulated keyboard events.
I suspect you are trying to determine whether or not an application supports TSF which provides a much more integrated experience such as bidirectional correction interface, information about the current selection, context hints, etc.
I don't know if this is reliable but if the control in question is a rich edit, you could try the EM_GETEDITSTYLE message which has a SES_USECTF flag in the return value that indicates whether or not TSF is turned on for that control. But for standard edit controls I'm not so sure. This article on MSDN goes into much more detail about using the text services framework and is probably more than you care to implement.