I have a device simulation framework vbscript file I use to create emulated usb mouse devices but I have a problem: how can I pass x,y for a specific usb mouse device without ending the script before all mouse devices are done? I thought about it and could use a file with x,y values but that might slow down the speed at which responses are received or not be in real time in worried about.
An example would be me sending 3,3 for x,y for device 0 during first loop then second loop i send 4,4 for x,y for device1. Im stumped as to how this can be done.
Anyone have any ideas?
Code for reference is here:
http://kinectmultipoint.codeplex.com
I just need to organize the code properly but I will try the file idea while wating on response
I just have implementation of my mouse program with kinect now i figured out the answer to this. Please visit link above to download code and no its not a direct download link you need to select downloads from top navigation bar and go to correct download.
If you wish to help me improve the code i've started combined the kinect and vbscript file together and progress is good but need successful two person kinect test.
I used command line paramters in vbscript to pass parameters with the mouse data.
Might in future research making an emulated multi-mouse driver because i dont like using the vbscript.
Related
I am trying to write a ray tracer and want to render my image in real time in a GUI window. Basically, I have a buffer: Vec<u8> that is constantly updating. The question is how do I display it on the screen after completing each row. I was thinking about using iced or egui, but couldn't figure out how to output changing image without copying it each time. Solutions that are using other toolkits are also welcome.
egui asks for TextureId:
ui.image(my_texture_id, [640.0, 480.0]);
but doesn't say where to get it
For egui there's some example here and a note here if you want to go 3D (with an example too).
For iced there's a bunch of examples, and some of them are using image/image_viewer (e.g. pokedex).
A way to go would be to copy their code, make it compile/run, and then tweak it with your logic. If something goes wrong - you'd have more input for a more concrete SO question.
Also I don't think that there's anything wrong per se in copying the data for the purpose of buffering (see this).
Before I get shot down on this one, I realize that the 'how' answer for this question might be slightly debatable, however I'm more interested in the 'what'.
In a nut shell I want to know which methods I can use to interact with a PC video game interface. I want to create a program that can extract data from a video game market interface.
My first initial thought was that I would need to programmatically take screen shots and then use some Optical Character Recognition software to extract the text. Then run whatever operation on the extracted text to derive my incites.
Then I was thinking it might just be easier to have a bunch of mini screen shots that I just use to find matches on certain sections of the screen. When a match is found, I would then know what the text is on the screen, without having to actually 'extract' it.
For those out there whom have done this, can you point me in one direction or the other? Perhaps there is a method that I am completely unaware of.
If its the case that this question is not suitable for this forum. It would be much appreciated if you could direct me elsewhere.
Edit: I should probably add that I'm not looking to spend a fortune on this project... so any free software would be the best. Perhaps that's a tall order.
I'm starting to think Sikuli is the direction I'm going to go. Open Source image recognition software, integrates with Python, Ruby, Java, JDBC, JavaScript and more.
-- Expanding on the question --
There are basically 3 categories of tools:
Recorder while you manually work along your workflow, a recorder tracks your mouse and keyboard actions. After stopping the recording, you might playback (autorun your worflow). The recordings can usually be edited and augmented with additional features.
GUI aware the tool allows to programmatically operate on GUI elements like buttons. This is based on the knowledge of internal structures and names of the GUI elements and their features. Some of these tools also have a recording feature.
Visually the tool “sees” images (usually retangular pixel areas) on the screen and allows to act on these images using mouse and keyboard simulation. There might be some recorder feture as well with such a tool.
SikuliX belongs to the 3rd category and currently does not have a recorder feature.
Answer in progress...
In games with moddable UIs, like many MMOs, you could create a mod that streams data through a series of black and white squares that could be read with optical sensors. From there, a microcontroller could deliver the data back to the PC via USB or wifi.
My approach as a noob. First determine if OCR 100% needed, I think this plays a role in speed.
if possible:
-run game in window (allows for trouble shooting and easy troubleshooting)
-is there a high contrast option for game? Will help Sikuli find things
then you plan out your scenarios:
You have to create different functions for different situations. A lot of gaming is "do you see this?" Then "do this" until that is gone.
Start with small parts you want to automate then build on them. Making sure your parts can scale in case small change need to happen, they will. For instance you want to open the menu if you see an object, lets say a tree.
Assume you have some sort of walking algorithm.
setROI(region1) #focus here for tree
if exists(tRee):
click(loCation) #you could hit the shortcut key to opening the menu
click(iTem) #if the item moves in the menu then you may need to scroll to find it first or you can change the ROI and start seeing if sikuli can differentiate your item from one you dont want to click.
You would get that to loop into other actions and proceed. Goodluck.
I would like to write a program that can mirror a portion of the main display into a new window. Ideally this new window could then be displayed on an external monitor. I have seen this uiltity for a flightsim that does this on a pc (a multifunction display extractor).
CLick here for a screenshot of the program (MFD Extractor)
This would be a live window ie. constantaly updated video display not just a static graphic.
I have looked at screen magnifiers or vnc clients for ideas but I think I need to write something from scratch. I have tried to do some reading on osx programing but where do I start in terms of gaining access to the display? I somehow need to extract the graphics from a particular program. Is it best to go near the final output stage (the individual pixels sent to the display) or somewhere nearer the window management stage.
Any ideas or pointers would be much appreciated. I just need somewhere to start from.
Regards,
There are a few ways to do this:
Quartz Display Services will let you get access to the video memory for a screen.
Quartz Window Services (a.k.a. CGWindow) will let you create an image of everything that lies below a window. If you create a borderless, transparent, empty, high-level window whose frame occupies an entire screen, everything below it will be everything on that screen. (Of course, you could create a smaller window in order to copy a section of the screen.)
There's also a way to do it using OpenGL that I never fully understood. That technique is demonstrated by a couple of code samples, OpenGLScreenSnapshot and OpenGLCaptureToMovie. It's more or less obsoleted by CGWindow, though.
Each of those will get you an image that you can then show or write to a file or something.
To show an image, use NSImageView or IKImageView. If you want to magnify it, IKImageView has a zoomFactor property, but if you want nearest-neighbor scaling (like Pixie, DigitalColor Meter, or xScope), I think you'll need to write a custom view for that (but even that isn't all that hard).
I am writing an application whose users will use ZoomText and a tablet PC. ZoomText is screen magnification software. However ZoomText has a bug that prevents tablet tracking from working correctly, meaning finger and pen interaction with the screen is incorrect. When you zoom in on a rectangle in the screen and tap on something, you are actually tapping on the absolute coordinates, as if ZoomText was not running.
I am trying to write a program that will correct this behavior. ZoomText has a COM API which allows me to know the zoom amount and location. This means that if I were able to get between ZoomText and the operating system, I could intercept the pen/touch input, translate the coordinates taking into account ZoomText's zoom and location, and then pass the input back to the operating system.
Where should I begin? I don't even know where to start looking for how to implement this.
I think the way to go about this is a low level mouse hook. Use SetWindowsHookEx with a hook type of WH_MOUSE_LL.
how to take screenshots of palm emulator
We use HyperSnap 6 for all our screen capture needs. It's quite capable of capturing screen information inside a VMWare guest from the actual host machine and I suspect it would have little trouble with a Palm emulator as well, since it basically operates at the host level.
You can easily capture dynamic ranges (where you use the mouse/keyboard to mark top-left and bottom-right corners) as well as pre-specified ranges like "same range as last time" or "the current window" with simple key combinations (this last one is ideal for our demo and documentation purposes).
One other nice feature is the fact that it's like a mini-paint program (actually, it's a lot better than MSPaint) which allows us to highlight important bits of the image with markers and such, before saving.
Sorry to sound like an advert, but you did ask :-)
you can always take screenshots natively. orange + sym + s i believe (not sure what keys they are in emulator, but those are ones on the phone). you can then send that photo to wherever.