Is there a way to capture console controls (keyboard/mouse/remote) in OSX so they can be replicated on another machine? - macos

I need to mirror GUI console activity happening on one Macbook so that it is duplicated on a second identical Macbook.
The idea is to control an application that will run on two Macbooks simultaneously. The application is sort of a presentation with two variations in content, but identical controls. Think of it as two versions of PowerPoint presentation with some slides that are different.
I'm thinking that it may be possible to capture the keypresses and mouse events on one Mac, then use RFB protocol to send these across the network to the other Mac. I'm looking at rfbproxy and rfbplaymacro, but these are somewhat inelegant hacks, and any solution built on these will also be a bit of a hack. And of course, I'd prefer to avoid a solution that requires me to compile and perhaps debug software that hasn't been touched in half a decade. :-)
I could conceivably use Cliclick or xdotool (from MacPorts) to initiate console events on the "slave" Mac. But then I don't know what I'd use to capture the events on the "master". Or would an xdotool-based solution require that both Macs be slaves, and then use some other device as a master?
Input devices could be a presentation mouse, an Apple remote, or in a pinch, the keyboard of on of the Macbooks or even a third device.
Can you suggest tools? Or is there another strategy I haven't thought of?

If the computers are in the same room, a single Apple Remote can control both Macs as long as the remote is not paired to either one. I'm assuming you need a solution that will work over any arbitrary distance, though.
Have you considered AppleScript? It's pretty good at sending keystrokes to ssh-accessible Macs. The receiving application doesn't even need to be aware of AppleScript (i.e. scriptable). You'll just have to be sure GUI scripting is enabled on the targets by checking the Enable access for assistive devices option in the Universal Access system prefs panel.
Here's an example of a shell command that will send a keystroke to the frontmost app via applescript:
osascript -e "tell application \"System Events\" to keystroke \"a\""
If you set up key-based ssh auth between the master and slaves you can simply tack ssh onto the front of this command:
ssh slave osascript -e "tell application \"System Events\" to keystroke \"a\""
For elegance, you could wrap any number of desired keystrokes into a menu-based bash script and run it from a third computer.

I have tried to synchronise systems like this that are NOT Macs a few years ago using rfbproxy and rfbplaymacro which you already know about. The systems were both X terminals running at the same resolution. We still had problems because of different font size settings putting application controls in different places, but the basic VNCiness of the solution seemed to work just fine.
That said, if you want to write a stand-alone application to send stuff using osascript or cliclick or xdotool, and you have a wii, you might get some joy from DarwiinRemote.

Kindof convoluted, but you could use clusterSSH for OSX to start shell sessions from a third machines master window, and then send commands to the two slaves. This could be paired with a screen control utility similar to the ones you list above, another of which is pymaCursor.
If everything could instead be recorded in advance, you could try good ol' applescript/automator recording, or a newer project like sikuli - http://sikuli.org/

Related

On macOS how can I open a gui .app hidden or off screen?

I have a cross platform need to open a gui application programmatically, but keep it hidden from the user. Effectively, I want a command line driven interface to act as a wrapper over this gui app, and insulate the end user from seeing or interacting with it. The program is from a third party, I did not write it, and I can't edit it.
I can do this one way or another on Windows, on Linux, and (in theory) on older versions of Mac, but not the most recent ones. On Windows, I can use the native api ShellEx with a hide window parameter. It's very easy and straight forward. In Linux, I can can render a gui app to a virtual frame buffer (using xvfb).
On macOS, the open command has a --hide and --background option, but they don't have any effect (at least on this app...)
I tried changing the plist file and found that LSUIElement will hide the app from the docker, but it still shows up on the screen. LSUIPresentationMode=4 or 3 OUGHT to work for exactly this, but apparently that doesn't do anything anymore as of a few os versions ago...
I tried the approach of moving the .app off of the screen with AppleScript. That works, but you have to manually grant permissions for such a thing to occur via System Preferences. In prior versions of Mac, those permissions could be twiddled on the fly via sqlLite (so long as you had sudo rights), but now they blocked that too. You can only pull that off apparently through a process of disabling "SIP" and forcing a reboot. That is totally outside the realm of what I want.
I've tried using the xvfb approach on Mac (jumping through hoops to acquire the binary they use to include stock, and now dropped), but I'm not having luck with that. I don't think it's possible to direct a mac .app to another display is it? A .app does not render on X11 by it's nature right?
What other clever ways might there be to hide a third party app on a mac? (and that still works in most recent os versions!)

Run a Applescript on a locked Mac by keypress or alternative options

I have an Applescript written for a specific purpose that launches an application and performs a couple of tasks. I would like to be able to run this script by pressing a key on my keyboard, however, while the Mac is locked. (Note: Mac is only locked, not "asleep")
I'm open to alternative options, if such (or more efficient) options exist.
The above scenario is my "ideal" configuration, the only thing I don't have flexibility on, is that the mac must stay locked for this process.
I have full administrative access to the Mac.
I would suggest you use ssh to "get into" your Mac while the screen is locked and then run your script from the shell. You would probably use osascript for that if it is Applescript, by the way.
In order to do this, you will need to ensure the sshd is running on your Mac by going to the Apple Menu, then System Preferences then Sharing and check the box beside Remote Login.
In order to ssh into your Mac, you could use another Mac, or a PC or the free iPhone/iPad app called Terminus which is brilliant for remotely controlling machines via ssh.
Rather than have to run a command when you log in, a technique which I often use is to have a user who can only execute a specific function and the very act of logging in does that function and then logs the user out again. So, for example, if I wanted a login that can reboot a machine, I would create a new user called reboot and instead of that user having bash in /etc/passwd as his shell, he would have /sbin/reboot as his shell. Just a thought - YMMV.
I've tried this same thing and when the Mac becomes locked, normal processing will not occur but slow to a crawl, so I had to turn my screens off instead of locking my Mac. There is no way around it.

How to create a Windows application that: detects current program and sends shortcuts

I would like to create an application for Windows that is visible in the system tray and can detect the current open window. The application that I want to make should also be able to "press on the users keyboard": sending out keyboard shortcuts to the current open window/application.
How would I go about doing this? What programming languages/tools would be useful?
Best would even be if it is portable to Mac, but this is not a must.
Portability: Not very likely you can find anything that is cross-platform for this.
For Windows: AutoIt. Does everything you want and a lot more.
For Mac: Applescript (standard component of OSX) can probably do this, but I could be wrong about that. I do have a Mac, but I never had a reason to play around with this sort of stuff.

VMware fusion and applescript with global hotkeys

I need a terminal to a Linux vm inside my osx installation. Its a dropdown terminal and in the vm I can use F12 as a global hotkey to toggle show/hide.
The name of the dropdown terminal is yakuake, I know there is a dropdown terminal that works with osx to, but for various reasons, I need both a Linux terminal and easy access to it.
There is several things that makes this difficult.
VMware fusion doesnt have any good applescript support
I need to be able to run several vm's on the same time.
Speed
Hiding the application.
I am using iKey to get a global hotkey on my mac binded to a little oneliner:
/Library/Application\ Support/VMware\ Fusion/vmrun -T fusion -gu username -gp password runScriptInGuest "/Users/username/Documents/Virtual Machines.localized/Ubuntu.vmwarevm/Ubuntu.vmx" -interactive "/bin/bash" "/home/username/bin/toggle_yakuake"
The /home/username/bin/toggle_yakuake script is just a:
/usr/bin/xdotool key Control+Shift+Y # Which is what I bound to yakuake
This works, except its taking around 1-2 seconds, and yakuake will show up below every other windows.
Therefor I will use logic to just do this if yakuake isnt running, and then I can just hide/show it in osx to toggle it instantly.
However, I am able to show yakuake using a simple "tell application "Yakuake - Ubuntu" to activate. I am not able to hide it without hiding every other application belonging to the same vm.
It seems that every application spawned by vmware is tied together a little to tight.
Even if I find the pid of yakuake on my osx installastion, and uses it like:
tell application "System Events"
set yakuakeproc to every process whose unix id is 58518
repeat with proc in yakuakeproc
set the frontmost of proc to true
end repeat
end tell
It will hide every application belonging to that vm.
I have tried to figure out a solution for this for days now. Is there anyone that have any tips or anything? How can I get applescript to hide one specific application which is inside my vm?
The way you're starting yakukake is unnecessarily complex. You should be able to simply run the VMware proxy application, rather than using a shell script; i.e. your tell application "Yakukake - Ubuntu" to activate.
Then your only real problem is how to show and hide an application from a script. I think it is possible to do this with System Events, but it's easier to do it with my tool appswitch:
I don't have GUI Linux installed in VMware Fusion but here's an example with Windows.
To view information about a process:
% appswitch -La Notepad
PSN PID TYPE CREA NAME PATH (bundle identifier)
15994688.0 24758 APPL ???? Notepad /Users/nicholas/Library/Application Support/VMware Fusion/Virtual Machines/Boot Camp/Boot Camp.vmwarevm/Applications/Notepad — Windows 7.app (com.vmware.proxyApp.564dbc100b31a4b5-4f8d8088bf0c3705.2009924420)
To hide it:
% appswitch -ha Notepad
To show it:
% appswitch -a Notepad
That's it.

Alternative to SendKeys that does NOT require an unlocked session

Situation:
A GUI app contains functionality (off a menu option) that produces a frequently updated image to a directory.
A logged-in, running instance of the app is the ONLY source for this image (functionality 'reliant' on display device). I have researched this to death - it is a sad fact.
The GUI application offers COM interfaces, but none that generate the image.
GUI code cannot be change in the least (big surprise).
Requirement:
These current images are needed by other processes at various times.
Obvious solution:
A process that creates an instance of the GUI app and uses SendKeys to manipulate the controls to produce the image.
Roadblocks (do I need to elaborate)
Aside from the flakiness of Sendkeys - assuming that Sendkeys WAS reliable....
Sendkeys can't work when console session is locked (locked is production requirement)
SendMessage API can't send key combinations like 'shift/letter' (required to invoke menu option).
Questions
Is there any other way to programatically interact with the app when the session is locked?
Can a windows service unlock/lock the sesion at predetermined times - long enough to allow an image generation to occur.
I know, I know, its crap. ANY high level ideas and MOST opinions are appreciated ;)
Virtual PC.
Lock the host, not the virtual machine.
But to actually answer your question: i don't think you can send keys to a locked computer. Why? What if there are multiple logged in sessions; which one would it send the keys to?

Resources