I need to run a GUI script (AutoHotKey, which makes mouse clicks and press keys) on:
A different windows user (i.e. I run the script, then switch user
while keeping the session active)
A remote Amazon windows server (using remote desktop) where I also
run the script then close the Remote Desktop while keeping the
instance running
Unfortunately, in both cases, the script doesn't run as it seems that Windows enters a "GUI-less" mode where all GUI components are not active anymore.
Is there any solution to this?
Any hint would be greatly appreciated as I've now lost days trying to solve this!
Many thanks, Thomas
I do not believe this is possible due to the way Remote Desktop is implemented. When you close the RD connection, the GUI is no longer drawn. Therefore, AutoHotkey is unable to perform mouse clicks and key presses.
A possible workaround would be to make some registry tweaks (if possible on your server) which allow GUI interaction while minimized.
Locate HKEY_CURRENT_USER\Software\Microsoft\Terminal Server Client
Create RemoteDesktop SuppressWhenMinimized as DWORD with a value of 2
Registry Tweak Source
Related
I understand that desktop/GUI apps are not supported in Windows containers. They do run but there's no built-in way to interact with them. I had the following idea - maybe I could use the Desktop Sharing API (https://learn.microsoft.com/en-us/windows/win32/api/_rdp/) for this purpose, the idea is to run a desktop app, then run a sharing program that uses the Desktop Sharing API, and connect to it using a Desktop Sharing API viewing program from the host.
I had to do some recap about window stations and desktops, and I noticed that when starting the container with cmd in interactive mode, I'm logged with ContainerAdministrator as a service (logon type 5). I tried running some WinAPI functions that deal with desktops and winstation and got some access denied results, so I switched to running cmd as system.
The window station of the cmd process (and other child processes) is not the interactive WinSta0, but instead some other service window station, which makes sense since I'm logged on as a service, and I figured that I can't use this window station, so I used a little program I wrote to run notepad in Winsta0 in the Default desktop. Afterwards I ran another program that enumerates the windows on WinSta0\Default, and the notepad window does get enumerated and I also get it's title, so it's running somewhere.
So now I tried running the desktop sharing API program (also on WinSta0\Default). It runs and I can connect from the host, but I only get a black screen without anything on it. I also tried running a program that takes a screenshot of the windows but I get an empty bitmap.
So I thought maybe the Default desktop is not the active desktop, and by using the OpenInputDesktop function I could confirm it - the current desktop was the Winlogon desktop, so I used the SwitchDesktop function to switch to the Default desktop (I used OpenInputDesktop again to verify that it actually worked).
Unfortunately, this didn't change anything, I still get an empty screen and empty bitmaps.
I know that containers are built for micro services and are not supposed to run GUI apps and so, but still - is there a way to make this work? Or any ideas of what else I can check? Alternatively, if you know that it can't work - I would also be happy to hear a good technical explanation of why it doesn't work.
As in title. Is there any way to achieve this?
I have problem, because when you close remote desktop, gui and whole interface is being locked and any gui based applications or my autoit scripts won't work (they are crashing constantly).
I tried using 'tscon.exe' but this works only on windows server 2008.
I have the same issue, and I resolved it doing the following:
Connect to RD to your server with user "A"
From the remote computer, connect to RD to localhost to user "B" (create it if necessary)
From the new session, run anything you need.
Disconnect the FIRST session, with user "A".
In this way, the session running with user "B" will not be locked, and it will continue running your GUI controls.
Log off means the current session (created by logging in during RDP) is terminated. What you want is to keep the Logon Session live but disconnect the RDP.
Why not use AutoLogon to boot into console and run the scripts? However if the screen locks or someone RDPs and disconnects, the problem you are facing will reoccur.
What I have is to keep GUI and whole windows interface alive when i close remote desktop. Session is alive but gui and whole workstation is getting locked everytime I close rd.
I want to write a script that involves mouse click and stuff. I am little confused though. Is there a way mouse clicks (in the form of x and y co-ordinates) would work on a VPS which is logged on, but disconnected from the client?
Example:
MouseClick("left",588, 268)
To my knowledge, these VPSes don't have their own native screen resolution. I have already written the script for my local pc and it's working fine on my resolution. But when tried with the VPS, it doesn't work.
PS: I am running the script from the remote desktop itself and not from my local pc. Please correct me if there's anything I am doing wrong :)
Update: For anyone still looking for an answer, consider installing a vnc server, that way the screen resolution you specify is preserved even when not connected to the server.
Ex: https://www.realvnc.com/en/connect/download/vnc/
It's relevant to almost any automation tool, not only AutoIt.
Remote Desktop has its own GUI context only if it's connected to the remote PC. But you can lose focus for a Remote Desktop window (without disconnection & without minimizing) and continue other local work.
The same effect takes place for VNC server software.
One team in our organization uses it for massive test runs automation: a master server creates a remote session for each test machine and keeps them all connected until tests passes.
Use ControlClick to have the mouse click over RDP connections, even when you're not connected. For some reason it can still be iffy for me, but at least it worked more so that Click which was none at all.
Ok, this is a tricky one. I'm trying to set up a Selenium Grid 2 with some Windows 7 VMs to run Webdriver tests. To automatize the whole process I use some ant script that connects to the VMs through ssh to start/stop/reconfigure the nodes.
Everything works great, the nodes can register with the hub host and execute the test. Only problem is that I don't see any browser window during the test run. I can see the process and I see the test log being executed, but there is no graphical interface.
On the other hand, if I start the node manually through Windows, everything is normal.
I suppose the problem is that processes executed under cygwin cannot start Windows displays, but in that case, shouldn't throw an error? The other option I'm thinking is that Webdriver is using HTMLunit as a fallback, but then... why do I see the firefox process as long as the test lasts and consuming CPU and memory?
Through ssh, you only exchange with Windows stdin, stdout and stderr streams. The ssh connection is tunneling those streams and nothing else. You don't see Windows Desktop interface, but the Desktop object exists on the Windows machine, the programs (here the browsers) are connected to it, and all GUI interactions are live in there.
If the GUI doesn't require any user interaction, everything is fine that way. The dialog boxes are created, the program runs, once it finishes, the dialog boxes are destroyed by the application and the application closes. Nothing is blocking in terms of GUI our application.
If you program requires an user action in the created yet invisible dialog boxes, your program will be there waiting for your interaction to move forward. You will see the process in the task manager, doing nothing but waiting. As you don't have access to the Windows Desktop where the dialog boxes are created and virtually 'displayed', the program seems to hang.
A typical case 2 is if you remote run a program waiting for a user to do something, say notepad. You can launch notepad, it will be spawned and then it will wait for you to type some text or close it.
With your Selenium tests, you are in case 1: all the browsers' interactions needed to make the GUI working are actually done by Selenium server that does the navigation clicks and the program exit for you. Their GUI actually are living by browsing through your test web servers, you just don't see it.
Some further readings from Microsoft website on Desktops and Desktop Creation.
If you want to see the tests and have valid screenshots, you need to have a user logged in and those tests need to run as that user. Everything must run through that single desktop session, so you cannot use RDP to remotely connect to the machine. Your best bet is to use VNC, since that will connect to an already established session.
I am using the below JAVA code to capture the desktop of a remote machine
Robot robot = new Robot();
BufferedImage screenShot = robot.createScreenCapture(new Rectangle(Toolkit.getDefaultToolkit().getScreenSize()));
ByteArrayOutputStream imageBytes = new ByteArrayOutputStream();
ImageIO.write(screenShot, "PNG", imageBytes);
return imageBytes.toByteArray();
However the captured image is blank, when the terminal session to the remote machine is either minimized or disconnected. I appreciate your help in resolving the issue, at the very least the minimized scenario.
Configuration:
I have the same issues with a physical machine running windows 7 and a virtual machine running windows server 2008 R2.
More insights from MSDN:
Why you get black screen when you disconnect from RDP ?
http://msdn.microsoft.com/en-us/library/aa383015%28VS.85%29.aspx
Here is my attempt to make things work, but none of the following did the trick:
How to get data when RDP window minimized ?
You can force the RDP display driver to send data when minized, try these steps and let me know how it goes:
1) Add the following key
HKEY_LOCAL_MACHINE\Software\Microsoft\Terminal Server Client\ Created a new DWORD value and named it RemoteDesktop_SuppressWhenMinimized. Specified 2 as the value data.
Note: Also tried adding the registry key to HKEY_LOCAL_MACHINE\Software\Wow6432Node\Microsoft\Terminal Server Client\
2) Disable bitmap caching (http://technet.microsoft.com/en-us/library/cc737325(WS.10).aspx)
In the Remote Desktop Connection window, click Options.
On the Experience tab, verify that the Bitmap caching check box is selected. Or, to disable bitmap caching, clear theBitmap caching check box
If you minimize the Remote Desktop window, Windows switches the remote session to the GUI-less mode and does not display windows and controls. As a result, TestComplete (or TestExecute) will be unable to interact with the tested application’s GUI, as it does not exist and your automated GUI test will fail.
To work around the issue, you can change the Remote Desktop’s registry settings on your local computer (where you launch the Remote Desktop):
On your local computer, close all open Remote Desktop sessions.
Launch the Registry editor (regedit.exe).
Navigate to one of the following Registry keys, depending on whether you wish to modify the Remote Desktop settings only for the current user or for all users on the computer:
HKEY_CURRENT_USER\Software\Microsoft\Terminal Server Client
HKEY_LOCAL_MACHINE\Software\Microsoft\Terminal Server Client
Create a DWORD value named RemoteDesktop_SuppressWhenMinimized and set it to 2.
Or you can do it programmatically by following steps:
Transparent the window
Restore the Window
Capture
Minimize it again
Remove transparency
We had similar problem in our project last year...we could find any stable and permanent solution....however here is how a work around worked for us...
We had to run on 5 remote desktops (which will take screens capture as well during failure) ..however as you have figured already if we disconnect session or minimise the RDC Window blank screenshot is captured. Therefore we had added one more VM to connect those other five test boxes. The task for that VM is to keep session active and resized to other 5 boxes...this worked for us.
If you connect a remote desktop to the test machine, make sure to reboot the machine when you are done, otherwise the desktop will remain locked and screen captures will not work.
I don't believe there is any other way around the issue.
I just configured the clients to auto logon, disabled the screen saver and installed a VNC server on each client.
Basically, make sure the screen is always on, and don't RDP into them.
This worked on both physical PCs and on virtual machines hosted on a Hyper-V server.
I even wrote a small .NET desktop client that ran multiple VNC clients inside a single window, so we could see what was happening on all the clients. We had an old PC running this with it's monitor on top of a cupboard to (a) let the developers see if any client had hung, had hundreds of browser windows open, etc. and (b) to look impressive for any non-developers walking past.
A simpler alternative to the above answers to to transmute the terminal session (RDP) into the console session. The session will then display to the physical screen (Switching the user of anyone currently logged into the physical machine). The following command does this:
for /f "skip=1 tokens=3" %%s in ('query user %USERNAME%') do (tscon.exe %%s /dest:console)