I have been struggling with bootstrapping my Windows instances in AWS. I need to download some things from S3 and other places when the instance starts up and execute them.
This seems to be really straightforward for Linux instances, but not so much for Windows instances.
I have a user data script that works when I run it after logging into the instance, but it doesn't work when it runs as part of the EC2Launch/EC2Config described here: https://docs.aws.amazon.com/AWSEC2/latest/WindowsGuide/ec2-windows-user-data.html
I've found that the reason it doesn't work is that I am unable to download things from the internet due to: "The operation being requested was not performed because the user has not logged on to the network. The specified service does not exist."
Really what I'm trying to do is the following:
Download AWS_CLI Installer
Right now I'm using bitsadmin to try to download the installer from https://s3.amazonaws.com/aws-cli/AWSCLI64.msi
Install AWS_CLI
I am able to install the CLI with the CLI installer using msiexec and the /qn flags once it's on the box
Pull items from S3
This should be able to be done with aws s3 cp or sync
Install Python (installer stored in S3)
I am able to install the installer with msiexec once it's on the box similar to the CLI installer
Execute (python) scripts pulled from S3
Does anyone know of a better way to do this? This all works after I RDP in, but not as part of the Launch Script.
TL;DR - I need to download things using the EC2Launch User Data script provided to my EC2 Windows instances but the network doesn't seem to be available when it runs.
Thanks in advance for your help!
I Ran into a similar issue, I was using BITS to download an installer on a "user data" script and was getting the same error: "The operation being requested was not performed because the user has not logged on to the network. The specified service does not exist."
This not due to network issues, but rather to how BITS works, the user starting the download job must be logged on the machine, which it seems not to be the case for "user data" execution, which "runs as" the local Administrator without the user being logged in a way that BITS recognizes. From BITS documentation:
For BITS to detect that a user is logged on, the user must use one of the following interactive logon options:
Log on through the Welcome screen.
Log on to a terminal client.
Use fast user switching.
Starting with Windows 10, version, log on from another device using Remote Powershell.
During the script execution the network is mounted and ready to use, you can download your files using other methods that do not rely on BITS(e.g.: Invoke-WebRequest (rather slow for bigger files) or using System.Net.WebClient, both on PowerShell).
I am responsible for running centralized backups of Windows Security logs on a network of ~15 Windows boxes. To automate this task, I have been writing a Powershell script that utilizes wevtutil's /r parameter to do it all remotely. All of the boxes are connected to a Sharepoint network drive that I was hoping to copy the logs to so that I could centralize all of the logs, but I've run into some trouble.
The script runs fine when I pass it the ip of the Windows box that it's running on. The logs are copied to the Sharepoint without a hitch. The script also runs fine when I just tell it to copy the logs locally. However, when the script trys to remotely copy the computer's log to the sharepoint, I get a Failed to Archive Security log. The account is not authorized to log in from this station. error.
The format of the command is
wevtutil epl Security \\path\to\sharepoint\[hostname]-[datetime]Security.evtx /r:[hostname]
I am running the script as a domain admin. I have also ran the script with the credentials of a local admin and I got a generic access denied error.
A google search for the error message mostly includes computers not being able to access network drives (not remote wevtutil specifically) and include a lot of messing around in
HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\LanmanWorkstation\Parameters
or
HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\LanmanServer\Parameters
which have not yielded any results. I am also restricted to Powershell 1, so I cannot use Powershell itself remotely AFAIK.
I could just put the script on each machine and run it locally with task scheduler, but I was hoping for a more elegant solution. Does anyone have experience with using wevtutil in this way and can point me in the right direction, or perhaps even suggest a better technique/tool?
How much latitude do you have to implement another solution?
If you wanted to do something with the logs, or make them easily searchable, you could set up a free Splunk server and either use a Splunk forwarder to ship the logs off box, or you could also use Powershell to send the logs to Splunk's HTTP event collector.
I have a script on my domain stored on the Active Directory server. every machine on the domain has a task that when fired, calls this script to run.
Running the task from the AD server works. Running the task from another machine doesn't work. However, running the command that is triggered from cmd manually on the remote computer works?
Could anyone shine some light on this. Basically I call it like this so that the trigger is set up like...
Action: PowerShell.exe
Arguments: -noprofile –ExecutionPolicy Bypass -File "\\<>NameOfADServer<>\C$\Tasks\script.ps1" "Argument 1" "Argument 2"
Running as SYSTEM is probably your issue - it wont have any access outside of the PC its running on.
When you run it manually youll have the access.
There's several problems here.
You're running the task as the local SYSTEM accounts. SYSTEM generally does not have access to any network resources.
You're using the administrative share (\\<servername>\C$) to share the script. Only users that have Administrator access to the server can access the administrative shares. Administrative shares are heavily restricted by design and you cannot modify the access on them.
My guess is that the script works when you run it manually is because it's using the current user's credentials for network access when you do that, but don't quote me on that.
The simplest solution with the least amount of change is to do this:
Create a group in Active Directory. Add the Computer accounts, or, preferably, groups with Computer accounts which you want to be able to run the script to this new group. If you really want any SYSTEM account on any computer in the domain to be able to run the script, you can add the "Domain Computers" group to the group.
Create a folder on the server. Put the script in the folder. Don't put anything in this folder you don't want your users to read. Assign the "Read" NTFS permission to the group created above to the folder.
Share the folder out. Grant the group you just created the "Full Control" share access. If you want, you can make it a hidden share by adding a dollar sign to the end of the name.
Update your scheduled tasks to use \\<servername>\<sharename>\script.ps1.
This is almost certainly not the best method to accomplish what you're actually trying to do, but this is probably the best way to use scheduled tasks running scripts on a network share with the SYSTEM account.
I am having trouble when I have one Windows service try to install another Windows service.
Specifically, I have a TeamCity agent running tests for me on a Windows 2008 AWS instance. The tests are written in Java, which shell out to a .bat script to install a service (let's call it Service A), giving it a unique name each time.
The offending line is in the .bat script: sc create "%serviceName%" binPath= %binPath% DisplayName= "%serviceDisplayName:"=%" start= %serviceStartType%. I believe as long as the service name is unique that should work.
And indeed it does work if I run the tests manually on the command line, using an administrator account. Service A is installed, the test completes and Service A is uninstalled at the end.
I have tried running the TeamCity agent as LocalSystem, as Administrator, and as another user that is member of the administrators group. I have also tried disabling UAC completely.
Presumably the problem is access denied type errors, although that is not clear at this point. There are a few avenues to explore still, but it is a simple question really: are processes running as services forbidden from installing other services? Are there special things I have to do to configure the machine/ account to allow it to do this?
The point of the test it to install and use Service A, so workarounds are not relevant - Service A must be operated as a black box.
Thanks!
There are no restrictions on creating services with regards to how the creating process can execute, as long as the process has the appropriate permissions. That is to say, a process could be running as a service and create another service -- the only consideration here is the appropriate permission level.
The problem that often occurs with running batch scripts from within processes (as opposed to directly through user input on the command line) is that the environment expected isn't always the environment that is loaded. In this case, it appears that the env variables referred to in the batch script weren't properly set when running as a service, which of course then caused the service install failure. Correcting the environment loaded when the batch script is shelled out is the correct solution here.
Suppose some Windows service uses code that wants mapped network drives and no UNC paths. How can I make the drive mapping available to the service's session when the service is started? Logging in as the service user and creating a persistent mapping will not establish the mapping in the context of the actual service.
Use this at your own risk. (I have tested it on XP and Server 2008 x64 R2)
For this hack you will need SysinternalsSuite by Mark Russinovich:
Step one:
Open an elevated cmd.exe prompt (Run as administrator)
Step two:
Elevate again to root using PSExec.exe:
Navigate to the folder containing SysinternalsSuite and execute the following command
psexec -i -s cmd.exe
you are now inside of a prompt that is nt authority\system and you can prove this by typing whoami. The -i is needed because drive mappings need to interact with the user
Step Three:
Create the persistent mapped drive as the SYSTEM account with the following command
net use z: \\servername\sharedfolder /persistent:yes
It's that easy!
WARNING: You can only remove this mapping the same way you created it, from the SYSTEM account. If you need to remove it, follow steps 1 and 2 but change the command on step 3 to net use z: /delete.
NOTE: The newly created mapped drive will now appear for ALL users of this system but they will see it displayed as "Disconnected Network Drive (Z:)". Do not let the name fool you. It may claim to be disconnected but it will work for everyone. That's how you can tell this hack is not supported by M$.
I found a solution that is similar to the one with psexec but works without additional tools and survives a reboot.
Just add a sheduled task, insert "system" in the "run as" field and point the task to a batch file with the simple command
net use z: \servername\sharedfolder /persistent:yes
Then select "run at system startup" (or similar, I do not have an English version) and you are done.
You'll either need to modify the service, or wrap it inside a helper process: apart from session/drive access issues, persistent drive mappings are only restored on an interactive logon, which services typically don't perform.
The helper process approach can be pretty simple: just create a new service that maps the drive and starts the 'real' service. The only things that are not entirely trivial about this are:
The helper service will need to pass on all appropriate SCM commands (start/stop, etc.) to the real service. If the real service accepts custom SCM commands, remember to pass those on as well (I don't expect a service that considers UNC paths exotic to use such commands, though...)
Things may get a bit tricky credential-wise. If the real service runs under a normal user account, you can run the helper service under that account as well, and all should be OK as long as the account has appropriate access to the network share. If the real service will only work when run as LOCALSYSTEM or somesuch, things get more interesting, as it either won't be able to 'see' the network drive at all, or require some credential juggling to get things to work.
A better way would be to use a symbolic link using mklink.exe. You can just create a link in the file system that any app can use. See http://en.wikipedia.org/wiki/NTFS_symbolic_link.
There is a good answer here:
https://superuser.com/a/651015/299678
I.e. You can use a symbolic link, e.g.
mklink /D C:\myLink \\127.0.0.1\c$
You could us the 'net use' command:
var p = System.Diagnostics.Process.Start("net.exe", "use K: \\\\Server\\path");
var isCompleted = p.WaitForExit(5000);
If that does not work in a service, try the Winapi and PInvoke WNetAddConnection2
Edit: Obviously I misunderstood you - you can not change the sourcecode of the service, right? In that case I would follow the suggestion by mdb, but with a little twist: Create your own service (lets call it mapping service) that maps the drive and add this mapping service to the dependencies for the first (the actual working) service. That way the working service will not start before the mapping service has started (and mapped the drive).
ForcePush,
NOTE: The newly created mapped drive will now appear for ALL users of this system but they will see it displayed as "Disconnected Network Drive (Z:)". Do not let the name fool you. It may claim to be disconnected but it will work for everyone. That's how you can tell this hack is not supported by M$...
It all depends on the share permissions. If you have Everyone in the share permissions, this mapped drive will be accessible by other users. But if you have only some particular user whose credentials you used in your batch script and this batch script was added to the Startup scripts, only System account will have access to that share not even Administrator.
So if you use, for example, a scheduled ntbackuo job, System account must be used in 'Run as'.
If your service's 'Log on as: Local System account' it should work.
What I did, I didn't map any drive letter in my startup script, just used net use \\\server\share ... and used UNC path in my scheduled jobs. Added a logon script (or just add a batch file to the startup folder) with the mapping to the same share with some drive letter: net use Z: \\\... with the same credentials. Now the logged user can see and access that mapped drive. There are 2 connections to the same share. In this case the user doesn't see that annoying "Disconnected network drive ...". But if you really need access to that share by the drive letter not just UNC, map that share with the different drive letters, e.g. Y for System and Z for users.
Found a way to grant Windows Service access to Network Drive.
Take Windows Server 2012 with NFS Disk for example:
Step 1: Write a Batch File to Mount.
Write a batch file, ex: C:\mount_nfs.bat
echo %time% >> c:\mount_nfs_log.txt
net use Z: \\{your ip}\{netdisk folder}\ >> C:\mount_nfs_log.txt 2>&1
Step 2: Mount Disk as NT AUTHORITY/SYSTEM.
Open "Task Scheduler", create a new task:
Run as "SYSTEM", at "System Startup".
Create action: Run "C:\mount_nfs.bat".
After these two simple steps, my Windows ActiveMQ Service run under "Local System" priviledge, perform perfectly without login.
The reason why you are able to access the drive in when you normally run the executable from command prompt is that when u are executing it as normal exe you are running that application in the User account from which you have logged on . And that user has the privileges to access the network. But , when you install the executable as a service , by default if you see in the task manage it runs under 'SYSTEM' account . And you might be knowing that the 'SYSTEM' doesn't have rights to access network resources.
There can be two solutions to this problem.
To map the drive as persistent as already pointed above.
There is one more approach that can be followed. If you open the service manager by typing in the 'services.msc'you can go to your service and in the properties of your service there is a logOn tab where you can specify the account as any other account than 'System' you can either start service from your own logged on user account or through 'Network Service'. When you do this .. the service can access any network component and drive even if they are not persistent also.
To achieve this programmatically you can look into 'CreateService' function at
http://msdn.microsoft.com/en-us/library/ms682450(v=vs.85).aspx and can set the parameter 'lpServiceStartName ' to 'NT AUTHORITY\NetworkService'. This will start your service under 'Network Service' account and then you are done.
You can also try by making the service as interactive by specifying SERVICE_INTERACTIVE_PROCESS in the servicetype parameter flag of your CreateService() function but this will be limited only till XP as Vista and 7 donot support this feature.
Hope the solutions help you.. Let me know if this worked for you .
I find a very simple method: using command "New-SmbGlobalMapping" of powershell, which will mount drive globally:
$User = "usernmae"
$PWord = ConvertTo-SecureString -String "password" -AsPlainText -Force
$creds = New-Object -TypeName System.Management.Automation.PSCredential -ArgumentList $User, $PWord
New-SmbGlobalMapping -RemotePath \\192.168.88.11\shares -Credential $creds -LocalPath S:
You wan't to either change the user that the Service runs under from "System" or find a sneaky way to run your mapping as System.
The funny thing is that this is possible by using the "at" command, simply schedule your drive mapping one minute into the future and it will be run under the System account making the drive visible to your service.
I can't comment yet (working on reputation) but created an account just to answer #Tech Jerk #spankmaster79 (nice name lol) and #NMC issues they reported in reply to the "I found a solution that is similar to the one with psexec but works without additional tools and survives a reboot." post #Larry had made.
The solution to this is to just browse to that folder from within the logged in account, ie:
\\servername\share
and let it prompt to login, and enter the same credentials you used for the UNC in psexec. After that it starts working. In my case, I think this is because the server with the service isn't a member of the same domain as the server I'm mapping to. I'm thinking if the UNC and the scheduled task both refer to the IP instead of hostname
\\123.456.789.012\share
it may avoid the problem altogether.
If I ever get enough rep points on here i'll add this as a reply instead.
Instead of relying on a persistent drive, you could set the script to map/unmap the drive each time you use it:
net use Q: \\share.domain.com\share
forfiles /p Q:\myfolder /s /m *.txt /d -0 /c "cmd /c del #path"
net use Q: /delete
This works for me.