There is a huge variance in the launch times of Windows AMIs (EBS-backed) that I am using. Some start up in just 3 minutes. Others can take 20+ minutes. My understanding is that the default Windows AMIs can be slow as they require two reboots to get active, but in my case these are all customized machines, either public or snapshots I have created.
On a similar note, I was retrieving the log files in the EC2 console to know when my machine is started. However, some of the machines DO NOT seem to generate any logs?? So, realistically, I have a variable startup time and variable logging, in which case how can I even really tell that a Windows machine has become avialable?
It does take a varying amount of time to launch a Windows AMI in EC2. You can minimize it, a bit, by setting a fixed machine name for the instance. Do this as you would on any Windows computer - in the properties of "My Computer", "Computer Name" tab. Then, run "EC2ConfigService Settings" from the Start Menu's "All Programs" list. That program is installed there by Amazon on most base AMIs. In that program, on the "General" tab, UnCheck "Set Computer Name". This eliminates the system from re-booting itself once while starting up the image, as it would have to, in order to set the name.
Still, you would like to be notified when your instance is ready! This is a perfect job for Amazon's Simple Notification Service. The service (also known as SNS) is simple to use programmatically (from a windows .NET project for example), free (for the first 100,000 messages, less than 1gb in total), and the notifications are immediate.
Code to send a notification (in VB.NET):
Imports Amazon.EC2.AmazonEC2Client
Imports Amazon.SimpleNotificationService
DIM LabSNS As New AmazonSimpleNotificationServiceClient(Lab_AWSKey, Lab_AWSSecretKey)
Dim PubReq As New Amazon.SimpleNotificationService.Model.PublishRequest
Dim Msg As String 'Messege to be built up, then be sent. It is body of eMail.
Msg = "The instance is running and ready!"
Msg = Msg + vbCrLf + "Previous State of machine was:" & PreviousState 'A made-up global
Msg = Msg + [Any other info. I want to send myself about the start of the instance.]
PubReq.WithTopicArn(Topic)'Topic is a global. It's value is a key from SNS topic setup.
PubReq.WithSubject("EC2 Instance is Ready!")
PubReq.WithMessage(Msg)
LabSNS.Publish(PubReq)
The code requires Amazon's SDK for .NET which is free. Write a program including some code like the above. Set the program to run after the computer starts, and before login, using the Windows Task Scheduler - create a task triggered "at system startup" that calls the program.
The setup for SNS is documented here:SNS Documentation
It looks like a lot of trouble to send eMail, however, Amazon's EC2 environment is highly restrictive when it comes to sending eMail. Many have tried to use EC2 as a spam platform, so Amazon has been thorough in blocking SMTP (eMail) traffic, except as prescribed by Amazon. You can't just open up a port on the Amazon security group to bypass Amazon's blocks.
Amazon does have a general eMail facility one can use from within EC2. It is called, Amazon Simple Email Service (SES). That will not work well for you, as it is designed for bulk eMail. So, SES's pricing, exception handling and messaging won't fit well with what you need, I don't think.
SNS, on the other hand works great for this. It includes an initial eMail to the recipients (you, and perhaps others you may want to notify of your server coming on-line) asking if they want to receive future messages of the topic; they are given an option to opt out, and must reply to receive further.
The setup process (shown in blocks above) is all easily doable from Amazon's AWS Management Console. (Your question implies you that you already have an AWS EC2 account needed for this.) Once setup, any instance launched from the AMI would send out an eMail containing any information (available to your program) of your choosing as soon as the machine is ready.
It'll be gotcha-free in setting up, and solid as a rock in operation.
Regardless of the source of your Windows AMI, it will reboot a number of times during its startup process before it becomes available via RDP. All Windows AMIs are derived from the Windows AMIs produced by Amazon, which have this boot process by design. [It's been suggested that this boot process is hard-coded into a custom kernel that is running inside the guest VM.]
Console logs typically take between 2 and 5 minutes to appear.
Unfortunately, Windows on EC2 is more difficult to automate and track than linux. The RightScale and Scalr folks have done some excellent work integrating Windows into their management platform. And the Opscode Chef configuration Management tool also supports Windows in EC2 and can help you discover when your instances are ready for use.
Related
I want to create an Azure Virtual Machine that I only need to run approximately for 1 or 2 hours once or twice per day. I don't want to pay for the server when I'm not using it.
I know I can just go to the dashboard and shut it off, but I often forget to do so, I'm getting senile! I would like for a timer to start when the system (Windows 10) is started, and when the timer reaches zero, the image is made inactive (no charges incurred) unless I request more time.
Any ideas would be appreciated.
The dashboard has an automatic shutdown feature. This can also be configured to send a webhook notification with a link which will delay the shutdown. Currently in-machine notifications are not supported.
Although there are many tools like PowerShell scripts and apps that you could use from within the VM to trigger automatic shutdown, there are some billing gotchas to be aware of.
With an Azure VM, you pay per second and cease paying only when the machine is completely deallocated and no longer reserving memory and cores on the platform. (There is still a nominal storage charge associated with storage of the VM image). You cannot deallocate the VM from within itself.
To ensure the VM isn't incurring charges, check that the status is 'Stopped (deallocated').
I've got the following totally reproducible scenario, which I'm unable to understand:
There is a very simple application, which does nothing else than calling CreateObject("Word.Application"), that is creating an instance of MS Word for COM interop. This application is located on a Windows Terminal Server. The test case is to connect via RDP, execute the application and the application will output the time taken for the CreateObject call.
The problem now is that the execution time is significantly longer, if I connect from a specific notebook (HP Spectre): It takes 1,7s (+/- 0.1s).
If I connect from any other machine (notebook or desktop computer), then the execution time is between 0,2-0,4s.
The execution times don't depend on the used RDP account, or screen resolution, or local printers. I even did a fresh install of Windows on that HP notebook to rule out any other side-effects. It doesn't matter if the HP notebook is connected via WLAN or an USB network card. I'm at a loss understanding the 4x to 8x execution time difference to any other machine.
Which reason (component/setting) could explain this big difference in execution time?
Some additional information: I tried debugging the process using an API monitor and could see that >90% of the execution time is actually being spent between a call to RpcSend and RpcReceive. Unfortunately I can't make sense of this information.
It could be the credential management somehow being in the way.
Open the .rdp file with notepad and add
enablecredsspsupport:i:0
This setting determines whether RDP will use the Credential Security Support Provider (CredSSP) for authentication if it is available
Related documentation
https://learn.microsoft.com/en-us/previous-versions/windows/it-pro/windows-server-2008-R2-and-2008/ff393716%28v%3dws.10%29
According to your information about RpcSend and RpcReceive time consumption, it could be the case you have some service stopped on your client machine, like DCOM server or some other COM-related (they usually have "COM" or "transaction" in their names).
Some of that services could be started/stopped (if Manually mode selected) by system to/after transfer your request, but there is a time delay to starting service.
I suggest you to open Computer Management - Services or run -> services.msc and compare COM-related services running on your "slow" client and on your "fast" clients, and try to set Automatically running instead Manually or Disabled.
Also, try to run API Monitor on such processes to determine the time-consuming place more precisely.
I would like to use a EC2 instance for accounting purpose as the instance is required to be accessed by multiple party. Since EC2 charges are per hour basis. I am planning to turn on the instance only when some one needs to use then turn if off. The instance is not required to be live 24x7.
Start the instance with a mouse click. (maybe some script or panel or url.
Remote Desktop Connect- My accountant finishes the work.
User shutdown the instance or auto shutdown after 30 mins of inactivity.
How do we go about it ?
Thanks,
Dwija
You can Start an Amazon EC2 instance that was previously Stopped by sending a StartInstance command. This can be done from many different programming languages, or via the AWS Command-Line Interface (CLI) (available for Windows, Mac and Linux).
Turning off the instance when it is not in use is harder because you need to identify when it is appropriate to turn it off. You could have a program on the instance itself (much like a screensaver) that detects when the system is 'idle' and issues a Shutdown command on the virtual machine (just like Choosing Shutdown from the Start menu). This will Stop the Amazon EC2 instance.
An alternative, depending on your use-case, would be to use Amazon Workspaces, which is a virtual desktop -- a computer that is running in the cloud that you can connect with from Windows, Mac, iOS or Android devices. It is charged either on a monthly subscription or on a combination of subscription + hourly charge. It automatically turns off after the user has disconnected for a period of time. An Amazon Workspace is typically used by one person as their 'personal' computer, but it could be shared amongst other users.
At my workplace, we have lab machines that we use to do our testing.
The standard procedure to reserve a machine for testing was to walk around the office to make sure that no one was using the machine.
This is highly inefficient and time consuming.
At first, I set up a web page where people could reserve the lab machine but nobody was keeping the page updated so that turned up to be useless.
I finally found a solution using Microsoft log parser and wanted to share it to the stack overflow community.
It is a batch file that runs on the machine so the user can identify the last users that use the machine and easily IM them to ask if the machine is free.
Is there a better solution to do this?
Use the built-in command qwinsta (Query Win Station) to figure out what sessions (including console) are active or inactive (disconnected) and then act on the given information (creds to krusty.ar btw for linking this already).
If you feel people are abusing the machine in question, refer to rwinsta to nuke their sessions into oblivion...
You will need to install the Microsoft Log Parser
Then create the following 2 files
TSLoginsDetails.sql
SELECT
timegenerated,
EXTRACT_TOKEN(Strings,1,'|') AS Domain,
EXTRACT_TOKEN(Strings,0,'|') AS User,
EXTRACT_TOKEN(Strings,3,'|') AS SessionName,
EXTRACT_TOKEN(Strings,4,'|') AS ClientName,
EXTRACT_TOKEN(Strings,5,'|') AS ClientAddress,
EventID
FROM Security
WHERE EventID=682
ORDER BY timegenerated DESC
TSLogins.bat
echo off
cls
c:
cd "c:\Program Files\Log Parser 2.2\"
logparser.exe file:TSLoginsDetails.sql -o:DATAGRID
Now by placing this batch file on the desktop, the user can see who were the last people to login and contact them by IM to verify if they are done.
How about posting the information from the log file to the website that tells who is currently using the machine as well.
Check and notify when they log in.
Updated the "who is using the machine" page you made prior.
Run a AT job that checks every couple of hours who is on it.
Totally out of the box:
You can install the Software Testing Automation Framework (STAF) on your servers and desktops to manage your tests. It's written in Java, so you can use it on Windows and Unix/Linux desktops and servers.
Using STAF, you can create a resource pool of test servers on which you conduct tests, then write STAX jobs (STAX is a STAF execution framework) to conduct the tests. The job can grab the first available server from the resource pool, run the test, monitor the test status, log results, notify the submitter, then release the server back into the pool when done. If you have multiple people submitting jobs for tests, STAF will manage the queue of requests and satisfy them as they came in. Users can either monitor the job from their desktop, or you can set up email alerts to notify them when the test is complete.
I'm not sure if I understand you, but there are a set of command line tools to deal with terminal server sessions, and there's also a Windows API to do the same if you need to do this from a program.
Since it sounds like you're a microsoft shop, you can set up the machines as resources in outlook/exchange and reserve them that way.
We're running a lightweight web app on a single EC2 server instance, which is fine for our needs, but we're wondering about monitoring and restarting it if it goes down.
We have a separate non-Amazon server we'd like to use to monitor the EC2 and start a fresh instance if necessary and shut down the old one. All our user data is on Elastic Storage, so we're not too worried about losing anything.
I was wondering if anyone has any experience of using EC2 in this way, and in particular of automating the process of starting the new instance? We have no problem creating something from scratch, but it seems like it should be a solved problem, so I was wondering if anyone has any tips, links, scripts, tutorials, etc to share.
Thanks.
You should have a look at puppet and its support for AWS. I would also look at the RightScale AWS library as well as this post about starting a server with the RightScale scripts. You may also find this article on web serving with EC2 useful. I have done something similar to this but without the external monitoring, the node monitored itself and shut down when it was no longer needed then a new one would start up later when there was more work to do.
Couple of points:
You MUST MUST MUST back up your Amazon EBS volume.
They claim "better" reliability, but not 100%, and it's SEVERAL orders of magnitude off of S3's "12 9's" of durability. S3 durability >> EBS durability. That's a fact. EBS supports a "snapshots" feature which backs up your storage efficiently and incrementally to S3. Also, with EBS snapshots, you only pay for the compressed deltas, which is typically far far less than the allocated volume size. In another life, I've sent lost-volume emails to smaller customers like you who "thought" that EBS was "durable" and trusted it with the only copy of a mission-critical database... it's heartbreaking.
Your Q: automating start-up of a new instance
The design path you mention is relatively untraveled; here's why... Lots of companies run redundant "hot-spare" instances where the second instance is booted and running. This allows rapid failover (seconds) in the event of "failure" (could be hardware or software). The issue with a "cold-spare" is that it's harder to keep the machine up to date and ready to pick up where the old box left off. More important, it's tricky to VALIDATE that the spare is capable of successfully recovering your production service. Hardware is more reliable than untested software systems. TEST TEST TEST. If you haven't tested your fail-over, it doesn't work.
The simple automation of starting a new EBS instance is easy, bordering on trivial. It's just a one-line bash script calling the EC2 command-line tools. What's tricky is everything on top of that. Such a solution pretty much implies a fully 100% automated deployment process. And this is all specific to your application. Can your app pull down all the data it needs to run (maybe it's stored in S3?). Can you kill you instance today and boot a new instance with 0.000 manual setup/install steps?
Or, you may be talking about a scenario I'll call "re-instancing an EBS volume":
EC2 box dies (root volume is EBS)
Force detach EBS volume
Boot new EC2 instance with the EBS volume
... That mostly works. The gotchas:
Doesn't protect against EBS failures, either total volume loss or an availability loss
Recovery time is O(minutes) assuming everything works just right
Your services need to be configured to restart automatically. It does no good to bring the box back if Nginx isn't running.
Your DNS routes or other services or whatever need to be ok with the IP-address changing. This can be worked around with ElasticIP.
How are your host SSH keys handled? Same name, new host key can break SSH-based automation when it gets the strong-warning for host-key-changed.
I don't have proof of this (other than seeing it happen once), but I believe that EC2/EBS _already_does_this_ automatically for boot-from-EBS instances
Again, the hard part here is on your plate. Can you stop your production service today and bring it up RELIABLY on a new instance? If so, the EC2 part of the story is really really easy.
As a side point:
All our user data is on Elastic Storage, so we're not too worried about losing anything.
I'd strongly suggest to regularly snapshot your EBS (Elastic Block Storage) to S3 if you are not doing that already.
You can use an autoscale group with a min/max/desired quantity of 1. Place the instance behind an ELB and have the autoscale group be triggered by the ELB healthy node count. This allows you to have built in monitoring by cloudwatch and the ELB health check. Anytime there is an issue the instance be replaced by the autoscale service.
If you have not checked 'Protect against accidental termination' you might want to do so.
Even if you have disabled 'Detailed Monitoring' for your instance you should still see the 'StatusCheckFailed' metric for your instance over which you can configure an alarm (In the CloudWatch dashboard)
Your application (hosted in a different server) should receive the alarm and start the instance using the AWS API (or CLI)
Since you have protected against accidental termination you would never need to spawn a new instance.