I'd like to use a GCE server and VNC to run some desktop programs I've written. However, I'd like to not have the GCE server running all the time (to save money) and I'd like to have the setup of my VNC desktop persisted (to save time). It seems like the easiest way to accomplish this is to somehow hibernate the GCE instance. Is there a way to do this either from a script or from the command line?
The only currently supported technique that I'm aware of would be to use a bootable persistent disk, which outlives the virtual machine so you could delete and restart the VM with moderately quick (under 30s) startup time but I don't think that would meet your needs because unless your app has built-in, robust checkpoint/resume logic, you might not be able to pick up where you left off on the desktop.
Another way to approach this would be to use a lightweight container, like Docker, which has suspend/resume and checkpointing built into its design. You can't currently run Docker containers on Google Compute Engine but we're interested in exploring the potential of Docker on GCE so stay tuned.
Sounds like what you'd really like is a native suspend capability with near instantaneous resume and no or very low charge for VMs in the suspended state. I think that's an interesting idea - I've submittted an internal feature request to the Compute Engine engineering team for future consideration.
Compute Engine has alpha support for suspending virtual machines. A quote from that documentation:
gcloud alpha compute instances suspend is used to suspend a Google Compute Engine virtual machine. Suspending a VM is the equivalent of sleep or standby mode: the guest receives an ACPI S3 suspend signal, after which all VM state is saved to temporary storage. An instance can only be suspended while it is in the RUNNING state. A suspended instance will be put in SUSPENDED state.
The command is not yet available in the Compute Engine web UI but can be invoked from the command line like:
gcloud alpha compute instances suspend my-vm
Related
I want to create an Azure Virtual Machine that I only need to run approximately for 1 or 2 hours once or twice per day. I don't want to pay for the server when I'm not using it.
I know I can just go to the dashboard and shut it off, but I often forget to do so, I'm getting senile! I would like for a timer to start when the system (Windows 10) is started, and when the timer reaches zero, the image is made inactive (no charges incurred) unless I request more time.
Any ideas would be appreciated.
The dashboard has an automatic shutdown feature. This can also be configured to send a webhook notification with a link which will delay the shutdown. Currently in-machine notifications are not supported.
Although there are many tools like PowerShell scripts and apps that you could use from within the VM to trigger automatic shutdown, there are some billing gotchas to be aware of.
With an Azure VM, you pay per second and cease paying only when the machine is completely deallocated and no longer reserving memory and cores on the platform. (There is still a nominal storage charge associated with storage of the VM image). You cannot deallocate the VM from within itself.
To ensure the VM isn't incurring charges, check that the status is 'Stopped (deallocated').
I've got the following totally reproducible scenario, which I'm unable to understand:
There is a very simple application, which does nothing else than calling CreateObject("Word.Application"), that is creating an instance of MS Word for COM interop. This application is located on a Windows Terminal Server. The test case is to connect via RDP, execute the application and the application will output the time taken for the CreateObject call.
The problem now is that the execution time is significantly longer, if I connect from a specific notebook (HP Spectre): It takes 1,7s (+/- 0.1s).
If I connect from any other machine (notebook or desktop computer), then the execution time is between 0,2-0,4s.
The execution times don't depend on the used RDP account, or screen resolution, or local printers. I even did a fresh install of Windows on that HP notebook to rule out any other side-effects. It doesn't matter if the HP notebook is connected via WLAN or an USB network card. I'm at a loss understanding the 4x to 8x execution time difference to any other machine.
Which reason (component/setting) could explain this big difference in execution time?
Some additional information: I tried debugging the process using an API monitor and could see that >90% of the execution time is actually being spent between a call to RpcSend and RpcReceive. Unfortunately I can't make sense of this information.
It could be the credential management somehow being in the way.
Open the .rdp file with notepad and add
enablecredsspsupport:i:0
This setting determines whether RDP will use the Credential Security Support Provider (CredSSP) for authentication if it is available
Related documentation
https://learn.microsoft.com/en-us/previous-versions/windows/it-pro/windows-server-2008-R2-and-2008/ff393716%28v%3dws.10%29
According to your information about RpcSend and RpcReceive time consumption, it could be the case you have some service stopped on your client machine, like DCOM server or some other COM-related (they usually have "COM" or "transaction" in their names).
Some of that services could be started/stopped (if Manually mode selected) by system to/after transfer your request, but there is a time delay to starting service.
I suggest you to open Computer Management - Services or run -> services.msc and compare COM-related services running on your "slow" client and on your "fast" clients, and try to set Automatically running instead Manually or Disabled.
Also, try to run API Monitor on such processes to determine the time-consuming place more precisely.
I would like to use a EC2 instance for accounting purpose as the instance is required to be accessed by multiple party. Since EC2 charges are per hour basis. I am planning to turn on the instance only when some one needs to use then turn if off. The instance is not required to be live 24x7.
Start the instance with a mouse click. (maybe some script or panel or url.
Remote Desktop Connect- My accountant finishes the work.
User shutdown the instance or auto shutdown after 30 mins of inactivity.
How do we go about it ?
Thanks,
Dwija
You can Start an Amazon EC2 instance that was previously Stopped by sending a StartInstance command. This can be done from many different programming languages, or via the AWS Command-Line Interface (CLI) (available for Windows, Mac and Linux).
Turning off the instance when it is not in use is harder because you need to identify when it is appropriate to turn it off. You could have a program on the instance itself (much like a screensaver) that detects when the system is 'idle' and issues a Shutdown command on the virtual machine (just like Choosing Shutdown from the Start menu). This will Stop the Amazon EC2 instance.
An alternative, depending on your use-case, would be to use Amazon Workspaces, which is a virtual desktop -- a computer that is running in the cloud that you can connect with from Windows, Mac, iOS or Android devices. It is charged either on a monthly subscription or on a combination of subscription + hourly charge. It automatically turns off after the user has disconnected for a period of time. An Amazon Workspace is typically used by one person as their 'personal' computer, but it could be shared amongst other users.
I've browsed through other questions on SO concerning backing up EC2, and it has provided me with a good basis, but I'm still a bit confused as to how I should approach my solution and develop a contingency plan. Most questions are fairly specific, but I have a pretty vanilla setup, and I think this information will be beneficial to future users. Let me provide my basic setup:
Basic small instance
Pushing files to S3
Running MongoDB
Running nginx
Now, due to the ephemeral nature of EC2, it's apparent that I need to bind my EC2 instance to EBS to ensure persistent storage. The reason I'm attempting to develop a contingency plan is that I'm worried my instance may disappear at any time (due to outages, etc). If my instance were to disappear, I'm concerned that I would have to spin up a new instance and reinstall all my applications before getting everything up an running again. A few questions:
How do I backup my instance to ensure that if it were to disappear, I could quickly bring it back up (preferably without having to reinstall all previous software)? I don't need a series of backups, just the previous days (or weeks) backup to ensure that a previous working version exists that can be started quickly.
If I use EBS instead of instance storage, it essentially acts in place of my hard drive right? So if I have MongoDB installed, I'm assuming the database it is writing to cab placed on EBS?
If I go with the small instance with 160 GB of storage and I use EBS, would I need to allocate 160 GB of EBS out of the gate, or is the 160 GB for instance storage only?
So, in summary, I would like a solution (either manual or automatic) that can create a snapshot of my EC2 instance to ensure that if it were to disappear, it could be reconstructed without having to spend the time to manually reconstruct everything.
In an ideal world, if my instance were to disappear, I could spin up an version of my instance with everything intact (to a point where things were backed up). Any resources or suggestions? Thanks in advance.
OK, here we go:
For backups:
Create your instance from one of the stock AWS images. Make sure it is an EBS-backed VM - depending on the size of the VM you pick, you'll get a volume assigned with 'n' GB of space, attached as the boot volume (/dev/sda1).
Configure the VM with whatever software you need, apply patches, tune for disk fragmentation, CPU consumption (task priorities, etc), and any other configuration you need that makes the VM tailored to your requirements.
Stop the VM and take a snapshot of the EBS volume, then restart it (re-assigning the Elastic IP is there is one). This is your backup snapshot - repeat as desired at whatever frequency you like. Remember to stop the VM when you snap it to prevent the OS writing to the volume whilst you're taking a copy of it.
For recovery:
Your VM will fail, eventually. You'll break something and render it damaged or inoperative, or the hardware it's running on will suffer a fault. It will happen.
When it does, terminate it (if it didn't self-terminate) and spin-up a new VM of the same type from the AWS stock list. Wait until it shows as 'Running', and then stop it.
Detach its EBS volume and delete it.
Create a new EBS volume from whichever backup snapshot you last created, and attach that new volume to the VM as /dev/sda1.
Start the VM and assign your EIP if appropriate.
About EBS storage:
It's a chunk of storage space. If you format it to look like a standard disk, you can use it exactly as you would a physical disk. Install stuff on it, point software at it for use as storage space, whatever.
you have two options: (BUT NOT EXACTLY AS YOU WANT ;( )
1- Have an 'external' EBS attached into your EC2 instance, and manually (you can do automatically through cronjobs), make snapshots from it ! But it's not what you want, why ? If your EC2 instance disappears, you will need to re-create all your environment again and re-attach your EBS... So it's a nice way of having backups of HUGE datas on your EC2, but your enviroment is destructed...
2- The best way, but not so perfect, is after you finished configuring your EC2, make a private AMI from it, so anytime you want you can launch more instances like that , from that AMI, so everything is cloned.... BUT the worst part of it, is that every time you change a configuration from the instance, you still need to make a new AMI, and every time you make a new AMI, you need to reboot your instance to grant data integrity on your new private AMI !
I recommend you to take a closer look into RESERVED EC2 instances, that have a better stability from normal ones. BUT YOU STILL CAN HAVE HARDWARE DISASTERS as normal instances too...
We're running a lightweight web app on a single EC2 server instance, which is fine for our needs, but we're wondering about monitoring and restarting it if it goes down.
We have a separate non-Amazon server we'd like to use to monitor the EC2 and start a fresh instance if necessary and shut down the old one. All our user data is on Elastic Storage, so we're not too worried about losing anything.
I was wondering if anyone has any experience of using EC2 in this way, and in particular of automating the process of starting the new instance? We have no problem creating something from scratch, but it seems like it should be a solved problem, so I was wondering if anyone has any tips, links, scripts, tutorials, etc to share.
Thanks.
You should have a look at puppet and its support for AWS. I would also look at the RightScale AWS library as well as this post about starting a server with the RightScale scripts. You may also find this article on web serving with EC2 useful. I have done something similar to this but without the external monitoring, the node monitored itself and shut down when it was no longer needed then a new one would start up later when there was more work to do.
Couple of points:
You MUST MUST MUST back up your Amazon EBS volume.
They claim "better" reliability, but not 100%, and it's SEVERAL orders of magnitude off of S3's "12 9's" of durability. S3 durability >> EBS durability. That's a fact. EBS supports a "snapshots" feature which backs up your storage efficiently and incrementally to S3. Also, with EBS snapshots, you only pay for the compressed deltas, which is typically far far less than the allocated volume size. In another life, I've sent lost-volume emails to smaller customers like you who "thought" that EBS was "durable" and trusted it with the only copy of a mission-critical database... it's heartbreaking.
Your Q: automating start-up of a new instance
The design path you mention is relatively untraveled; here's why... Lots of companies run redundant "hot-spare" instances where the second instance is booted and running. This allows rapid failover (seconds) in the event of "failure" (could be hardware or software). The issue with a "cold-spare" is that it's harder to keep the machine up to date and ready to pick up where the old box left off. More important, it's tricky to VALIDATE that the spare is capable of successfully recovering your production service. Hardware is more reliable than untested software systems. TEST TEST TEST. If you haven't tested your fail-over, it doesn't work.
The simple automation of starting a new EBS instance is easy, bordering on trivial. It's just a one-line bash script calling the EC2 command-line tools. What's tricky is everything on top of that. Such a solution pretty much implies a fully 100% automated deployment process. And this is all specific to your application. Can your app pull down all the data it needs to run (maybe it's stored in S3?). Can you kill you instance today and boot a new instance with 0.000 manual setup/install steps?
Or, you may be talking about a scenario I'll call "re-instancing an EBS volume":
EC2 box dies (root volume is EBS)
Force detach EBS volume
Boot new EC2 instance with the EBS volume
... That mostly works. The gotchas:
Doesn't protect against EBS failures, either total volume loss or an availability loss
Recovery time is O(minutes) assuming everything works just right
Your services need to be configured to restart automatically. It does no good to bring the box back if Nginx isn't running.
Your DNS routes or other services or whatever need to be ok with the IP-address changing. This can be worked around with ElasticIP.
How are your host SSH keys handled? Same name, new host key can break SSH-based automation when it gets the strong-warning for host-key-changed.
I don't have proof of this (other than seeing it happen once), but I believe that EC2/EBS _already_does_this_ automatically for boot-from-EBS instances
Again, the hard part here is on your plate. Can you stop your production service today and bring it up RELIABLY on a new instance? If so, the EC2 part of the story is really really easy.
As a side point:
All our user data is on Elastic Storage, so we're not too worried about losing anything.
I'd strongly suggest to regularly snapshot your EBS (Elastic Block Storage) to S3 if you are not doing that already.
You can use an autoscale group with a min/max/desired quantity of 1. Place the instance behind an ELB and have the autoscale group be triggered by the ELB healthy node count. This allows you to have built in monitoring by cloudwatch and the ELB health check. Anytime there is an issue the instance be replaced by the autoscale service.
If you have not checked 'Protect against accidental termination' you might want to do so.
Even if you have disabled 'Detailed Monitoring' for your instance you should still see the 'StatusCheckFailed' metric for your instance over which you can configure an alarm (In the CloudWatch dashboard)
Your application (hosted in a different server) should receive the alarm and start the instance using the AWS API (or CLI)
Since you have protected against accidental termination you would never need to spawn a new instance.