When I try to install an apt package in the Cloud Shell, I install it and it works fine. Next time I open the Cloud Shell, it is not there. How do I fix this issue?
According to the Cloud Shell documentation:
When you start Cloud Shell, it provisions a Compute Engine virtual machine running a Debian-based Linux operating system. Cloud Shell instances are provisioned on a per-user, per-session basis. The instance persists while your Cloud Shell session is active; after an hour of inactivity, your session terminates and its VM is discarded. For more information about those limitations you can check the Cloud Shell usage limits.
This means that Cloud Shell is only intended to interact with the Google Cloud Platform on a temporary basis during a limited time.
In case you want to configure a shell environment the right product to use would be a Compute Engine instance. Here is a quickstart that shows how to create a Linux virtual machine (VM) instance in Compute Engine.
While a Cloud Shell environment is ephemeral (i.e., it gets torn down after session end), your Cloud Shell home directory is persistent. You can add a customization script to your home directory to reconfigure your environment when Cloud Shell starts.
Related
Im trying to install Velero in a Kubernetes cluster on GCP, I have successfully installed it the first time but every time I logged back to the GCP shell the command is not available anymore, (as if the installation disappeared), I still can see my velero backups tho when I install it again
How can I persistently install any utility?
Bear in mind that Cloud Shell VM is ephemeral and is turned down when you are not using it interactively. See the documentation below:
The virtual machine instance that backs your Cloud Shell session is
not permanently allocated to a Cloud Shell session and terminates if
the session is inactive for an hour. After the instance is terminated,
any modifications that you made to it outside your $HOME are lost.
Also, I am not sure how exactly you have installed Valero in your GCP cluster but you might find this guide useful. It shows how to install Velero with HELM in GCP, step by step.
Basically, I need to run a set of custom shell scripts on ec2 instances to provision some software. Is there any workflow manager like oozie or airflow with api access to schedule the same. I am asking for alternatives like oozie and airflow, as those are that of hadoop environment schedulers and my environment is not. I can ensure that there can be ssh access from the source machine that will run the workflow manager and the ec2 instance where want to install the software. Is there any such open source workflow schedulers?
I would recommend using Cadence Workflow for your use case. There are multiple provisioning solutions built on top of it. For example Banzai Cloud Pipeline Platform
See the presentation that goes over Cadence programming model.
I have a script that I need to run once a day that requires a lot of memory. I would like to run it on a dedicated amazon box.
Is there some automated way to build a box, download all required software (like ruby) and then run my script. After the script is ran, I would like to shutdown the box.
The two options I can think of are:
I am thinking about hacking EMR to do this. (My script is a mapper against an empty directory)
Chef - This seemed like too much for one simple script.
You can accomplish setting up a new EC2 instance on startup using the official Ubuntu AMIs, the official Amazon Linux AMIs, and any other AMI that supports the concept of a user-data script.
Create a script (bash, Perl, Python,
whatever) that starts with #!
Pass this script as the user-data when running the EC2 instance.
The script will automatically be run as root on the first boot.
Here's the article where I introduced the concept of a user-data script:
Automate EC2 Instance Setup with user-data Scripts
http://alestic.com/2009/06/ec2-user-data-scripts
Your user-data script can install the required software, configure it, install your work script, and set up a cron job that runs the work script once a day.
ENHANCEMENT:
If the installation script don't take a long time to run (e.g., under an hour or few) then you don't even have to run a single dedicated instance 24 hours a day. You can instead use an approach that lets AWS start an instance for you on a regular schedule.
Here's an article I wrote that provides details on this approach with sample commands:
Running EC2 Instances on a Recurring Schedule with Auto Scaling
http://alestic.com/2011/11/ec2-schedule-instance
The general approach is to use Auto Scaling to start an instance with your user-data script on a regular schedule. Your job will terminate the instance when it has completed. They key is to suspend Auto Scaling's normal desire to re-start instances that terminate so that you don't pay for a running instance until the next time your job starts.
I want to run some scripts to run at the boot of the instance (server by node, and a redis database).
Since Amazon AMI is based on debian, I thought I could use update-rc.d to manage scripts.
Hwoever, when I type update-rc.d it says the command is not found.
What is the correct way of adding a service to the init script?
I know about CloudFront, but that is for the case when one wants to start up the instance for the first time and install some basic programs, right? For my case, I just want my instance to run some programs when it starts running from the reboot.
The image that I am using is amzn-ami-2011.09.2.x86_64.ext4.
I was searching for EC2 EBS storage Centos 5.4 AMI in the community AMI and eventually I found Rightscale AMI (I think they called it RightImage).
Now I have created instance using that AMI, but I found out there is some Rightscale stuff inside which is worrying me about the safety on using it. I found out there are the following files in that AMI:
/etc/init.d/rightimage
/etc/init.d/rightlink
/etc/init.d/rightscale
/home/ec2
/home/s3sync
(may be more other files I haven't found out yet)
I know I can look into the script and folder and see what they do, but since a lot of user here recommended using Rightscale Centos AMI in EC2, I hope may be there is already some gurus here know what those mentioned script and folder doing and could advice me
i)whether is it safe to delete them. (I'm more concern on whether my data in the server will be safe by using this AMI)
ii)any installed apps in RightScale AMI that should be deleted
And if you think there is other free EC2 Centos AMI that is secure and solid, do suggest as well, thanks !
In order for RightScale to properly manage instances in ec2 they use a ruby based daemon called RightLink as a communication device between their core platform and each instance that is launched. The init scripts that you saw are required for the instance to self configure itself to the point where it can be managed by RightScale properly.
/etc/init.d/rightimage is the first script that is run. Essentially it just determines the OS, arch version, and installs the correct RightLink package from the S3 bucket. Afterwards it kicks off the /opt/rightscale/bin/post_install.sh script which uses the OS init control tools to register the startup scripts to be invoked on future boots of the OS; this ensures that RightLink will always be started.
/etc/init.d/rightscale is the next script that is run. It initializes RightScale-specific (but not RightLink-specific) system state. It is responsible for caching launch settings (aka userdata) and metadata in /var/spool and installing any available patches to the RightLink agent.
/etc/init.d/rightlink is the final script that is run. It configures and enrolls the RightLink agent idempotently. If configuration and enrollment succeed, rightlink starts the sandboxed monit which starts the persistent agent process. If you're not launching the AMI using the RightScale platform this will never properly enroll because they aren't expecting it to, as such RightScale will have no communication with the instance at all.
Removing all three of these from the image shouldn't in any way harm the overall stability of the image, but from a security standpoint they shouldn't cause any problems if they are present.
If you have any further specific questions about it I'd suggest hopping on their forums at https://forums.rightscale.com/
You could also try #rightscale on freenode.