I'm trying to create an EC2 User-data script to run other scripts on boot up. However, the scripts that I run fail to recognize some commands and variables that I'd already declared. I'm running the commands as the "ubuntu" user but it still isn't working.
My user-data script looks something like this:
export user="ubuntu"
sudo su $user -c ". ./run_script"
Within the script, I have these lines:
THIS_PATH="/some/path"
echo "export SOME_PATH=$THIS_PATH" >> ~/.bashrc
source ~/.bashrc
However, the script can't run SOME_PATH/application, and echo $SOME_PATH this returns a blank line. I'm confused because $SOME_PATH/application works when I log into the EC2 using SSH and my debug logs using whoami returns "ubuntu."
Am I missing something here?
Your data script is executed as root and su command leaves $HOME and other ENV variables intact (note that sudo is redundant). "su -" does not help either
So, do not use ~ or $HOME but full path /home/ubuntu/.bashrc
I found out the problem. It seems that source ~/.bashrc isn't enough to restart the shell -- the environment variables worked after I referenced them in another bash script.
I am trying to execute the following user data script
sudo wget https://files.mysite.com/downloads/myFile.tar -P /opt
sudo tar -xvf /opt/myTar.tar -C /opt
sudo /opt/myFile.sh
When I execute the .sh file manually I see this:
Extracting...
Unpacking...
Cleaning up...
Complete
and it creates a directory in /opt/myDirectory
I do see the console in /var/log/cloud-init-output.sh but it doesn't seem to create the directory when run as part of the userdata script.
You're calling a shell script within a shell script, so you need to make sure:
have this as the first line of your ec2 user data script #!/bin/bash
call myFile.sh with the source command (alias is .) like this: . /opt/myFile.sh so it will run the myFile script
Note: the ec2 user data script runs as root so you do not need to have sudo each time you run a command.
Solution: CD into the directory first.
#!/bin/bash
cd /opt
wget https://example.com/myTAR.tar
tar -xvf myTAR.tar
/opt/mySH.sh
I usually use bash scripts to setup my environments (mostly aliases that interact with Docker), ie:
# ops-setup.sh
#!/bin/bash
PROJECT_NAME="my_awesome_project"
PROJECT_PATH=`pwd`/${BASH_SOURCE[0]}
WEB_CONTAINER=${PROJECT_NAME}"_web_1"
DB_CONTAINER=${PROJECT_NAME}"_db_1"
alias chroot_project="cd $PROJECT_PATH"
alias compose="chroot_project;COMPOSE_PROJECT_NAME=$PROJECT_NAME docker-compose"
alias up="compose up -d"
alias down="compose stop;compose rm -f --all nginx web python"
alias web_exec="docker exec -ti $WEB_CONTAINER"
alias db="docker exec -ti $DB_CONTAINER su - postgres -c 'psql $PROJECT_NAME'"
# ...
I'd like them to be run when I open the embedded terminal.
I tried Startup Tasks but they are not run in my terminal contexts.
Since I have a dedicated script for each of my projects, I can't run them from .bashrc or other.
How can I get my aliases automatically set at terminal opening ?
Today I'm running . ./ops-setup.sh manually each time I open a new embedded terminal.
You can create an alias in your .bashrc file like so:
alias ops-setup='bash --init-file <(echo '. /home/test/ops-setup.sh'; echo '. /home/test/.bashrc')'
If you call ops-setup, it will open up a new bash inside your terminal, and source .bashrc like it normally would, as well as your own script.
The only way I see to completely automate this is to modify the source code of your shell, e.g. bash, and recompile it. The files that are sourced are hardcoded into the source code.
I have a shell script which I want to run without using the "sh" or "bash" commands. For example:
Instead of: sh script.sh
I want to use: script.sh
How can I do this?
P.S. (i) I don't use shell script much and I tried reading about aliases, but I did not understand how to use them.
(ii) I also read about linking the script with another file in the PATH variables. I am using my university server and I don't have permissions to create a file in those locations.
Add a "shebang" at the top of your file:
#!/bin/bash
And make your file executable (chmod +x script.sh).
Finally, modify your path to add the directory where your script is located:
export PATH=$PATH:/appropriate/directory
(typically, you want $HOME/bin for storing your own scripts)
These are the prerequisites of directly using the script name:
Add the shebang line (#!/bin/bash) at the very top.
Use chmod u+x scriptname to make the script executable (where scriptname is the name of your script).
Place the script under /usr/local/bin folder.
Note: I suggest placing it under /usr/local/bin because most likely that path will be already added to your PATH variable.
Run the script using just its name, scriptname.
If you don't have access to /usr/local/bin then do the following:
Create a folder in your home directory and call it bin.
Do ls -lA on your home directory, to identify the start-up script your shell is using. It should be either .profile or .bashrc.
Once you have identified the start up script, add the following line:
PATH="$PATH:$HOME/bin"
Once added, source your start-up script or log out and log back in.
To source, put . followed by a space and then your start-up script name, e.g. . .profile or . .bashrc
Run the script using just its name, scriptname.
Just make sure it is executable, using chmod +x. By default, the current directory is not on your PATH, so you will need to execute it as ./script.sh - or otherwise reference it by a qualified path. Alternatively, if you truly need just script.sh, you would need to add it to your PATH. (You may not have access to modify the system path, but you can almost certainly modify the PATH of your own current environment.) This also assumes that your script starts with something like #!/bin/sh.
You could also still use an alias, which is not really related to shell scripting but just the shell, and is simple as:
alias script.sh='sh script.sh'
Which would allow you to use just simply script.sh (literally - this won't work for any other *.sh file) instead of sh script.sh.
In this example the file will be called myShell
First of all we will need to make this file we can just start off by typing the following:
sudo nano myShell
Notice we didn't put the .sh extension?
That's because when we run it from the terminal we will only need to type myShell in order to run our command!
Now, in nano the top line MUST be #!/bin/bash then you may leave a new line before continuing.
For demonstration I will add a basic Hello World! response
So, I type the following:
echo Hello World!
After that my example should look like this:
#!/bin/bash
echo Hello World!
Now save the file and then run this command:
chmod +x myShell
Now we have made the file executable we can move it to /usr/bin/ by using the following command:
sudo cp myShell /usr/bin/
Congrats! Our command is now done! In the terminal we can type myShell and it should say Hello World!
You have to enable the executable bit for the program.
chmod +x script.sh
Then you can use ./script.sh
You can add the folder to the PATH in your .bashrc file (located in your home directory).
Add this line to the end of the file:
export PATH=$PATH:/your/folder/here
You can type sudo install (name of script) /usr/local/bin/(what you want to type to execute said script)
ex: sudo install quickcommit.sh /usr/local/bin/quickcommit
enter password
now can run without .sh and in any directory
Add . (current directory) to your PATH variable.
You can do this by editing your .profile file.
put following line in your .profile file
PATH=$PATH:.
Just make sure to add Shebang (#!/bin/bash) line at the starting of your script and make the script executable(using chmod +x <File Name>).
Here is my backup script that will give you the idea and the automation:
Server: Ubuntu 16.04
PHP: 7.0
Apache2, Mysql etc...
# Make Shell Backup Script - Bash Backup Script
nano /home/user/bash/backupscript.sh
#!/bin/bash
# Backup All Start
mkdir /home/user/backup/$(date +"%Y-%m-%d")
sudo zip -ry /home/user/backup/$(date +"%Y-%m-%d")/etc_rest.zip /etc -x "*apache2*" -x "*php*" -x "*mysql*"
sudo zip -ry /home/user/backup/$(date +"%Y-%m-%d")/etc_apache2.zip /etc/apache2
sudo zip -ry /home/user/backup/$(date +"%Y-%m-%d")/etc_php.zip /etc/php
sudo zip -ry /home/user/backup/$(date +"%Y-%m-%d")/etc_mysql.zip /etc/mysql
sudo zip -ry /home/user/backup/$(date +"%Y-%m-%d")/var_www_rest.zip /var/www -x "*html*"
sudo zip -ry /home/user/backup/$(date +"%Y-%m-%d")/var_www_html.zip /var/www/html
sudo zip -ry /home/user/backup/$(date +"%Y-%m-%d")/home_user.zip /home/user -x "*backup*"
# Backup All End
echo "Backup Completed Successfully!"
echo "Location: /home/user/backup/$(date +"%Y-%m-%d")"
chmod +x /home/user/bash/backupscript.sh
sudo ln -s /home/user/bash/backupscript.sh /usr/bin/backupscript
change /home/user to your user directory and type: backupscript anywhere on terminal to run the script! (assuming that /usr/bin is in your path)
Enter "#!/bin/sh" before script.
Then save it as script.sh for example.
copy it to $HOME/bin or $HOME/usr/bin
The directory can be different on different linux distros but they end with 'bin' and are in home directory
cd $HOME/bin or $HOME/usr/bin
Type chmod 700 script.sh
And you can run it just by typing run.sh on terminal.
If it not work, try chmod +x run.sh instead of chmod 700 run.sh
Make any file as executable
Let's say you have an executable file called migrate_linux_amd64 and you want to run this file as a command like "migrate"
First test the executable file from the file location:
[oracle#localhost]$ ./migrate.linux-amd64
Usage: migrate OPTIONS COMMAND [arg...]
migrate [ -version | -help ]
Options:
-source Location of the migrations (driver://url)
-path Shorthand for -source=file://path
-database Run migrations against this database (driver://url)
-prefetch N Number of migrations to load in advance before executing (default 10)
-lock-timeout N Allow N seconds to acquire database lock (default 15)
-verbose Print verbose logging
-version Print version
-help Print usage
Commands:
goto V Migrate to version V
up [N] Apply all or N up migrations
down [N] Apply all or N down migrations
drop Drop everyting inside database
force V Set version V but don't run migration (ignores dirty state)
version Print current migration version
Make sure you have execute privileges on the file
-rwxr-xr-x 1 oracle oinstall 7473971 May 18 2017 migrate.linux-amd64
if not, run chmod +x migrate.linux-amd64
Then copy your file to /usr/local/bin. This directory is owned by root, use sudo or switch to root and perform the following operation
sudo cp migrate.linux-amd64 /usr/local/bin
sudo chown oracle:oracle /user/local/bin/migrate.linux.amd64
Then create a symbolic link like below
sudo ln /usr/local/bin/migrate.linux.amd64 /usr/local/bin/migrate
sudo chown oracle:oracle /usr/local/bin/migrate
Finally add /usr/local/bin to your path or user profile
export PATH = $PATH:/usr/local/bin
Then run the command as "migrate"
[oracle#localhost]$ migrate
Usage: migrate OPTIONS COMMAND [arg...]
migrate [ -version | -help ]
Options:
-source Location of the migrations (driver://url)
-path Shorthand for -source=file://path
-database Run migrations against this database (driver://url)
-prefetch N Number of migrations to load in advance before executing (default 10)
-lock-timeout N Allow N seconds to acquire database lock (default 15)
-verbose Print verbose logging
-version Print version
-help Print usage
Commands:
goto V Migrate to version V
up [N] Apply all or N up migrations
down [N] Apply all or N down migrations
drop Drop everyting inside database
force V Set version V but don't run migration (ignores dirty state)
version Print current migration version
Make the script file as executable by using file's properties
Create alias for the executable in ~/.bashrc. alias <alias namme> = <full script file path>'
refresh the user session to apply it. source ~/.bashrc
Just to add to what everyone suggested. Even with those solutions, the problem will persist if the user wants to execute the script as sudo
example:
chmod a+x /tmp/myscript.sh
sudo ln -s /tmp/myscript.sh /usr/local/bin/myscript
typing myscript would work but typing sudo myscript would return command not found.
As sudo you would have to still type sudo sh myscript or sudo bash myscript.
I can't think of a solution around this.
Just:
/path/to/file/my_script.sh
My bash script:
#!/bin/bash
cd /tmp
Before running my script:
pwd: /
After running my script:
pwd: /
After runnig my script trough sourcing it:
pwd: /tmp
How I can stay at the path from the script without sourcing it ?
You can't. Changes to the current directory only affect the current process.
Let me elaborate a little bit on this:
When you run the script, bash creates a new process for it, and changes to the current directory only affect that process.
When you source the script, the script is executed directly by the shell you are running, without creating extra processes, so changes to the current directory are visible to your main shell process.
So, as Ignacio pointed out, this can not be done
Ignacio is correct. However, as a heinous hack (totally ill advised and this really should get me at least 10 down votes) you can exec a new shell when you're done
#!/bin/bash
...
cd /
exec bash
Here's a silly idea. Use PROMPT_COMMAND. For example:
$ export PROMPT_COMMAND='test -f $CDFILE && cd $(cat $CDFILE) && rm $CDFILE'
$ export CDFILE=/tmp/cd.$$
Then, make the last line of your script be 'pwd > $CDFILE'
If you really need this behavior, you can make your script return the directory, then use it somehow. Something like:
#!/bin/bash
cd /tmp
echo $(pwd)
and then you can
cd $(./script.sh)
Ugly, but does the trick in this simple case.
You can define a function to run in the current shell to support this. E.g.
md() { mkdir -p "$1" && cd "$1"; }
I have the above in my ~/.bashrc