I have a cloud-init bash script that pulls a zip from S3, unzips it, and then runs an Ansible playbook contained in it. Now, I also have environment variables baked into /etc/environment, and the Ansible playbook uses lookup('env') to grab those values. It works fine when run through bash, either as the main user or as root. But when it fires via cloud-init, the variables are not transferred through.
In my bash script, the first line is source /etc/environment, and I can echo them out just fine. It's only when the Ansible playbook does the lookup that it fails. Interestingly, I can force the variables as so:
FOO=$FOO BAR=$BAR ansible-playbook -c local ...
and that works. Does anyone have any idea how I can get around having to hardcode the variables into the playbook line, and just have them work as expected, i.e. pull from /etc/environment?
Edit: here's the cloud-init:
#!/bin/bash
source /etc/environment
doit() {
aws s3 cp s3://my/scripts/dev-s3-push.tar.gz /tmp/my.tar.gz
mkdir -p /app/deploy
tar -C /app/deploy -zxvf /tmp/my.tar.gz
cd /app/deploy
FOO=$FOO BAR=$BAR ansible-playbook -i "localhost," -c local run.yml
}
doit
This is added into the User Data section in AWS.
Okay, so I figured it out. lookup() uses os.getenv underneath, and I found a few other questions related to os.getenv not returning properly.
The issue is that in my /etc/environment, i had it as FOO=bar, where it should have been export FOO=bar. Changing all the values over to that makes it work. I still have the source line in the cloud-init function, but I think this is solved now.
Related
I am running a bash script with sudo and have tried the below but am getting the error below using aws cp. I think the problem is that the script is looking for the config in /root which does not exist. However doesn't the -E preserve the original location? Is there an option that can be used with aws cp to pass the location of the config. Thank you :).
sudo -E bash /path/to/.sh
- inside of this script is `aws cp`
Error
The config profile (name) could not be found
I have also tried `export` the name profile and `source` the path to the `config`
You can use the original user like :
sudo -u $SUDO_USER aws cp ...
You could also run the script using source instead of bash -- using source will cause the script to run in the same shell as your open terminal window, which will keep the same env together (such as user) - though honestly, #Philippe answer is the better, more correct one.
I'm trying to create an EC2 User-data script to run other scripts on boot up. However, the scripts that I run fail to recognize some commands and variables that I'd already declared. I'm running the commands as the "ubuntu" user but it still isn't working.
My user-data script looks something like this:
export user="ubuntu"
sudo su $user -c ". ./run_script"
Within the script, I have these lines:
THIS_PATH="/some/path"
echo "export SOME_PATH=$THIS_PATH" >> ~/.bashrc
source ~/.bashrc
However, the script can't run SOME_PATH/application, and echo $SOME_PATH this returns a blank line. I'm confused because $SOME_PATH/application works when I log into the EC2 using SSH and my debug logs using whoami returns "ubuntu."
Am I missing something here?
Your data script is executed as root and su command leaves $HOME and other ENV variables intact (note that sudo is redundant). "su -" does not help either
So, do not use ~ or $HOME but full path /home/ubuntu/.bashrc
I found out the problem. It seems that source ~/.bashrc isn't enough to restart the shell -- the environment variables worked after I referenced them in another bash script.
I'd like to ask how to show the output of debug as if I used register using the ansible command, not play a playbook.
For an example, I just to do below, and do a playbook:
$ ansible -m raw -a "df -h"
Thank you.
I assume you want to see the output of the command, so instead of using the module raw, use the shell module -m shell and it will work.
If you don't specify a module, ansible uses the module command so it will works also.
The module shell is missing:
$ ansible -m shell raw -a "df -h"
I can run a Python script on a remote machine like this:
ssh -t <machine> python <script>
And I can also set environment variables this way:
ssh -t <machine> "PYTHONPATH=/my/special/folder python <script>"
I now want to append to the remote PYTHONPATH and tried
ssh -t <machine> 'PYTHONPATH=$PYTHONPATH:/my/special/folder python <script>'
But that doesn't work because $PYTHONPATH won't get evaluated on the remote machine.
There is a quite similar question on SuperUser and the accepted answer wants me to create an environment file which get interpreted by ssh and another question which can be solved by creating and copying a script file which gets executed instead of python.
This is both awful and requires the target file system to be writable (which is not the case for me)!
Isn't there an elegant way to either pass environment variables via ssh or provide additional module paths to Python?
How about using /bin/sh -c '/usr/bin/env PYTHONPATH=$PYTHONPATH:/.../ python ...' as the remote command?
EDIT (re comments to prove this should do what it's supposed to given correct quoting):
bash-3.2$ export FOO=bar
bash-3.2$ /usr/bin/env FOO=$FOO:quux python -c 'import os;print(os.environ["FOO"])'
bar:quux
WFM here like this:
$ ssh host 'grep ~/.bashrc -e TEST'
export TEST="foo"
$ ssh host 'python -c '\''import os; print os.environ["TEST"]'\'
foo
$ ssh host 'TEST="$TEST:bar" python -c '\''import os; print os.environ["TEST"]'\'
foo:bar
Note the:
single quotes around the entire command, to avoid expanding it locally
embedded single quotes are thus escaped in the signature '\'' pattern (another way is '"'"')
double quotes in assignment (only required if the value has whitespace, but it's good practice to not depend on that, especially if the value is outside your control)
avoiding of $VAR in command: if I typed e.g. echo "$TEST", it would be expanded by shell before replacing the variable
a convenient way around this is to make var replacement a separate command:
$ ssh host 'export TEST="$TEST:bar"; echo "$TEST"'
foo:bar
I'm asking myself why Ansible doesn't source ~/.profile file before execute template module on one host ?
Distant host ~/.profile:
export ENV_VAR=/usr/users/toto
A single Ansible task:
- template: src=file1.template dest={{ ansible_env.ENV_VAR }}/file1
Ansible fail with:
fatal: [distant-host] => One or more undefined variables: 'dict object' has no attribute 'ENV_VAR'
Ansible is not running remote tasks (command, shell, ...) in an interactive nor login shell. It's same like when you execute command remotely via 'ssh user#host "which python"'
To source ~/.bashrc won't work often because ansible shell is not interactive and ~/.bashrc implementation by default ignores non interactive shell (check its beginning).
The best solution for executing commands as user after its ssh interactive login I found is:
- hosts: all
tasks:
- name: source user profile file
#become: yes
#become_user: my_user # in case you want to become different user (make sure acl package is installed)
shell: bash -ilc 'which python' # example command which prints
register: which_python
- debug:
var: which_python
bash: '-i' means interactive shell, so .bashrc won't be ignored
'-l' means login shell which sources full user profile (/etc/profile and ~/.bash_profile, or ~/.profile - see bash manual page for more details)
Explanation of my example: my ~/.bashrc sets specific python from anaconda installed under that user.
Ansible is not running tasks in an interactive shell on the remote host. Michael DeHaan has answered this question on github some time ago:
The uber-basic description is ansible isn't really doing things through the shell, it's transferring modules and executing scripts that it transfers, not using a login shell.
i.e. Why does an SSH remote command get fewer environment variables then when run manually?
It's not a continous shell environment basically, nor is it logging in and typing commands and things.
You should see the same result (undefined variable) by running this:
ssh <host> echo $ENV_VAR
In a lot of places I've used below structure:
- name: Task Name
shell: ". /path/to/profile;command"
when ansible escalates the privilige to sudo it don't invoke the login shell of sudo user
we need to make changes in the way we call sudo like invoking it with -i and -H flags
"sudo_flags=-H" in your ansible.cfg file
If you can run as root, you can use runuser.
- shell: runuser -l '{{ install_user }}' -c "{{ cmd }}"
This effectively runs the command as install_user in a fresh login shell, as if you had used su - *install_user* (which loads the profile, though it might be .bash_profile and not .profile...) and then executed *cmd*.
I'd try not to run everything as root just so you can run it as someone else, though...
If you can modify the configuration of your target host and don't want to change your ansible yaml code. You can try this:
add the variable ENV_VAR=/usr/users/toto into /etc/environment file rather than ~/.profile.
shell: "bash -l scala -version"
by using bash -l will allow ansible to load corresponding bash_profile.
bash: '-i' (interactive shell) won't allow the ansible to run other task.
add the variable ENV_VAR=/usr/users/toto into /etc/environment file rather than ~/.profile.
You really can use /etc/environment, but only if a variable has a static value. If we use variable which gets the value of another variable it doesn't work. For example, if we put this line to /etc/environment
XDG_RUNTIME_DIR=/run/user/$(id -u)
Ansible can see exactly XDG_RUNTIME_DIR=/run/user/$(id -u), not XDG_RUNTIME_DIR=/run/user/1012.
And if we put this line to ~/.bash_profile or ~/.bashrc:
export XDG_RUNTIME_DIR=/run/user/$(id -u)
User can see XDG_RUNTIME_DIR=/run/user/1012 (if user's id is 1012) when he works manually, but Ansible doesn't get variable XDG_RUNTIME_DIR at all.