I was trying to use user data section to initialize EC2 and run some bash scripts on instance startup. Somehow all the scripts are running except the (*) statement. Does anyone know what is happening?
#!/bin/bash
cd /home/ubuntu
apt-get update -y
apt-get -y install nodejs npm build-essential libssl-dev awscli ruby2.0
curl https://raw.githubusercontent.com/creationix/nvm/v0.25.0/install.sh > 123
curl https://raw.githubusercontent.com/creationix/nvm/v0.25.0/install.sh | bash (*)
Related
I would like to know what are the alternatives to using curl ... | bash ... to upgrade a package to a given version in a Dockerfile. It has been found that someone placed a backdoor in Codecov Bash Uploader, which would allow an attacker to export info from a target's CI environment. I am on RHEL 8 and for the moment, my Dockerfile uses something like:
RUN curl -sL <script-url> | bash - && \
yum remove -y <package> && \
yum install -y <package> && \
...
I started with dockerizing my python hello world program, and the dockerfile looks something like this.
FROM ubuntu:16.04
MAINTAINER Bhavani Ravi
RUN apt-get update
RUN apt-get install -y software-properties-common vim
RUN add-apt-repository ppa:jonathonf/python-3.6
RUN apt-get update
RUN apt-get install -y build-essential python3.6 python3.6-dev python3-pip python3.6-venv
RUN apt-get install -y git
# update pip
RUN python3.6 -m pip install pip --upgrade
RUN python3.6 -m pip install wheel
RUN echo 'alias python=python3.6' >> ~/.bash_aliases
COPY hello.py .
ENTRYPOINT python hello.py
The problem is now when I run the image I get /bin/sh: 1: python: not found.
As you can see in the dockerfile, I have set up bash_alias. When I override entrypoint and run the image with /bin/bash and then use python command it works.
How to make the alaias work for all command environments
Can you run dockerfile commands in bash instead of shell
Maybe try wrapping each command in /bin/bash -c "yourcommand" if bash is available on your Docker image it will force it.
Example
RUN /bin/bash -c "apt-get update"
...
ENTRYPOINT /bin/bash -c "python hello.py"
I am running this code while launching ec2 instance, python is installed, but the folder is not created.
#!/bin/bash
sudo yum update -y
sudo yum install python36 -y
mkdir venv
cd venv
virtualenv -p /usr/bin/pyton3.6 python36
echo "source /home/ec2-user/venv/python36/bin/activate" > /home/ec2-user/.bashrc
pip install boto3
A couple of things could go wrong with that script. I suggest a more robust way to write it:
#!/bin/bash
cd "$(dirname "$0")"
sudo yum update -y
sudo yum install python36 -y
if [ ! -d venv ]; then
mkdir venv
virtualenv -p /usr/bin/pyton3.6 venv/python36
echo "source venv/python36/bin/activate" >> ~/.bashrc
source venv/python36/bin/activate
pip install boto3
fi
Improved points:
Make sure we are in the right directory, by doing a cd into the directory of the script
Do not hardcode the user home directory location, use ~
Do not truncate ~/.bashrc if already exists
Before installing boto3, it's important to activate the virtual env, otherwise pip will not install it inside the virtual env (it will try to install system-wide)
Thank you for inputs. This worked.
Mainly:
clear paths
activate virtual environment for boto3 install
'#!/bin/bash
sudo yum update -y
sudo yum install python36 -y
mkdir /home/ec2-user/venv
cd /home/ec2-user/venv
virtualenv -p /usr/bin/python3.6 python36
echo "source /home/ec2-user/venv/python36/bin/activate" >> /home/ec2-user/.bashrc
source /home/ec2-user/venv/python36/bin/activate
pip install boto3
When I try to
apt-get install -y tzdata
the command line option for picking timezone shows up. I am trying to use this in a script to do some setup, how can I make the apt-get run without user input?
I know to reconfigure the tzdata I can do
echo "America/New_York" > /etc/timezone
dpkg-reconfigure -f noninteractive tzdata
But when installing I need it to run fully even if it doesn't set the right timezone, I can always reconfigure it.
I tried
echo 5 | apt-get install -y tzdata
but it is not working as expected.
This is the script I used
(Updated Version with input from #elquimista from the comments)
#!/bin/bash
ln -fs /usr/share/zoneinfo/America/New_York /etc/localtime
DEBIAN_FRONTEND=noninteractive apt-get install -y tzdata
dpkg-reconfigure --frontend noninteractive tzdata
Seems to work fine.
As one liner:
DEBIAN_FRONTEND=noninteractive apt-get install -y --no-install-recommends tzdata
If someone wants to achieve it in Dockerfile, use as below.
RUN DEBIAN_FRONTEND=noninteractive apt-get -y install tzdata
To avoid playing directly with symlinks and to run configuration only once, I suggest to use debconf-set-selections command:
echo 'tzdata tzdata/Areas select Europe' | debconf-set-selections
echo 'tzdata tzdata/Zones/Europe select Paris' | debconf-set-selections
DEBIAN_FRONTEND="noninteractive" apt install -y tzdata
I have recently found the following solution in a Dockerfile building the Cingulata FHE library:
ln -snf /usr/share/zoneinfo/$(curl https://ipapi.co/timezone) /etc/localtime
It basically uses the API provided by ipapi.co to retrieve the timezone information. This automatically configures the timezone properly instead of skipping the dialog and using the default (UTC).
All credit for this should go to #PYA but the right order should be:
ln -fs /usr/share/zoneinfo/America/New_York /etc/localtime
export DEBIAN_FRONTEND=noninteractive
apt-get install -y tzdata
dpkg-reconfigure --frontend noninteractive tzdata
Here is how I did it:
echo 1 > input.txt
echo 1 >> input.txt
apt-get install -y tzdata < input.txt
ln -fs /usr/share/zoneinfo/America/Los_Angeles /etc/localtime
echo America/Los_Angeles > /etc/timezone
The first two echo statements create a text file that contains the selection numbers for the geographic area menu and the city/region menu. This file is then used to provide input to the apt-get install command. The tzdata package will be installed without asking for any user input. The timezone will be set to Africa/Abidjan as if you entered 1 and 1 in response to the prompts you would normally get. Then I change the timezone to what I want with the last two commands.
Instead of 1 and 1, you could use the actual numbers for the geographic area and city/region that you want, but it seems to me that those numbers could change.
here is what worked for me:
from ubuntu:bionic
ARG DEBIAN_FRONTEND=noninteractive
RUN apt-get update && apt-get install -y tzdata
RUN unlink /etc/localtime
RUN ln -s /usr/share/zoneinfo/America/New_York /etc/localtime
After reading the comments, I did two steps below to use the TZ environment variable:
Added the following to the Dockerfile
RUN DEBIAN_FRONTEND=noninteractive apt-get install -y --no-install-recommends tzdata && apt-get clean
Added the following to the docker CMD script:
if [ ! -z "${TZ}" ]; then
echo "${TZ}" > /etc/timezone
dpkg-reconfigure -f noninteractive tzdata
fi
This worked for me and allowed me to set the time zone when starting the container.
I'm trying to setup Vagrant virtual machines to support my learning through Seven Databases in Seven Weeks. I'm provisioning software using basic shell scripts which performs appropriate actions within a sudo environment. However, I'm using the vagrant user to run the tutorials, and would like the provisioning to install the appropriate node / NPM modules as Vagrant, rather than through sudo.
My current npm command is the last line in this provisioning script, but the module is unavailable when vagrant tried to execute node scripts.
apt-get update
apt-get -y install build-essential
apt-get -y install tcl8.5
wget http://redis.googlecode.com/files/redis-2.6.0-rc3.tar.gz
tar xzf redis-2.6.0-rc3.tar.gz
cd redis-2.6.0-rc3
make
make install
make test
mkdir /etc/redis
mv redis.conf /etc/redis/redis.conf
sed -i.bak 's/127.0.0.1/0.0.0.0/g' /etc/redis/redis.conf
sed -i.bak 's/daemonize no/daemonize yes/g' /etc/redis/redis.conf
sed -i.bak 's/dir .\//dir \/var\/lib\/redis/g' /etc/redis/redis.conf
cd src/
wget https://raw.github.com/gist/1053791/880a4a046e06028e160055406d02bdc7c57f3615/redis-server
mv redis-server.1 /etc/init.d/redis-server
mv redis-cli /etc/init.d/redis-cli
chmod +x /etc/init.d/redis-server
sed -i.bak 's/DAEMON=\/usr\/bin\/redis-server/DAEMON=\/usr\/local\/bin\/redis-server/g' /etc/init.d/redis-server
useradd redis
mkdir -p /var/lib/redis
mkdir -p /var/log/redis
chown redis.redis /var/lib/redis
chown redis.redis /var/log/redis
update-rc.d redis-server defaults
/etc/init.d/redis-server start
cd /etc/init.d/
echo ./redis-cli
echo http://blog.hemantthorat.com/install-redis-2-6-on-ubuntu/
apt-get -y install python-software-properties python g++ make
add-apt-repository -y ppa:chris-lea/node.js
apt-get update
apt-get -y install nodejs
npm install hiredis redis csv
Simply set privileged to false in your VagrantFile like this:
Vagrant.configure(VAGRANTFILE_API_VERSION) do |config|
...
config.vm.provision :shell, privileged: false, path: "script.sh"
...
end
The shell provision runs as the root user. If you with to run as the vagrant user, you can do something like this:
sudo -u vagrant npm install hiredis redis
..or for multiple lines:
sudo -u vagrant << EOF
[...]
npm install hiredis
npm install redis
EOF
Maybe use npm install -g to install it globally in the vm?
sed -i 's/.*requiretty$/Defaults !requiretty/' /etc/sudoers