How to properly run entrypoint bash script on docker? - bash

I would like to build a docker image for dumping large SQL Server tables into S3 using the bcp tool by combining this docker and this script. Ideally I could pass table, database, user, password and s3 path as arguments for the docker run command.
The script looks like
#!/bin/bash
TABLE_NAME=$1
DATABASE=$2
USER=$3
PASSWORD=$4
S3_PATH=$5
# read sqlserver...
# write to s3...
# .....
And the Dockerfile is:
# SQL Server Command Line Tools
FROM ubuntu:16.04
LABEL maintainer="SQL Server Engineering Team"
# apt-get and system utilities
RUN apt-get update && apt-get install -y \
curl apt-transport-https debconf-utils \
&& rm -rf /var/lib/apt/lists/*# SQL Server Command Line Tools
FROM ubuntu:16.04
LABEL maintainer="SQL Server Engineering Team"
# apt-get and system utilities
RUN apt-get update && apt-get install -y \
curl apt-transport-https debconf-utils \
&& rm -rf /var/lib/apt/lists/*
# adding custom MS repository
RUN curl https://packages.microsoft.com/keys/microsoft.asc | apt-key add -
RUN curl https://packages.microsoft.com/config/ubuntu/16.04/prod.list > /etc/apt/sources.list.d/mssql-release.list
# install SQL Server drivers and tools
RUN apt-get update && ACCEPT_EULA=Y apt-get install -y msodbcsql mssql-tools awscli
RUN echo 'export PATH="$PATH:/opt/mssql-tools/bin"' >> ~/.bashrc
RUN /bin/bash -c "source ~/.bashrc"
ADD ./sql2sss.sh /opt/mssql-tools/bin/sql2sss.sh
RUN chmod +x /opt/mssql-tools/bin/sql2sss.sh
RUN apt-get -y install locales
RUN locale-gen en_US.UTF-8
RUN update-locale LANG=en_US.UTF-8
ENTRYPOINT ["/opt/mssql-tools/bin/sql2sss.sh", "DB.dbo.TABLE", "SQLSERVERDB", "USER", "PASSWORD", "S3PATH"]
If I replae the entrypoint for CMD /bin/bash and run the image with -it, I can manually run the sql2sss.sh and it works properly, reading and writing to s3. However if I try to use the entrypoint as shown yelds bcp: command not found.
I also noticed if I use CMD /bin/sh in iterative mode it will produce the same error. Am I missing some configuration in order for the entrypoint to run the script properly?

Have you tried
ENV PATH="/opt/mssql-tools/bin:${PATH}"
Instead of exporting the bashrc?
As David Maze pointed out docker doesn't read dot files
Basically add your env definitions in the ENV primitive

Related

Docker gives `no such file or directory` on `docker run <image_id>`

So I've just created my very first docker image (woohoo) and was able to run it on the original host system where it was created (Ubuntu 20.04 Desktop PC). The image was executed using docker run -it <image_id>. The expected command (defined in CMD which is just a bash script) was run, and the expected output was seen. I assumed this meant I successfully created my very first docker image and so I pushed this to Docker Hub.
Docker Hub
GitHub repo with original docker-compose.yml and Dockerfile
Here's the Dockerfile:
FROM ubuntu:20.04
# Required for Debian interaction
# (https://stackoverflow.com/questions/62299928/r-installation-in-docker-gets-stuck-in-geographic-area)
ENV DEBIAN_FRONTEND noninteractive
WORKDIR /home/benchmarking-programming-languages
# Install pre-requisites
# Versions at time of writing:
# gcc -- version (Ubuntu 9.3.0-17ubuntu1~20.04) 9.3.0
# make -- GNU Make 4.2.1
# curl -- 7.68.0
RUN apt update && apt install make build-essential curl wget tar -y
# Install `column`
RUN wget https://mirrors.edge.kernel.org/pub/linux/utils/util-linux/v2.35/util-linux-2.35-rc1.tar.gz
RUN tar xfz util-linux-2.35-rc1.tar.gz
WORKDIR /home/benchmarking-programming-languages/util-linux-2.35-rc1
RUN ./configure
RUN make column
RUN cp .libs/column /bin/
WORKDIR /home/benchmarking-programming-languages
RUN rm -rf util-linux-2.35-rc1*
RUN apt install python3 pip -y
RUN ln -s /usr/bin/python3 /usr/bin/python
RUN apt install default-jdk-headless -y
RUN apt install rustc -y
# Install GoLang
RUN wget https://go.dev/dl/go1.17.8.linux-amd64.tar.gz
RUN rm -rf /usr/local/go && tar -C /usr/local -xzf go1.17.8.linux-amd64.tar.gz
ENV PATH="/usr/local/go/bin:${PATH}"
# Install Haxe and Haxelib
RUN wget https://github.com/HaxeFoundation/haxe/releases/download/4.2.5/haxe-4.2.5-linux64.tar.gz
RUN tar xfz haxe-4.2.5-linux64.tar.gz
RUN ln -s /home/benchmarking-programming-languages/haxe_20220306074705_e5eec31/haxe /usr/bin/haxe
RUN ln -s /home/benchmarking-programming-languages/haxe_20220306074705_e5eec31/haxelib /usr/bin/haxelib
# # Install Neko (Haxe VM)
# RUN add-apt-repository ppa:haxe/snapshots -y
# RUN apt update
# RUN apt install neko -y
RUN if ! test -d /home/benchmarking-programming-languages; then mkdir /home/benchmarking-programming-languages && echo "Created directory /home/benchmarking-programming-languages."; fi
COPY . /home/benchmarking-programming-languages
RUN pip install -r /home/benchmarking-programming-languages/requirements_dev.txt
CMD [ "/home/benchmarking-programming-languages/benchmark.sh -v" ]
However, upon pulling the same image on my Windows 10 machine (same machine as above just dual booted) and a Windows 11 laptop using both the Docker Desktop application and the command line (docker pull mariosyian/benchmarking-programming-languages followed by docker run -it <image_id>). Both which give me the following error
Error invoking remote method 'docker-run-container': Error: (HTTP code 400) unexpected - failed to create shim: OCI runtime create failed: container_linux.go:380: starting container process caused: exec: "/home/benchmarking-programming-languages/benchmark.sh -v": stat /home/benchmarking-programming-languages/benchmark.sh -v: no such file or directory: unknown
Despite this, running the image as a container with a shell (docker run -it <image_id> sh), I am successfully able to, not only see the file, but execute it with no errors! Can someone suggest a reason for why the error happens in the first place, and how to fix it?
In your Dockerfile you have specified the CMD as
CMD [ "/home/benchmarking-programming-languages/benchmark.sh -v" ]
This uses the JSON syntax of the CMD instruction, i.e. is an array of strings where the first string is the executable and each following string is a parameter to that executable.
Since you only have a single string specified docker tries to invoke the executable /home/benchmarking-programming-languages/benchmark.sh -v - i.e. a file named "benchmark.sh -v", containing a space in its name and ending with -v. But what you actually intended to do was to invoke the benchmark.sh script with the -v parameter.
You can do this by correctly specifying the parameter(s) as separate strings:
CMD ["/home/benchmarking-programming-languages/benchmark.sh", "-v"]
or by using the shell syntax:
CMD /home/benchmarking-programming-languages/benchmark.sh -v

Running multiple ROS process in a Docker container

I want to create a bash script which installs all required software to run a docker, create a new image and then runs, in a container, all required processes. My bash script looks like this:
#! /bin/sh
sudo apt-get update
sudo apt-get install docker-ce docker-ce-cli containered.io
sudo groupadd docker
sudo gpasswd -a $USER docker
docker pull ros:indigo-robot
docker build -t myimage .
docker run --name myimage-cont -dit myimage
And the Dockerfile:
FROM ros:indigo-robot
RUN apt-get update && apt-get install -y \
git \
ros-indigo-ardrone-autonomy
I am new to Docker and do not know best practices, but what I need to achieve is running 3 different process at the same time.
- roscore
- rosrun ardrone_autonomy ardrone_driver
- rostopic pub ardrone/takeoff std_msgs/Empty "{}" --once
I was able to achieve it 'manually' by opening 3 terminals and executing docker exec myimage-cont... commands. However, what I need it is make it automatically run by the code once I execute my bash script. What is the best way to do it?

I have made a dockerfile and I was going to run it on AWS ECS but I cant as it requires -t

Here is my docker run and the docker file is there a reason why it requires -t and isnt working on ECS thanks for any help. I dont
understand what -t does so if someone could also help with that thanks.
This is just a basic docker that connects to my rds and uses wordpress. I dont have any plugins and shapely is the theme i'm using .
command docker run -t --name wordpress -d -p 80:80 dockcore/wordpress
FROM ubuntu
#pt-get clean all
RUN apt-get -y update
RUN DEBIAN_FRONTEND=noninteractive apt-get -y install unzip wget mysql-client mysql-server apache2 libapache2-mod-php7.0 pwgen python-setuptools vim-tiny php7.0-mysql php7.0-lda
RUN rm -fr /var/cashe/*files neeeded
ADD wordpress.conf /etc/apache2/sites-enabled/000-default.conf
# Wordpress install
RUN wget -P /var/www/html/ https://wordpress.org/latest.zip
RUN unzip /var/www/html/latest.zip -d /var/www/html/
RUN rm -fr /var/www/html/latest.zip
# Copy he wp config file
RUN cp /var/www/html/wordpress/wp-config-sample.php /var/www/html/wordpress/wp-config.php
# Expose web port
EXPOSE 80
# wp config for database
RUN sed -ie 's/database_name_here/wordpress/g' /var/www/html/wordpress/wp-config.php
RUN sed -ie 's/username_here/root/g' /var/www/html/wordpress/wp-config.php
RUN sed -ie 's/password_here/password/g' /var/www/html/wordpress/wp-config.php
RUN sed -ie 's/localhost/wordpressrds.xxxxxxxxxxxxxx.ap-southeast-2.rds.amazonaws.com:3306/g' /var/www/html/wordpress/wp-config.php
RUN rm -fr /var/www/html/wordpress/wp-content/themes/*
RUN rm -fr /var/www/html/wordpress/wp-content/plugins/*
ADD /shapely /var/www/html/wordpress/wp-content/themes/
# Start apache on boot
RUN echo "service apache2 start" >> ~/.bashrc
I see a couple problems. First of all your container should never require -t in order to run unless it is a temporary container that you plan to interact with using a shell. Background containers shouldn't require an interactive TTY interface, they just run in the background autonomously.
Second in your docker file I see a lot of RUN statements which are basically the build time commands for setting up the initial state of the container, but you don't have any CMD statement.
You need a CMD which is the process to actually kick off and start in the container when you try to run the container. RUN statements only execute once during the initial docker build, and then the results of those run statements are saved into the container image. When you run a docker container it has the initial state that was setup by the RUN statements, and then the CMD statement kicks off a running process in the container.
So it looks like that last RUN in your Dockerfile should be a CMD since the Apache server is the long running process that you want to run with the container state that you previously setup using all those RUN statements.
Another thing you should do is chain many of those consecutive RUN statements into one. Docker creates a separate layer for each RUN command, where each layer is kind of like a Git commit of the state of the container. So it is very wasteful to have so many RUN statements because it makes way too many container layers. You can do something like this ot chain RUN statements together instead to make a smaller, more efficient container:
RUN apt-get -y update && \
DEBIAN_FRONTEND=noninteractive apt-get -y install unzip wget mysql-client mysql-server apache2 libapache2-mod-php7.0 pwgen python-setuptools vim-tiny php7.0-mysql php7.0-lda && \
rm -fr /var/cashe/*files neeeded
I recommend reading through this guide from Docker that covers best practices for writing a Dockerfile: https://docs.docker.com/engine/userguide/eng-image/dockerfile_best-practices/#cmd

Docker refusing to run bash

I have the following docker setup:
python27.Dockerfile
FROM python:2.7
COPY ./entrypoint.sh /entrypoint.sh
RUN mkdir /src
RUN apt-get update && apt-get install -y bash libmysqlclient-dev python-pip build-essential && pip install virtualenv
ENTRYPOINT ["/entrypoint.sh"]
EXPOSE 8000
WORKDIR /src
CMD source /src/env/bin/activate && python /src/manage.py runserver
entrypoint.sh
#!/bin/bash
# some code here...
# some code here...
# some code here...
exec "$#"
Whenever I try to run my docker container I get python27 | /bin/sh: 1: source: not found.
I understand that the error comes from the fact that the command is run with sh instead of bash, but I can't understand why is that happening, given the fact that I have the correct shebang at the top of my entrypoint.
Any ideas why is that happening and how can I fix it?
The problem is that for CMD you're using the shell form that uses /bin/sh, and the /src/env/bin/activate likely contains a "source" command, which isn't available on POSIX /bin/sh (the equivalent builtin would be just .).
You must use the exec form for CMD using brackets:
CMD ["/bin/bash", "-c", "source /src/env/bin/activate && python /src/manage.py runserver"]
More details in:
https://docs.docker.com/engine/reference/builder/#run
https://docs.docker.com/engine/reference/builder/#cmd
https://docs.docker.com/engine/reference/builder/#entrypoint

Docker ERROR: Container command not found or does not exist when running from Win10

This is driving me crazy...
I have Win10 and I have installed the Docker Toolbox with
Docker=1.10.2
Compose=1.6.0
VirtualBox=5.0.14
I have successfully launched the LAMP in Linux [Amazon linux] but when I try to do the same the terminal responds with "ERROR: Container command not found or does not exist"
As I understand, there is something wrong with the way Windows interpreter the CMD syntax.
I have tried
- CMD ["/run.sh"]
- ENTRYPOINT ["/run.sh"]
- CMD /run.sh
- CMD '/run.sh'
- CMD run.sh
- CMD "/run.sh"
but nothing seems to work.
Note: When I run CMD /run.sh the error does not appear but the container exits immediately.
Note2: I have exactly the same problem when trying to setup the LAMP with Docker-Machine on AWS
I have this DockerfileLamp :
FROM ubuntu
# -- Install needed packages --
ENV DEBIAN_FRONTEND noninteractive
# -- Install additional utilities --
RUN apt-get update && \
apt-get install -y supervisor git curl apache2 mcrypt cron wget nano unzip
# -- Install PHP 5.5 --
RUN apt-get -y update && \
apt-get -y install php5 libapache2-mod-php5 mysql-server-5.5 php5-mysql pwgen php-apc php5-mcrypt php5-xdebug php5-gd php5-curl php-pear openssh-server php5-cli php5-apcu php5-intl php5-imagick php5-json
# -- Set localhost to apache conf file --
RUN echo "ServerName localhost" >> /etc/apache2/apache2.conf
# -- Add image configuration and scripts --
ADD ./lamp/start-apache2.sh /start-apache2.sh
ADD ./lamp/start-mysqld.sh /start-mysqld.sh
ADD ./lamp/run.sh /run.sh
RUN chmod 755 /*.sh
ADD ./lamp/my.cnf /etc/mysql/conf.d/my.cnf
ADD ./lamp/supervisord-apache2.conf
/etc/supervisor/conf.d/supervisord-apache2.conf
ADD ./lamp/supervisord-mysqld.conf
/etc/supervisor/conf.d/supervisord-mysqld.conf
# -- Remove pre-installed database --
RUN rm -rf /var/lib/mysql/*
# -- Add MySQL utils --
ADD ./lamp/setup_MySQL.sh /setup_MySQL.sh
RUN chmod 755 /*.sh
# -- config to enable .htaccess --
##ADD apache_default /etc/apache2/sites-available/000-default.conf
RUN a2enmod rewrite
# -- Environmental variables to configure php --
ENV PHP_UPLOAD_MAX_FILESIZE 10M
ENV PHP_POST_MAX_SIZE 10M
# -- Add volumes for MySQL --
##VOLUME ["/etc/mysql", "/var/lib/mysql" ]
# -- Set up SSH server --
RUN mkdir /var/run/sshd
RUN echo 'root:root' |chpasswd
RUN sed -ri 's/^PermitRootLogin\s+.*/PermitRootLogin yes/'
/etc/ssh/sshd_config
RUN sed -ri 's/UsePAM yes/#UsePAM yes/g' /etc/ssh/sshd_config
ADD ./lamp/supervisord-openssh-server.conf
/etc/supervisor/conf.d/supervisord-openssh-server.conf
# -- Install Python & pip --
RUN apt-get update && \
apt-get upgrade -y && \
apt-get install -y python python-pip python-dev && \
pip install --upgrade pip
# -- Install xvfb --
RUN apt-get install -y xvfb
EXPOSE 80 3306 22
CMD /run.sh
and the run.sh :
#!/bin/bash
VOLUME_HOME="/var/lib/mysql"
sed -ri -e "s/^upload_max_filesize.*/upload_max_filesize = ${PHP_UPLOAD_MAX_FILESIZE}/" \
-e "s/^post_max_size.*/post_max_size = ${PHP_POST_MAX_SIZE}/" /etc/php5/apache2/php.ini
if [[ ! -d $VOLUME_HOME/mysql ]]; then
echo "=> An empty or uninitialized MySQL volume is detected in $VOLUME_HOME"
echo "=> Installing MySQL ..."
mysql_install_db > /dev/null 2>&1
echo "=> Done!"
/setup_MySQL.sh
else
echo "=> Using an existing volume of MySQL"
fi
exec supervisord -n
and the docker-compose.yml :
lamp: # apache + mysql/php
build: .
dockerfile: DockerfileLamp
ports:
- "8181:80" # open apache to public
- "3333:3306" # open mysql to public
- "2222:22" # open SSH to public
Docker is process centric, in other words your containers dies when your CMD script dies. At the end of your script run ...
tail -f logfile (where logfile is some logfile you are interested in)
This will
1 - stop your container exiting
2 - allow you to do
docker logs -f containerName
To help u debug
3 - allow you to enter into the container with
docker exec -it bash containerName
Then u can run the command that you think is failing inside the container and try n sort this out
Whilst this doesn't directly answer your question it should give u sufficient weaponry to attack this issue
For another project I tried to get to work on Windows with Docker Machine I ran into the same ambiguous error message of docker-compose Container command not found or does not exist.
Your comment about line endings triggered me to try dos2unix ./*/*.sh within git-bash (multiple scripts, in subfolders), which fixed the issue for me.
My suspicion is that git clone saves the files with DOS line endings, which results in incorrect syntax for the top line !#/bin/bash.
$ docker-compose -v
docker-compose version 1.6.2, build e80fc83
$ docker version
Client:
Version: 1.10.3
API version: 1.22
Go version: go1.5.3
Git commit: 20f81dd
Built: Thu Mar 10 21:49:11 2016
OS/Arch: windows/amd64
Server:
Version: 1.10.3
API version: 1.22
Go version: go1.5.3
Git commit: 20f81dd
Built: Thu Mar 10 21:49:11 2016
OS/Arch: linux/amd64
I solved it by simplifying the file. I commented out all the controls because whatever I tried it would keep throwing Syntax Errors
#!/bin/bash
VOLUME_HOME="/var/lib/mysql"
sed -ri -e "s/^upload_max_filesize.*/upload_max_filesize = ${PHP_UPLOAD_MAX_FILESIZE}/" \
-e "s/^post_max_size.*/post_max_size = ${PHP_POST_MAX_SIZE}/" /etc/php5/apache2/php.ini
#if [[ ! -d $VOLUME_HOME/mysql ]]; then
echo "=> An empty or uninitialized MySQL volume is detected in $VOLUME_HOME"
echo "=> Installing MySQL ..."
mysql_install_db > /dev/null 2>&1
echo "=> Done!"
/setup_MySQL.sh
#else
# echo "=> Using an existing volume of MySQL"
#fi
exec supervisord -n
It works for my case so I am not going to investigate further. Cheers!
EDITED
The above solution was not so complete.
It worked because I was making changes from INSIDE the container.
The permanent solution goes like this :
I migrated the run.sh file to a private Gist . [It does not need to be private but ok]
I think the problem is that when I try to build the Dockerfile from Windows machine [either locally or on a cloud provider] it messes up the syntax , EOF , line breaks and whatnot.
So I broke out of it by ADDing the gist url
ADD http://gist_url/run.sh /run.sh
Note1: You must use the raw file URL otherwise you are going to get the complete HTML.
Note2: The private gist is not protected.You don't need authentication to fetch the URL.

Resources