Dockerfile with entrypoint for executing bash script - bash

I downloaded docker files from official repository (version 2.3), and now I want to build the image and upload some local data (test.json) into the container. It is not enough just to run COPY test.json /usr/share/elasticsearch/data/, because in this case the indexing of data is not done.
What I want to achieve is to be able to run sudo docker run -d -p 9200:9200 -p 9300:9300 -v /home/gosper/tests/tempESData/:/usr/share/elasticsearch/data test/elasticsearch, and after its execution I want to be able to see the mapped data on http://localhost:9200/tests/test/999.
If I use the below-given Dockerfile and *sh script, then I get the following error: Failed to connect to localhost port 9200: Connection refused
This is the Dockerfile from which I build the image:
FROM java:8-jre
# grab gosu for easy step-down from root
ENV GOSU_VERSION 1.7
RUN set -x \
&& wget -O /usr/local/bin/gosu "https://github.com/tianon/gosu/releases/download/$GOSU_VERSION/gosu-$(dpkg --print-architecture)" \
&& wget -O /usr/local/bin/gosu.asc "https://github.com/tianon/gosu/releases/download/$GOSU_VERSION/gosu-$(dpkg --print-architecture).asc" \
&& export GNUPGHOME="$(mktemp -d)" \
&& gpg --keyserver ha.pool.sks-keyservers.net --recv-keys B42F6819007F00F88E364FD4036A9C25BF357DD4 \
&& gpg --batch --verify /usr/local/bin/gosu.asc /usr/local/bin/gosu \
&& rm -r "$GNUPGHOME" /usr/local/bin/gosu.asc \
&& chmod +x /usr/local/bin/gosu \
&& gosu nobody true
# https://www.elastic.co/guide/en/elasticsearch/reference/current/setup-repositories.html
# https://packages.elasticsearch.org/GPG-KEY-elasticsearch
RUN apt-key adv --keyserver ha.pool.sks-keyservers.net --recv-keys 46095ACC8548582C1A2699A9D27D666CD88E42B4
ENV ELASTICSEARCH_VERSION 2.3.4
ENV ELASTICSEARCH_REPO_BASE http://packages.elasticsearch.org/elasticsearch/2.x/debian
RUN echo "deb $ELASTICSEARCH_REPO_BASE stable main" > /etc/apt/sources.list.d/elasticsearch.list
RUN set -x \
&& apt-get update \
&& apt-get install -y --no-install-recommends elasticsearch=$ELASTICSEARCH_VERSION \
&& rm -rf /var/lib/apt/lists/*
ENV PATH /usr/share/elasticsearch/bin:$PATH
WORKDIR /usr/share/elasticsearch
RUN set -ex \
&& for path in \
./data \
./logs \
./config \
./config/scripts \
; do \
mkdir -p "$path"; \
chown -R elasticsearch:elasticsearch "$path"; \
done
COPY config ./config
VOLUME /usr/share/elasticsearch/data
COPY docker-entrypoint.sh /
EXPOSE 9200 9300
RUN chmod +x /docker-entrypoint.sh
ENTRYPOINT ["/docker-entrypoint.sh"]
CMD ["elasticsearch"]
COPY template.json /usr/share/elasticsearch/data/
RUN /bin/bash -c "source /docker-entrypoint.sh"
This is the docker-entrypoint.sh in which I added the line curl -XPOST http://localhost:9200/uniko-documents/document/978-1-60741-503-9 -d "/usr/share/elasticsearch/data/template.json":
#!/bin/bash
set -e
# Add elasticsearch as command if needed
if [ "${1:0:1}" = '-' ]; then
set -- elasticsearch "$#"
fi
# Drop root privileges if we are running elasticsearch
# allow the container to be started with `--user`
if [ "$1" = 'elasticsearch' -a "$(id -u)" = '0' ]; then
# Change the ownership of /usr/share/elasticsearch/data to elasticsearch
chown -R elasticsearch:elasticsearch /usr/share/elasticsearch/data
set -- gosu elasticsearch "$#"
#exec gosu elasticsearch "$BASH_SOURCE" "$#"
fi
curl -XPOST http://localhost:9200/tests/test/999 -d "/usr/share/elasticsearch/data/test.json"
# As argument is not related to elasticsearch,
# then assume that user wants to run his own process,
# for example a `bash` shell to explore this image
exec "$#"

Remove the following from your docker-entrypoint.sh:
curl -XPOST http://localhost:9200/tests/test/999 -d "/usr/share/elasticsearch/data/test.json"
It's running before you exec the service at the end.
In your Dockerfile, move the following after any commands that modify the directory:
VOLUME /usr/share/elasticsearch/data
Once you create a volume, future changes to the directory are typically ignored.
Lastly, in your Dockerfile, this line at the end likely doesn't do what you think, I'd remove it:
RUN /bin/bash -c "source /docker-entrypoint.sh"
The entrypoint.sh should be run when you start the container, not when you're building it.

#Klue in case you still need it.. you need to change the -d option on your curl command to --data-binary. -d strips the newlines. that's why you are getting the errors.

Related

neo4j 4.0 testing with embedded database DatabaseManagementServiceBuilder is found nowhere

I'm a beginner in neo4j. I'm trying to build tests using embedded neo4j database inside a springboot application. I haven't succeeded since the class DatabaseManagementServiceBuilder is found nowhere Please note I'm using version 4.0.2 Any help please ?
The full classname is org.neo4j.dbms.api.DatabaseManagementServiceBuilder.
Here is a sample class that uses the builder.
I've struggled with using an embedded neo4j db for my tests a few months back as well.
In case you don't find a suitable solution for the embedded version, I ended up starting a real instance of the db...
I adjusted a bit neo4j's official Dockerfile to use jdk instead of jre and was able to run my tests against it.
Here's the Dockerfile, starting from the official 3.4.5-enterprise Dockerfile:
FROM openjdk:8-jdk-slim
RUN apt-get update && apt-get install -y --no-install-recommends \
make && \
rm -rf /var/lib/apt/lists/*
ADD maven-settings.xml /root/.m2/settings.xml
# from official neo4j:3.4.5-enterprise image content (changed base image to use jdk instead of jre)
RUN addgroup --system neo4j && adduser --system --no-create-home --home /var/lib/neo4j --ingroup neo4j neo4j
ENV NEO4J_SHA256=0629f17a99ba90d6900c98f332c775a732cc2ad6298b8df41a2872277b19e6e3 \
NEO4J_TARBALL=neo4j-enterprise-3.4.5-unix.tar.gz \
NEO4J_EDITION=enterprise \
NEO4J_ACCEPT_LICENSE_AGREEMENT=yes \
TINI_VERSION="v0.18.0" \
TINI_SHA256="12d20136605531b09a2c2dac02ccee85e1b874eb322ef6baf7561cd93f93c855"
ARG NEO4J_URI=http://dist.neo4j.org/neo4j-enterprise-3.4.5-unix.tar.gz
RUN apt update \
&& apt install -y \
bash \
curl \
&& curl -L --fail --silent --show-error "https://github.com/krallin/tini/releases/download/${TINI_VERSION}/tini" > /sbin/tini \
&& echo "${TINI_SHA256} /sbin/tini" | sha256sum -c --strict --quiet \
&& chmod +x /sbin/tini \
&& curl --fail --silent --show-error --location --remote-name ${NEO4J_URI} \
&& echo "${NEO4J_SHA256} ${NEO4J_TARBALL}" | sha256sum -c --strict --quiet \
&& tar --extract --file ${NEO4J_TARBALL} --directory /var/lib \
&& mv /var/lib/neo4j-* /var/lib/neo4j \
&& rm ${NEO4J_TARBALL} \
&& mv /var/lib/neo4j/data /data \
&& chown -R neo4j:neo4j /data \
&& chmod -R 777 /data \
&& chown -R neo4j:neo4j /var/lib/neo4j \
&& chmod -R 777 /var/lib/neo4j \
&& ln -s /data /var/lib/neo4j/data
# Install latest su-exec
RUN set -ex; \
\
curl -o /usr/local/bin/su-exec.c https://raw.githubusercontent.com/ncopa/su-exec/master/su-exec.c; \
\
fetch_deps='gcc libc-dev'; \
apt-get update; \
apt-get install -y --no-install-recommends $fetch_deps; \
rm -rf /var/lib/apt/lists/*; \
gcc -Wall \
/usr/local/bin/su-exec.c -o/usr/local/bin/su-exec; \
chown root:root /usr/local/bin/su-exec; \
chmod 0755 /usr/local/bin/su-exec; \
rm /usr/local/bin/su-exec.c; \
\
apt-get purge -y --auto-remove $fetch_deps
ENV PATH /var/lib/neo4j/bin:$PATH
ARG NEO4J_AUTH=neo4j/neo4jtest
ENV NEO4J_AUTH=${NEO4J_AUTH}
WORKDIR /var/lib/neo4j
VOLUME /data
COPY docker-entrypoint.sh /docker-entrypoint.sh
EXPOSE 7474 7473 7687
ENTRYPOINT ["/sbin/tini", "-g", "--", "/docker-entrypoint.sh"]
CMD ["neo4j"]
I used the original docker-entrypoint.sh script.

DB2+IBM MQ; enable_MQFunctions = Error -- while connecting to database

I build docker's image containing IBM MQ 9.1, DB2express-c 9.7 + ubuntu 16.04 64bit.
I want to enable MQ functions(sending msg to queue) on my Db2 database.
But when I used enable_MQFunctions than I got this error:
*** Error -- while connecting to TEST
Make sure that user(db2inst1) and password(pass) are valid and that the DB2 instance has started.
*** enable_MQFunction finished with error
Database, user, pass are all okey. And i Don't understand than before this command w/o problems connected to my database
Dockerfile I today used(with only DB2 and IBM MQ, w/o IIB):
# © Copyright IBM Corporation 2015, 2017
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#==============================
#========================
#FROM centos:7
FROM ubuntu:16.04
#FROM ubuntu:17.10
#LABEL maintainer "Arthur Barr <arthur.barr#uk.ibm.com>, Rob Parker <PARROBE#uk.ibm.com>"
#LABEL "ProductID"="98102d16795c4263ad9ca075190a2d4d" \
# "ProductName"="IBM MQ Advanced for Developers" \
# "ProductVersion"="9.0.4"
# The URL to download the MQ installer from in tar.gz format
#oryginal ARG MQ_URL=https://public.dhe.ibm.com/ibmdl/export/pub/software/websphere/messaging/mqadv/mqadv_dev904_ubuntu_x86-64.tar.gz
ARG MQ_URL=http://public.dhe.ibm.com/ibmdl/export/pub/software/websphere/messaging/mqadv/mqadv_dev910_ubuntu_x86-64.tar.gz
#ARG MQ_URL=http://public.dhe.ibm.com/ibmdl/export/pub/software/websphere/messaging/mqadv/mqadv_dev80_linux_x86-64.tar.gz
#ARG MQ_URL=\\172.29.5.249\mqadv_dev910_ubuntu_x86-64.tar.gz
# The MQ packages to install
ARG MQ_PACKAGES="ibmmq-server ibmmq-java ibmmq-jre ibmmq-gskit ibmmq-web ibmmq-msg-.*"
#RUN rm /var/lib/apt/lists/*
RUN apt-get clean -y
RUN apt-get autoclean -y
RUN export DEBIAN_FRONTEND=noninteractive \
# Install additional packages required by MQ, this install process and the runtime scripts
&& apt-get update -y \
&& apt-get install -y --no-install-recommends \
# && yum update -y \
# && yum install -y \
bash \
bc \
ca-certificates \
coreutils \
curl \
debianutils \
file \
findutils \
gawk \
grep \
libc-bin \
lsb-release \
mount \
passwd \
procps \
sed \
tar \
util-linux \
# Download and extract the MQ installation files
&& export DIR_EXTRACT=/tmp/mq \
&& mkdir -p ${DIR_EXTRACT} \
&& cd ${DIR_EXTRACT} \
&& curl -LO $MQ_URL \
&& tar -zxvf ./*.tar.gz \
# Recommended: Remove packages only needed by this script
#
#&& package-cleanup --leaves --all \ <-------moje dodanie
# Recommended: Create the mqm user ID with a fixed UID and group, so that the file permissions work between different images
&& groupadd --system --gid 990 mqm \
&& useradd --system --uid 990 --gid mqm mqm \
&& usermod -G mqm root \
# Find directory containing .deb files
&& export DIR_DEB=$(find ${DIR_EXTRACT} -name "*.deb" -printf "%h\n" | sort -u | head -1) \
# Find location of mqlicense.sh
&& export MQLICENSE=$(find ${DIR_EXTRACT} -name "mqlicense.sh") \
# Accept the MQ license
&& ${MQLICENSE} -text_only -accept \
&& echo "deb [trusted=yes] file:${DIR_DEB} ./" > /etc/apt/sources.list.d/IBM_MQ.list \
# Install MQ using the DEB packages
&& apt-get update \
&& apt-get install -y $MQ_PACKAGES \
# Remove 32-bit libraries from 64-bit container
&& find /opt/mqm /var/mqm -type f -exec file {} \; \
| awk -F: '/ELF 32-bit/{print $1}' | xargs --no-run-if-empty rm -f \
# Remove tar.gz files unpacked by RPM postinst scripts
&& find /opt/mqm -name '*.tar.gz' -delete \
# Recommended: Set the default MQ installation (makes the MQ commands available on the PATH)
&& /opt/mqm/bin/setmqinst -p /opt/mqm -i \
# Clean up all the downloaded files
&& rm -f /etc/apt/sources.list.d/IBM_MQ.list \
&& rm -rf ${DIR_EXTRACT} \
# Apply any bug fixes not included in base Ubuntu or MQ image.
# Don't upgrade everything based on Docker best practices https://docs.docker.com/engine/userguide/eng-image/dockerfile_best-practices/#run
&& apt-get upgrade -y sensible-utils \
# End of bug fixes
&& rm -rf /var/lib/apt/lists/* \
# Optional: Update the command prompt with the MQ version
&& echo "mq:$(dspmqver -b -f 2)" > /etc/debian_chroot \
&& rm -rf /var/mqm \
# Optional: Set these values for the Bluemix Vulnerability Report
&& sed -i 's/PASS_MAX_DAYS\t99999/PASS_MAX_DAYS\t90/' /etc/login.defs \
&& sed -i 's/PASS_MIN_DAYS\t0/PASS_MIN_DAYS\t1/' /etc/login.defs \
&& sed -i 's/password\t\[success=1 default=ignore\]\tpam_unix\.so obscure sha512/password\t[success=1 default=ignore]\tpam_unix.so obscure sha512 minlen=8/' /etc/pam.d/common-password
#==========db2 expres START====
#FROM centos:7
#MAINTAINER Leo Wu <leow#ca.ibm.com>
###############################################################
#
# System preparation for DB2
#
###############################################################
#********************z iib-mq-db2 git
RUN dpkg --add-architecture i386
RUN export DEBIAN_FRONTEND=noninteractive \
&& apt-get update && \
apt-get install -y --no-install-recommends \
curl \
bash \
bc \
coreutils \
curl \
debianutils \
findutils \
gawk \
grep \
libc-bin \
lsb-release \
libncurses-dev \
libstdc++6 \
gcc \
binutils \
make \
libpam0g:i386 \
lib32stdc++6 \
lib32gcc1 \
libcurl4-gnutls-dev:i386 \
numactl \
libaio1 \
libxml2 \
mount \
passwd \
procps \
rpm \
sed \
tar \
wget \
util-linux
RUN rm -rf /var/lib/apt/lists/*
RUN apt-get dist-upgrade -y
#******************
RUN groupadd db2iadm1 && useradd -G db2iadm1 db2inst1
# Required packages
#RUN yum install -y \
# vi \
# sudo \
# passwd \
# pam \
# pam.i686 \
# ncurses-libs.i686 \
# file \
# libaio \
# libstdc++-devel.i686 \
# numactl-libs \
# which \
# && yum clean all
ENV DB2EXPRESSC_DATADIR /home/db2inst1/data
# IMPORTANT Note:
# Due to compliance for IBM product, you have to host a downloaded DB2 Express-C Zip file yourself
# Here are suggested steps:
# 1) Please download zip file of db2 express-c from http://www-01.ibm.com/software/data/db2/express-c/download.html
# 2) Then upload it to a cloud storage like AWS S3 or IBM SoftLayer Object Storage
# 3) Acquire a URL and SHA-256 hash of file and pass it via Docker's build time argument facility
ARG DB2EXPRESSC_URL=ftp://ftp.software.ibm.com/software/data/db2/express/db2exc_images/db2exc_970_LNX_x86_64.tar.gz
#ARG DB2EXPRESSC_URL=http://lorenzana.gt/uploads/files/v10.5fp1_linuxx64_expc.tar.gz
#ARG DB2EXPRESSC_URL=\\172.29.5.249\public\image\v10.5fp1_linuxx64_expc.tar.gz
ADD db2expc.rsp /tmp/db2expc.rsp
ADD db2rfe.cfg /home/db2inst1/sqllib/instance/db2rfe.cfg
COPY db2expc.rsp /tmp
RUN curl -fkSLo /tmp/expc.tar.gz $DB2EXPRESSC_URL
RUN cd /tmp && tar xf expc.tar.gz
RUN rm -rf /home/db2inst1/sqllib
RUN mkdir /home/db2inst1/sqllib
RUN su - root -c "chmod -R 1777 /home/db2inst1/"
RUN su - db2inst1 -c "/tmp/expc/db2_install -f sysreq -b /home/db2inst1/sqllib"
# RUN su - db2inst1 -c "/tmp/expc/db2setup -r /tmp/db2expc.rsp" || echo "db2setup failed"
RUN echo '. /home/db2inst1/sqllib/db2profile' >> /home/db2inst1/.bash_profile \
&& rm -rf /tmp/db2* && rm -rf /tmp/expc* \
&& sed -ri 's/(ENABLE_OS_AUTHENTICATION=).*/\1YES/g' /home/db2inst1/sqllib/instance/db2rfe.cfg \
&& sed -ri 's/(RESERVE_REMOTE_CONNECTION=).*/\1YES/g' /home/db2inst1/sqllib/instance/db2rfe.cfg \
&& sed -ri 's/^\*(SVCENAME=db2c_db2inst1)/\1/g' /home/db2inst1/sqllib/instance/db2rfe.cfg \
&& sed -ri 's/^\*(SVCEPORT)=48000/\1=50000/g' /home/db2inst1/sqllib/instance/db2rfe.cfg \
&& mkdir $DB2EXPRESSC_DATADIR && chown db2inst1.db2iadm1 $DB2EXPRESSC_DATADIR
RUN su - db2inst1 -c "db2start && db2set DB2COMM=TCPIP && db2 UPDATE DBM CFG USING DFTDBPATH $DB2EXPRESSC_DATADIR IMMEDIATE && db2 create database db2inst1" \
&& su - db2inst1 -c "db2stop force" \
&& cd /home/db2inst1/sqllib/instance \
&& ./db2rfe -f ./db2rfe.cfg
#COPY docker-entrypoint.sh /entrypoint.sh
#ENTRYPOINT ["/entrypoint.sh"]
#VOLUME $DB2EXPRESSC_DATADIR
#EXPOSE 50000
#=========db2 express END ====
COPY *.sh /usr/local/bin/
COPY *.mqsc /etc/mqm/
COPY admin.json /etc/mqm/
COPY mq-dev-config /etc/mqm/mq-dev-config
RUN chmod +x /usr/local/bin/*.sh
# Always use port 1414 (the Docker administrator can re-map ports at runtime)
# Expose port 9443 for the web console
#VOLUME /home/db2inst1/data
EXPOSE 1414 9443 50000
ENV LANG=en_US.UTF-8
COPY docker-entrypoint.sh /entrypoint.sh
ENTRYPOINT ["/entrypoint.sh"]
#ENTRYPOINT ["mq.sh"]
entrypoint.sh (with MQ and DB2 commands):
#======= start MQ =====
set -e
mq-license-check.sh
echo "----------------------------------------"
source mq-parameter-check.sh
echo "----------------------------------------"
setup-var-mqm.sh
echo "----------------------------------------"
which strmqweb && source setup-mqm-web.sh
echo "----------------------------------------"
mq-pre-create-setup.sh
echo "----------------------------------------"
source mq-create-qmgr.sh
echo "----------------------------------------"
source mq-start-qmgr.sh
echo "----------------------------------------"
source mq-dev-config.sh
echo "----------------------------------------"
source mq-configure-qmgr.sh
echo "----------------------------------------"
exec mq-monitor-qmgr.sh ${MQ_QMGR_NAME}
#======== z MQ - END ======
pid=0
function log_info {
echo -e $(date '+%Y-%m-%d %T')"\e[1;32m $#\e[0m"
}
function log_error {
echo -e >&2 $(date +"%Y-%m-%d %T")"\e[1;31m $#\e[0m"
}
function stop_db2 {
log_info "stopping database engine"
su - db2inst1 -c "db2stop force"
}
function start_db2 {
log_info "starting database engine"
su - db2inst1 -c "db2start"
}
function restart_db2 {
# if you just need to restart db2 and not to kill this container
# use docker kill -s USR1 <container name>
kill ${spid}
log_info "Asked for instance restart doing it..."
stop_db2
start_db2
log_info "database instance restarted on request"
}
function terminate_db2 {
kill ${spid}
stop_db2
if [ $pid -ne 0 ]; then
kill -SIGTERM "$pid"
wait "$pid"
fi
log_info "database engine stopped"
exit 0 # finally exit main handler script
}
trap "terminate_db2" SIGTERM
trap "restart_db2" SIGUSR1
if [ ! -f ~/db2inst1_pw_set ]; then
if [ -z "$DB2INST1_PASSWORD" ]; then
log_error "error: DB2INST1_PASSWORD not set"
log_error "Did you forget to add -e DB2INST1_PASSWORD=... ?"
exit 1
else
log_info "Setting db2inst1 user password..."
(echo "$DB2INST1_PASSWORD"; echo "$DB2INST1_PASSWORD") | passwd db2inst1 > /dev/null 2>&1
if [ $? != 0 ];then
log_error "Changing password for db2inst1 failed"
exit 1
fi
touch ~/db2inst1_pw_set
fi
fi
if [ ! -f ~/db2_license_accepted ];then
if [ -z "$LICENSE" ];then
log_error "error: LICENSE not set"
log_error "Did you forget to add '-e LICENSE=accept' ?"
exit 1
fi
if [ "${LICENSE}" != "accept" ];then
log_error "error: LICENSE not set to 'accept'"
log_error "Please set '-e LICENSE=accept' to accept License before use the DB2 software contained in this image."
exit 1
fi
touch ~/db2_license_accepted
fi
if [[ $1 = "-d" ]]; then
log_info "Initializing container"
start_db2
log_info "Database db2diag log following"
tail -f ~db2inst1/sqllib/db2dump/db2diag.log &
export pid=${!}
while true
do
sleep 10000 &
export spid=${!}
wait $spid
done
else
exec "$1"
fi
and than:
docker run -e LICENSE=accept -e MQ_QMGR_NAME=MQ321 -e DB2INST1_PASSWORD=pass -p 41419:1414 -p 9459:9443 -p 5015:50000 allall4r
And after all, I used command from : HERE
So I executed:
root:
usermod -G mqm db2inst1
/opt/mqm/bin/setmqinst -i -n Installation1 -p /opt/mqm
mqm user:
PATH=$PATH:/opt/mqm/bin
db2inst1 user:
LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/opt/mqm/lib64
AMT_DATA_PATH=/opt/mqm
db2start
db2 create db testdb
db2 connect to testdb
cd ~/sqllib/cfg/mq
db2 –tvf amtsetup.sql
Upload with all files needed to build this image are here: UPLOAD LINK
Image will be about 3.1GB
I suspect that the cause of your symptom is that the account specified for enable_MQFunctions command line does not have a password at the time that enable_MQFunctions tries to run. You can prove this by looking at db2diag.log to see the exact authentication failure message, and/or by looking at the /etc/passwd entry for that account just before you run enable_MQFunctions.
You can expand the Dockerfile to configure the Db2 for MQ entirely during the docker build instead of running those steps after docker run or in entrypoints. That way you are responsible for all the steps inside the Dockerfile and it will be repeatable without manual intervention after the docker run command. It also means that your built image is pre-baked with all of the required configuration which will then be persistent. You need to have enough competence with scripting in the Dockerfile to get the desired outcome.
When correctly done, the enable_MQFunctions will operate properly during docker build, so if you are getting errors it's because you are doing it incorrectly.
I can successfully configure the database and run enable_MQFunctions all inside the Dockerfile, with these steps below (because of using a non-root install of Db2), so all the configuration is already in the built image.
after installing Db2 and before db2start the Dockerfile should
create /home/db2inst1/sqllib/userprofile (which will run whenever the instance-owner accounts dots in its db2profile from .bash_profile or .profile), to do these steps:
-- append /opt/mqm/lib64 to LD_LIBRARY_PATH
-- export AMT_DATA_PATH=/opt/mqm
-- prepend /opt/mqm/bin on the PATH
chown db2inst1:db2iadm1 /home/db2inst1/sqllib/userprofile
after installing Db2 and before db2start, the Dockerfile should run these steps:
-- db2set DB2COMM=TCPIP
-- db2set DB2ENVLIST=AMT_DATA_PATH
-- db2 -v update dbm cfg using federated yes immediate
set a password for db2inst1 account in the Dockerfile
the Dockerfile can then run db2start, create the database ( i call it sample, you can call it whatever you like) and run the fragment below as user db2inst1 to first create the required objects in the database used by the MQ functions:
su -db2inst1 -c "( db2 -v connect to sample ; \
db2 -tvf /home/db2inst1/sqllib/cfg/mq/amtsetup.sql; \
db2 -v list tables for schema DB2MQ ; \
exit 0 ) "
Notice that you have to run amtsetup.sql in a subshell ,as shown, to explicitly exit 0, because amtsetup.sql always returns non-zero exit code even when it completes successfully. So you want the docker build to continue in that case.
If all the above steps completed successfully and MQ is already successfully installed, later in the Dockerfile you can run the enable_MQFunctions as follows:
I use ARG INSTANCE_PASSWORD to specify the db2inst1 password, which can come from external.
su - db2inst1 -c "( . ./.profile ;\
db2start ;\
db2 -v activate database sample ;\
cd /home/db2inst1/sqllib/cfg ; \
/home/db2inst1/sqllib/bin/enable_MQFunctions -echo -force -n sample -u db2inst1 -p $INSTANCE_PASSWORD ; \
db2stop force ; \
exit 0)"
Problem was with environment variables. My image, after built, can't hold any variable. I try with export prefix but no change. So no password, no good LD_LIBRARY_PATH. Event after I change and logout, variable back to default.
After I used root -> passwd on my account (db2inst1) I can execute enable_MQFunction with good password
Next error is that I dont have valid license for db2..

docker container nginx start but keeps hanging

my dockerfile:
FROM AWS_ECR_IMAGE
RUN apt-get update && apt-get install -y \
cron \
python-dev \
git \
zlib1g-dev \
libffi-dev \
libssl-dev \
autotools-dev \
automake \
libbz2-dev \
libaio-dev \
libsasl2-dev \
python-pip
RUN pip install boto boto3 awscli
# Install Nginx.
RUN \
add-apt-repository -y ppa:nginx/stable && \
apt-get update && \
apt-get install -y nginx && \
rm -rf /var/lib/apt/lists/* && \
echo "\ndaemon off;" >> /etc/nginx/nginx.conf && \
chown -R www-data:www-data /var/lib/nginx
# Define mountable directories.
VOLUME ["/etc/nginx/sites-enabled", "/etc/nginx/certs", "/etc/nginx/conf.d", "/var/log/nginx", "/var/www/html"]
# Define working directory.
WORKDIR /etc/nginx
# Define default command.
CMD ["nginx"]
COPY nginx_conf /etc/nginx/sites-available/default
# Start service, replace server name, update web ui
COPY main.sh /opt/annotation-pipeline-docs/main.sh
RUN chmod 0755 /opt/annotation-pipeline-docs/main.sh
ENTRYPOINT [ "sh", "-c", "/opt/annotation-pipeline-docs/main.sh" ]
# Expose ports.
EXPOSE 80
And my entrypoint bash file (I need to update the server name first when the container runs) is:
#!/bin/bash -e
/usr/local/bin/aws s3 sync s3://${S3_Bucket}/docs/${ENVIRONMENT}/HEAD/ /var/www/html/
if [ "$ENVIRONMENT" == "prod" ]
then
sed -i.bak "s/REPLACE_ME/example.com/g" /etc/nginx/sites-available/default
else
sed -i.bak "s/REPLACE_ME/example-$ENVIRONMENT.com/g" /etc/nginx/sites-available/default
fi
nginx
while true; do
sleep 60
echo "s3 sync again:"
/usr/local/bin/aws s3 sync s3://${S3_Bucket}/docs/${ENVIRONMENT}/HEAD/ /var/www/html/
done
The issue is when
nginx
runs, it will hanging forever in the terminal:
and the while loop will never get called. Anyone know why is hanging and how to resolve it? Please help, tks in advanced.
The reason for my issue is waiting for the traffic, the while loop will never get called until Nginx start free the bash. However, Nginx will be running in the foreground and not release the focus.
The solution I tried is instead of letting Nginx running as a foreground service, I changed it run in the background. Since this is the only service in my container, should have no problem to do it.
the code changed is simply removed below line in my dockerfile:
echo "\ndaemon off;" >> /etc/nginx/nginx.conf
which will make Nginx as a foreground service

Failed to Call Access Method Exception when Creating a MedicationOrder in FHIR

I am using this http://fhirtest.uhn.ca/baseDstu2 test FHIR server and it worked okay so far.
Now I am getting an HTTP-500 - Failed to Call Access Method exception.
Anyone has any idea on what has gone wrong?
This happens frequently. Probably because someone tested weird queries or similar that put the server in an unstable status.
I suggest posting a comment in https://chat.fhir.org/#narrow/stream/hapi to get the server restarted,
or install http://hapifhir.io/doc_cli.html which does basically the same but you have full control.
I built a Dockerfile:
FROM debian:sid
MAINTAINER Günter Zöchbauer <guenter#yyy.com>
ENV DEBIAN_FRONTEND noninteractive
RUN \
apt-get -q update && \
DEBIAN_FRONTEND=noninteractive && \
apt-get install --no-install-recommends -y -q \
apt-transport-https \
apt-utils \
wget \
bzip2 \
default-jdk
# net-tools sudo procps telnet
RUN \
apt-get update && \
rm -rf /var/lib/apt/lists/*
https://github.com/jamesagnew/hapi-fhir/releases/download/v2.0/hapi-fhir-2.0-cli.tar.bz2 && \
ADD hapi-* /hapi_fhir_cli/
RUN ls -la
RUN ls -la /hapi_fhir_cli
ADD prepare_server.sh /hapi_fhir_cli/
RUN \
cd /hapi_fhir_cli && \
bash -c /hapi_fhir_cli/prepare_server.sh
ADD start.sh /hapi_fhir_cli/
WORKDIR /hapi_fhir_cli
EXPOSE 5555
ENTRYPOINT ["/hapi_fhir_cli/start.sh"]
Which requires in the same directory as the Dockerfile
prepare_server.sh
#!/usr/bin/env bash
ls -la
./hapi-fhir-cli run-server --allow-external-refs &
while ! timeout 1 bash -c "echo > /dev/tcp/localhost/8080"; do sleep 10; done
./hapi-fhir-cli upload-definitions -t http://localhost:8080/baseDstu2
./hapi-fhir-cli upload-examples -c -t http://localhost:8080/baseDstu2
start.sh
#!/usr/bin/env bash
cd /hapi_fhir_cli
./hapi-fhir-cli run-server --allow-external-refs -p 5555
Build
docker build myname/hapi_fhir_cli_dstu2 -t . #--no-cache
Run
docker run -d -p 5555:5555 [image id from docker build]
Hope this helps.

How to generate a Dockerfile from an image?

Is it possible to generate a Dockerfile from an image? I want to know for two reasons:
I can download images from the repository but would like to see the recipe that generated them.
I like the idea of saving snapshots, but once I am done it would be nice to have a structured format to review what was done.
How to generate or reverse a Dockerfile from an image?
You can. Mostly.
Notes: It does not generate a Dockerfile that you can use directly with docker build; the output is just for your reference.
alias dfimage="docker run -v /var/run/docker.sock:/var/run/docker.sock --rm alpine/dfimage"
dfimage -sV=1.36 nginx:latest
It will pull the target docker image automatically and export Dockerfile. Parameter -sV=1.36 is not always required.
Reference: https://hub.docker.com/r/alpine/dfimage
Now hub.docker.com shows the image layers with detail commands directly, if you choose a particular tag.
Bonus
If you want to know which files are changed in each layer
alias dive="docker run -ti --rm -v /var/run/docker.sock:/var/run/docker.sock wagoodman/dive"
dive nginx:latest
On the left, you see each layer's command, on the right (jump with tab), the yellow line is the folder that some files are changed in that layer
(Use SPACE to collapse dir)
Old answer
below is the old answer, it doesn't work any more.
$ docker pull centurylink/dockerfile-from-image
$ alias dfimage="docker run -v /var/run/docker.sock:/var/run/docker.sock --rm centurylink/dockerfile-from-image"
$ dfimage --help
Usage: dockerfile-from-image.rb [options] <image_id>
-f, --full-tree Generate Dockerfile for all parent layers
-h, --help Show this message
To understand how a docker image was built, use the
docker history --no-trunc command.
You can build a docker file from an image, but it will not contain everything you would want to fully understand how the image was generated. Reasonably what you can extract is the MAINTAINER, ENV, EXPOSE, VOLUME, WORKDIR, ENTRYPOINT, CMD, and ONBUILD parts of the dockerfile.
The following script should work for you:
#!/bin/bash
docker history --no-trunc "$1" | \
sed -n -e 's,.*/bin/sh -c #(nop) \(MAINTAINER .*[^ ]\) *0 B,\1,p' | \
head -1
docker inspect --format='{{range $e := .Config.Env}}
ENV {{$e}}
{{end}}{{range $e,$v := .Config.ExposedPorts}}
EXPOSE {{$e}}
{{end}}{{range $e,$v := .Config.Volumes}}
VOLUME {{$e}}
{{end}}{{with .Config.User}}USER {{.}}{{end}}
{{with .Config.WorkingDir}}WORKDIR {{.}}{{end}}
{{with .Config.Entrypoint}}ENTRYPOINT {{json .}}{{end}}
{{with .Config.Cmd}}CMD {{json .}}{{end}}
{{with .Config.OnBuild}}ONBUILD {{json .}}{{end}}' "$1"
I use this as part of a script to rebuild running containers as images:
https://github.com/docbill/docker-scripts/blob/master/docker-rebase
The Dockerfile is mainly useful if you want to be able to repackage an image.
The thing to keep in mind, is a docker image can actually just be the tar backup of a real or virtual machine. I have made several docker images this way. Even the build history shows me importing a huge tar file as the first step in creating the image...
I somehow absolutely missed the actual command in the accepted answer, so here it is again, bit more visible in its own paragraph, to see how many people are like me
$ docker history --no-trunc <IMAGE_ID>
A bash solution :
docker history --no-trunc $argv | tac | tr -s ' ' | cut -d " " -f 5- | sed 's,^/bin/sh -c #(nop) ,,g' | sed 's,^/bin/sh -c,RUN,g' | sed 's, && ,\n & ,g' | sed 's,\s*[0-9]*[\.]*[0-9]*\s*[kMG]*B\s*$,,g' | head -n -1
Step by step explanations:
tac : reverse the file
tr -s ' ' trim multiple whitespaces into 1
cut -d " " -f 5- remove the first fields (until X months/years ago)
sed 's,^/bin/sh -c #(nop) ,,g' remove /bin/sh calls for ENV,LABEL...
sed 's,^/bin/sh -c,RUN,g' remove /bin/sh calls for RUN
sed 's, && ,\n & ,g' pretty print multi command lines following Docker best practices
sed 's,\s*[0-9]*[\.]*[0-9]*\s*[kMG]*B\s*$,,g' remove layer size information
head -n -1 remove last line ("SIZE COMMENT" in this case)
Example:
~  dih ubuntu:18.04
ADD file:28c0771e44ff530dba3f237024acc38e8ec9293d60f0e44c8c78536c12f13a0b in /
RUN set -xe
&& echo '#!/bin/sh' > /usr/sbin/policy-rc.d
&& echo 'exit 101' >> /usr/sbin/policy-rc.d
&& chmod +x /usr/sbin/policy-rc.d
&& dpkg-divert --local --rename --add /sbin/initctl
&& cp -a /usr/sbin/policy-rc.d /sbin/initctl
&& sed -i 's/^exit.*/exit 0/' /sbin/initctl
&& echo 'force-unsafe-io' > /etc/dpkg/dpkg.cfg.d/docker-apt-speedup
&& echo 'DPkg::Post-Invoke { "rm -f /var/cache/apt/archives/*.deb /var/cache/apt/archives/partial/*.deb /var/cache/apt/*.bin || true"; };' > /etc/apt/apt.conf.d/docker-clean
&& echo 'APT::Update::Post-Invoke { "rm -f /var/cache/apt/archives/*.deb /var/cache/apt/archives/partial/*.deb /var/cache/apt/*.bin || true"; };' >> /etc/apt/apt.conf.d/docker-clean
&& echo 'Dir::Cache::pkgcache ""; Dir::Cache::srcpkgcache "";' >> /etc/apt/apt.conf.d/docker-clean
&& echo 'Acquire::Languages "none";' > /etc/apt/apt.conf.d/docker-no-languages
&& echo 'Acquire::GzipIndexes "true"; Acquire::CompressionTypes::Order:: "gz";' > /etc/apt/apt.conf.d/docker-gzip-indexes
&& echo 'Apt::AutoRemove::SuggestsImportant "false";' > /etc/apt/apt.conf.d/docker-autoremove-suggests
RUN rm -rf /var/lib/apt/lists/*
RUN sed -i 's/^#\s*\(deb.*universe\)$/\1/g' /etc/apt/sources.list
RUN mkdir -p /run/systemd
&& echo 'docker' > /run/systemd/container
CMD ["/bin/bash"]
Update Dec 2018 to BMW's answer
chenzj/dfimage - as described on hub.docker.com regenerates Dockerfile from other images. So you can use it as follows:
docker pull chenzj/dfimage
alias dfimage="docker run -v /var/run/docker.sock:/var/run/docker.sock --rm chenzj/dfimage"
dfimage IMAGE_ID > Dockerfile
This is derived from #fallino's answer, with some adjustments and simplifications by using the output format option for docker history. Since macOS and Gnu/Linux have different command-line utilities, a different version is necessary for Mac. If you only need one or the other, you can just use those lines.
#!/bin/bash
case "$OSTYPE" in
linux*)
docker history --no-trunc --format "{{.CreatedBy}}" $1 | # extract information from layers
tac | # reverse the file
sed 's,^\(|3.*\)\?/bin/\(ba\)\?sh -c,RUN,' | # change /bin/(ba)?sh calls to RUN
sed 's,^RUN #(nop) *,,' | # remove RUN #(nop) calls for ENV,LABEL...
sed 's, *&& *, \\\n \&\& ,g' # pretty print multi command lines following Docker best practices
;;
darwin*)
docker history --no-trunc --format "{{.CreatedBy}}" $1 | # extract information from layers
tail -r | # reverse the file
sed -E 's,^(\|3.*)?/bin/(ba)?sh -c,RUN,' | # change /bin/(ba)?sh calls to RUN
sed 's,^RUN #(nop) *,,' | # remove RUN #(nop) calls for ENV,LABEL...
sed $'s, *&& *, \\\ \\\n \&\& ,g' # pretty print multi command lines following Docker best practices
;;
*)
echo "unknown OSTYPE: $OSTYPE"
;;
esac
It is not possible at this point (unless the author of the image explicitly included the Dockerfile).
However, it is definitely something useful! There are two things that will help to obtain this feature.
Trusted builds (detailed in this docker-dev discussion
More detailed metadata in the successive images produced by the build process. In the long run, the metadata should indicate which build command produced the image, which means that it will be possible to reconstruct the Dockerfile from a sequence of images.
If you are interested in an image that is in the Docker hub registry and wanted to take a look at Dockerfile?.
Example:
If you want to see the Dockerfile of image "jupyter/datascience-notebook" type the word "Dockerfile" in the address bar of your browser as shown below.
https://hub.docker.com/r/jupyter/datascience-notebook/
https://hub.docker.com/r/jupyter/datascience-notebook/Dockerfile
Note:
Not all the images have Dockerfile, for example, https://hub.docker.com/r/redislabs/redisinsight/Dockerfile
Sometimes this way is much faster than searching for Dockerfile in Github.
docker pull chenzj/dfimage
alias dfimage="docker run -v /var/run/docker.sock:/var/run/docker.sock --rm chenzj/dfimage"
dfimage image_id
Below is the output of the dfimage command:
$ dfimage 0f1947a021ce
FROM node:8
WORKDIR /usr/src/app
COPY file:e76d2e84545dedbe901b7b7b0c8d2c9733baa07cc821054efec48f623e29218c in ./
RUN /bin/sh -c npm install
COPY dir:a89a4894689a38cbf3895fdc0870878272bb9e09268149a87a6974a274b2184a in .
EXPOSE 8080
CMD ["npm" "start"]
it is possible in just two step. First pull the image then run docker history command. also, shown in SS.
docker pull kalilinux/kali-rolling
docker history --format "{{.CreatedBy}}" kalilinux/kali-rolling --no-trunc
What is image2df
image2df is tool for Generate Dockerfile by an image.
This tool is very useful when you only have docker image and need to generate a Dockerfile whit it.
How does it work
Reverse parsing by history information of an image.
How to use this image
# Command alias
echo "alias image2df='docker run -v /var/run/docker.sock:/var/run/docker.sock --rm cucker/image2df'" >> ~/.bashrc
. ~/.bashrc
# Excute command
image2df <IMAGE>
See help
docker run --rm cucker/image2df --help
For example
$ echo "alias image2df='docker run -v /var/run/docker.sock:/var/run/docker.sock --rm cucker/image2df'" >> ~/.bashrc
$ . ~/.bashrc
$ docker pull mysql
$ image2df mysql
========== Dockerfile ==========
FROM mysql:latest
RUN groupadd -r mysql && useradd -r -g mysql mysql
RUN apt-get update && apt-get install -y --no-install-recommends gnupg dirmngr && rm -rf /var/lib/apt/lists/*
ENV GOSU_VERSION=1.12
RUN set -eux; \
savedAptMark="$(apt-mark showmanual)"; \
apt-get update; \
apt-get install -y --no-install-recommends ca-certificates wget; \
rm -rf /var/lib/apt/lists/*; \
dpkgArch="$(dpkg --print-architecture | awk -F- '{ print $NF }')"; \
wget -O /usr/local/bin/gosu "https://github.com/tianon/gosu/releases/download/$GOSU_VERSION/gosu-$dpkgArch"; \
wget -O /usr/local/bin/gosu.asc "https://github.com/tianon/gosu/releases/download/$GOSU_VERSION/gosu-$dpkgArch.asc"; \
export GNUPGHOME="$(mktemp -d)"; \
gpg --batch --keyserver hkps://keys.openpgp.org --recv-keys B42F6819007F00F88E364FD4036A9C25BF357DD4; \
gpg --batch --verify /usr/local/bin/gosu.asc /usr/local/bin/gosu; \
gpgconf --kill all; \
rm -rf "$GNUPGHOME" /usr/local/bin/gosu.asc; \
apt-mark auto '.*' > /dev/null; \
[ -z "$savedAptMark" ] || apt-mark manual $savedAptMark > /dev/null; \
apt-get purge -y --auto-remove -o APT::AutoRemove::RecommendsImportant=false; \
chmod +x /usr/local/bin/gosu; \
gosu --version; \
gosu nobody true
RUN mkdir /docker-entrypoint-initdb.d
RUN apt-get update && apt-get install -y --no-install-recommends \
pwgen \
openssl \
perl \
xz-utils \
&& rm -rf /var/lib/apt/lists/*
RUN set -ex; \
key='A4A9406876FCBD3C456770C88C718D3B5072E1F5'; \
export GNUPGHOME="$(mktemp -d)"; \
gpg --batch --keyserver ha.pool.sks-keyservers.net --recv-keys "$key"; \
gpg --batch --export "$key" > /etc/apt/trusted.gpg.d/mysql.gpg; \
gpgconf --kill all; \
rm -rf "$GNUPGHOME"; \
apt-key list > /dev/null
ENV MYSQL_MAJOR=8.0
ENV MYSQL_VERSION=8.0.24-1debian10
RUN echo 'deb http://repo.mysql.com/apt/debian/ buster mysql-8.0' > /etc/apt/sources.list.d/mysql.list
RUN { \
echo mysql-community-server mysql-community-server/data-dir select ''; \
echo mysql-community-server mysql-community-server/root-pass password ''; \
echo mysql-community-server mysql-community-server/re-root-pass password ''; \
echo mysql-community-server mysql-community-server/remove-test-db select false; \
} | debconf-set-selections \
&& apt-get update \
&& apt-get install -y \
mysql-community-client="${MYSQL_VERSION}" \
mysql-community-server-core="${MYSQL_VERSION}" \
&& rm -rf /var/lib/apt/lists/* \
&& rm -rf /var/lib/mysql && mkdir -p /var/lib/mysql /var/run/mysqld \
&& chown -R mysql:mysql /var/lib/mysql /var/run/mysqld \
&& chmod 1777 /var/run/mysqld /var/lib/mysql
VOLUME [/var/lib/mysql]
COPY dir:2e040acc386ebd23b8571951a51e6cb93647df091bc26159b8c757ef82b3fcda in /etc/mysql/
COPY file:345a22fe55d3e6783a17075612415413487e7dba27fbf1000a67c7870364b739 in /usr/local/bin/
RUN ln -s usr/local/bin/docker-entrypoint.sh /entrypoint.sh # backwards compat
ENTRYPOINT ["docker-entrypoint.sh"]
EXPOSE 3306 33060
CMD ["mysqld"]
reference

Resources