I ran into some problems running elasticsearch on a debian-based container, a combination of not being able to find log files and not being able to run as root. But even symlinking the config files didn't work for me. I was able to get it running and thought it might be helpful for somebody.
I installed elasticsearch with this command:
wget -qO - https://packages.elastic.co/GPG-KEY-elasticsearch | apt-key add - \
&& echo "deb http://packages.elastic.co/elasticsearch/2.x/debian stable main" | tee -a /etc/apt/sources.list.d/elasticsearch-2.x.list \
&& apt-get update \
&& apt-get install -y elasticsearch \
&& update-rc.d elasticsearch defaults 95 10
Update: See comment about deprecation from #Morfie
Setting the es.insecure.allow.root flag has been removed: https://github.com/elastic/elasticsearch/commit/3ccd59592a46aa26b0675025248a69b4ade3d516
Previous Answer:
Dockerfile:
RUN mkdir /usr/share/elasticsearch/config \
&& ln /etc/elasticsearch/logging.yml /usr/share/elasticsearch/config/ \
&& ln /etc/elasticsearch/elasticsearch.yml /usr/share/elasticsearch/config/ \
&& chmod 774 /usr/share/elasticsearch/config
COPY supervisord.conf /etc/supervisor/conf.d/supervisord.conf
CMD ["/usr/bin/supervisord"]
And elasticsearch entry in supervisord.conf
[program:elasticsearch]
command=/usr/share/elasticsearch/bin/elasticsearch -Des.insecure.allow.root=true
Related
I'm a beginner in neo4j. I'm trying to build tests using embedded neo4j database inside a springboot application. I haven't succeeded since the class DatabaseManagementServiceBuilder is found nowhere Please note I'm using version 4.0.2 Any help please ?
The full classname is org.neo4j.dbms.api.DatabaseManagementServiceBuilder.
Here is a sample class that uses the builder.
I've struggled with using an embedded neo4j db for my tests a few months back as well.
In case you don't find a suitable solution for the embedded version, I ended up starting a real instance of the db...
I adjusted a bit neo4j's official Dockerfile to use jdk instead of jre and was able to run my tests against it.
Here's the Dockerfile, starting from the official 3.4.5-enterprise Dockerfile:
FROM openjdk:8-jdk-slim
RUN apt-get update && apt-get install -y --no-install-recommends \
make && \
rm -rf /var/lib/apt/lists/*
ADD maven-settings.xml /root/.m2/settings.xml
# from official neo4j:3.4.5-enterprise image content (changed base image to use jdk instead of jre)
RUN addgroup --system neo4j && adduser --system --no-create-home --home /var/lib/neo4j --ingroup neo4j neo4j
ENV NEO4J_SHA256=0629f17a99ba90d6900c98f332c775a732cc2ad6298b8df41a2872277b19e6e3 \
NEO4J_TARBALL=neo4j-enterprise-3.4.5-unix.tar.gz \
NEO4J_EDITION=enterprise \
NEO4J_ACCEPT_LICENSE_AGREEMENT=yes \
TINI_VERSION="v0.18.0" \
TINI_SHA256="12d20136605531b09a2c2dac02ccee85e1b874eb322ef6baf7561cd93f93c855"
ARG NEO4J_URI=http://dist.neo4j.org/neo4j-enterprise-3.4.5-unix.tar.gz
RUN apt update \
&& apt install -y \
bash \
curl \
&& curl -L --fail --silent --show-error "https://github.com/krallin/tini/releases/download/${TINI_VERSION}/tini" > /sbin/tini \
&& echo "${TINI_SHA256} /sbin/tini" | sha256sum -c --strict --quiet \
&& chmod +x /sbin/tini \
&& curl --fail --silent --show-error --location --remote-name ${NEO4J_URI} \
&& echo "${NEO4J_SHA256} ${NEO4J_TARBALL}" | sha256sum -c --strict --quiet \
&& tar --extract --file ${NEO4J_TARBALL} --directory /var/lib \
&& mv /var/lib/neo4j-* /var/lib/neo4j \
&& rm ${NEO4J_TARBALL} \
&& mv /var/lib/neo4j/data /data \
&& chown -R neo4j:neo4j /data \
&& chmod -R 777 /data \
&& chown -R neo4j:neo4j /var/lib/neo4j \
&& chmod -R 777 /var/lib/neo4j \
&& ln -s /data /var/lib/neo4j/data
# Install latest su-exec
RUN set -ex; \
\
curl -o /usr/local/bin/su-exec.c https://raw.githubusercontent.com/ncopa/su-exec/master/su-exec.c; \
\
fetch_deps='gcc libc-dev'; \
apt-get update; \
apt-get install -y --no-install-recommends $fetch_deps; \
rm -rf /var/lib/apt/lists/*; \
gcc -Wall \
/usr/local/bin/su-exec.c -o/usr/local/bin/su-exec; \
chown root:root /usr/local/bin/su-exec; \
chmod 0755 /usr/local/bin/su-exec; \
rm /usr/local/bin/su-exec.c; \
\
apt-get purge -y --auto-remove $fetch_deps
ENV PATH /var/lib/neo4j/bin:$PATH
ARG NEO4J_AUTH=neo4j/neo4jtest
ENV NEO4J_AUTH=${NEO4J_AUTH}
WORKDIR /var/lib/neo4j
VOLUME /data
COPY docker-entrypoint.sh /docker-entrypoint.sh
EXPOSE 7474 7473 7687
ENTRYPOINT ["/sbin/tini", "-g", "--", "/docker-entrypoint.sh"]
CMD ["neo4j"]
I used the original docker-entrypoint.sh script.
I have a docker container that serves a webserver. On every startup of the container, I want to excecute a little shell script. The script that has to be executed has only one statement.
/var/www/html/app/Console/cake schema update -y
To achieve this, I created a .sh file called schemaupdate.sh which I copy into the docker container using the dockerfile into the /etc/init.d folder. Furthermore I make it executable and register it to the startup.
COPY schemaupdate.sh /etc/init.d/schemaupdate.sh
chmod 755 /etc/init.d/schemaupdate.sh
update-rc.d schemaupdate.sh defaults
The file is successfully copied into the container. However, the script is not executed when the docker container starts. When I manually call the sh file, everything is running fine.
How can I achieve, that the file / statement is executed on each startup of a container? It is important, that the script is executed at the startup and the container (so the webserver) still continues to run! The script only makes a little update check and after the check the webserver keeps on going.
The container is a debian based container. Here is inital dockerfile.
#start with base Image from php
FROM php:7.3-apache
#install system dependencies and enable PHP modules
RUN apt-get update && apt-get install -y \
libicu-dev \
libpq-dev \
libmcrypt-dev \
mysql-client \
git \
zip \
unzip \
&& rm -r /var/lib/apt/lists/* \
&& docker-php-ext-configure pdo_mysql --with-pdo-mysql=mysqlnd \
&& docker-php-ext-install \
intl \
mbstring \
pcntl \
pdo_mysql \
pdo_pgsql \
pgsql \
opcache
# zip \
# mcrypt \
#configure imap for mails
RUN apt-get update && \
apt-get install -y \
libc-client-dev libkrb5-dev && \
rm -r /var/lib/apt/lists/*
RUN docker-php-ext-configure imap --with-kerberos --with-imap-ssl && \
docker-php-ext-install -j$(nproc) imap
#install mcrypt
RUN apt-get update \
&& apt-get install -y libmcrypt-dev \
&& rm -rf /var/lib/apt/lists/* \
&& pecl install mcrypt-1.0.2 \
&& docker-php-ext-enable mcrypt
#install composer
RUN curl -sS https://getcomposer.org/installer | php -- --install-dir=/usr/bin/ --filename=composer
#set our application folder as an environment variable
ENV APP_HOME /var/www/html
#change uid and gid of apache to docker user uid/gid
RUN usermod -u 1000 www-data && groupmod -g 1000 www-data
#change the web_root to cakephp /var/www/html/webroot folder
#RUN sed -i -e "s/html/html\/webroot/g" /etc/apache2/sites-enabled/000-default.conf
# enable apache module rewrite
RUN a2enmod rewrite
#copy source files and run composer
#COPY src/ /var/www/html
#COPY src/ $APP_HOME
# install all PHP dependencies
#RUN composer install --no-interaction
#SET Volume
VOLUME /var/www/html/
#change ownership of our applications
RUN chown -R www-data:www-data $APP_HOME
#SET ENV VARIABLES
COPY schemaupdate.sh /etc/init.d/schemaupdate.sh
chmod 755 /etc/init.d/schemaupdate.sh
update-rc.d schemaupdate.sh defaults
EXPOSE 80
/etc/init.d/ isn't relevant. Containers aren't full blown operating systems with a heavyweight SysV init-style startup sequence. They run a single command, that's it.
You should either add the command as a RUN statement in the Dockerfile so its results are baked into the image, or you should have it called directly by the container's CMD or ENTRYPOINT directive.
I Finally used the Entrypoint. I deleted the COPY, chmod and update-rc. The Entrypoint looks like the following.
ENTRYPOINT [ "sh", "-c", "/var/www/html/app/Console/cake schema update -y && /var/www/html/app/Console/cake schema update -y && /usr/sbin/apachectl -D FOREGROUND"]
It first starts the update statment. After this is finished (so terminated), the apachectl is called to keep the webserver running.
my dockerfile:
FROM AWS_ECR_IMAGE
RUN apt-get update && apt-get install -y \
cron \
python-dev \
git \
zlib1g-dev \
libffi-dev \
libssl-dev \
autotools-dev \
automake \
libbz2-dev \
libaio-dev \
libsasl2-dev \
python-pip
RUN pip install boto boto3 awscli
# Install Nginx.
RUN \
add-apt-repository -y ppa:nginx/stable && \
apt-get update && \
apt-get install -y nginx && \
rm -rf /var/lib/apt/lists/* && \
echo "\ndaemon off;" >> /etc/nginx/nginx.conf && \
chown -R www-data:www-data /var/lib/nginx
# Define mountable directories.
VOLUME ["/etc/nginx/sites-enabled", "/etc/nginx/certs", "/etc/nginx/conf.d", "/var/log/nginx", "/var/www/html"]
# Define working directory.
WORKDIR /etc/nginx
# Define default command.
CMD ["nginx"]
COPY nginx_conf /etc/nginx/sites-available/default
# Start service, replace server name, update web ui
COPY main.sh /opt/annotation-pipeline-docs/main.sh
RUN chmod 0755 /opt/annotation-pipeline-docs/main.sh
ENTRYPOINT [ "sh", "-c", "/opt/annotation-pipeline-docs/main.sh" ]
# Expose ports.
EXPOSE 80
And my entrypoint bash file (I need to update the server name first when the container runs) is:
#!/bin/bash -e
/usr/local/bin/aws s3 sync s3://${S3_Bucket}/docs/${ENVIRONMENT}/HEAD/ /var/www/html/
if [ "$ENVIRONMENT" == "prod" ]
then
sed -i.bak "s/REPLACE_ME/example.com/g" /etc/nginx/sites-available/default
else
sed -i.bak "s/REPLACE_ME/example-$ENVIRONMENT.com/g" /etc/nginx/sites-available/default
fi
nginx
while true; do
sleep 60
echo "s3 sync again:"
/usr/local/bin/aws s3 sync s3://${S3_Bucket}/docs/${ENVIRONMENT}/HEAD/ /var/www/html/
done
The issue is when
nginx
runs, it will hanging forever in the terminal:
and the while loop will never get called. Anyone know why is hanging and how to resolve it? Please help, tks in advanced.
The reason for my issue is waiting for the traffic, the while loop will never get called until Nginx start free the bash. However, Nginx will be running in the foreground and not release the focus.
The solution I tried is instead of letting Nginx running as a foreground service, I changed it run in the background. Since this is the only service in my container, should have no problem to do it.
the code changed is simply removed below line in my dockerfile:
echo "\ndaemon off;" >> /etc/nginx/nginx.conf
which will make Nginx as a foreground service
I am using this http://fhirtest.uhn.ca/baseDstu2 test FHIR server and it worked okay so far.
Now I am getting an HTTP-500 - Failed to Call Access Method exception.
Anyone has any idea on what has gone wrong?
This happens frequently. Probably because someone tested weird queries or similar that put the server in an unstable status.
I suggest posting a comment in https://chat.fhir.org/#narrow/stream/hapi to get the server restarted,
or install http://hapifhir.io/doc_cli.html which does basically the same but you have full control.
I built a Dockerfile:
FROM debian:sid
MAINTAINER Günter Zöchbauer <guenter#yyy.com>
ENV DEBIAN_FRONTEND noninteractive
RUN \
apt-get -q update && \
DEBIAN_FRONTEND=noninteractive && \
apt-get install --no-install-recommends -y -q \
apt-transport-https \
apt-utils \
wget \
bzip2 \
default-jdk
# net-tools sudo procps telnet
RUN \
apt-get update && \
rm -rf /var/lib/apt/lists/*
https://github.com/jamesagnew/hapi-fhir/releases/download/v2.0/hapi-fhir-2.0-cli.tar.bz2 && \
ADD hapi-* /hapi_fhir_cli/
RUN ls -la
RUN ls -la /hapi_fhir_cli
ADD prepare_server.sh /hapi_fhir_cli/
RUN \
cd /hapi_fhir_cli && \
bash -c /hapi_fhir_cli/prepare_server.sh
ADD start.sh /hapi_fhir_cli/
WORKDIR /hapi_fhir_cli
EXPOSE 5555
ENTRYPOINT ["/hapi_fhir_cli/start.sh"]
Which requires in the same directory as the Dockerfile
prepare_server.sh
#!/usr/bin/env bash
ls -la
./hapi-fhir-cli run-server --allow-external-refs &
while ! timeout 1 bash -c "echo > /dev/tcp/localhost/8080"; do sleep 10; done
./hapi-fhir-cli upload-definitions -t http://localhost:8080/baseDstu2
./hapi-fhir-cli upload-examples -c -t http://localhost:8080/baseDstu2
start.sh
#!/usr/bin/env bash
cd /hapi_fhir_cli
./hapi-fhir-cli run-server --allow-external-refs -p 5555
Build
docker build myname/hapi_fhir_cli_dstu2 -t . #--no-cache
Run
docker run -d -p 5555:5555 [image id from docker build]
Hope this helps.
I downloaded docker files from official repository (version 2.3), and now I want to build the image and upload some local data (test.json) into the container. It is not enough just to run COPY test.json /usr/share/elasticsearch/data/, because in this case the indexing of data is not done.
What I want to achieve is to be able to run sudo docker run -d -p 9200:9200 -p 9300:9300 -v /home/gosper/tests/tempESData/:/usr/share/elasticsearch/data test/elasticsearch, and after its execution I want to be able to see the mapped data on http://localhost:9200/tests/test/999.
If I use the below-given Dockerfile and *sh script, then I get the following error: Failed to connect to localhost port 9200: Connection refused
This is the Dockerfile from which I build the image:
FROM java:8-jre
# grab gosu for easy step-down from root
ENV GOSU_VERSION 1.7
RUN set -x \
&& wget -O /usr/local/bin/gosu "https://github.com/tianon/gosu/releases/download/$GOSU_VERSION/gosu-$(dpkg --print-architecture)" \
&& wget -O /usr/local/bin/gosu.asc "https://github.com/tianon/gosu/releases/download/$GOSU_VERSION/gosu-$(dpkg --print-architecture).asc" \
&& export GNUPGHOME="$(mktemp -d)" \
&& gpg --keyserver ha.pool.sks-keyservers.net --recv-keys B42F6819007F00F88E364FD4036A9C25BF357DD4 \
&& gpg --batch --verify /usr/local/bin/gosu.asc /usr/local/bin/gosu \
&& rm -r "$GNUPGHOME" /usr/local/bin/gosu.asc \
&& chmod +x /usr/local/bin/gosu \
&& gosu nobody true
# https://www.elastic.co/guide/en/elasticsearch/reference/current/setup-repositories.html
# https://packages.elasticsearch.org/GPG-KEY-elasticsearch
RUN apt-key adv --keyserver ha.pool.sks-keyservers.net --recv-keys 46095ACC8548582C1A2699A9D27D666CD88E42B4
ENV ELASTICSEARCH_VERSION 2.3.4
ENV ELASTICSEARCH_REPO_BASE http://packages.elasticsearch.org/elasticsearch/2.x/debian
RUN echo "deb $ELASTICSEARCH_REPO_BASE stable main" > /etc/apt/sources.list.d/elasticsearch.list
RUN set -x \
&& apt-get update \
&& apt-get install -y --no-install-recommends elasticsearch=$ELASTICSEARCH_VERSION \
&& rm -rf /var/lib/apt/lists/*
ENV PATH /usr/share/elasticsearch/bin:$PATH
WORKDIR /usr/share/elasticsearch
RUN set -ex \
&& for path in \
./data \
./logs \
./config \
./config/scripts \
; do \
mkdir -p "$path"; \
chown -R elasticsearch:elasticsearch "$path"; \
done
COPY config ./config
VOLUME /usr/share/elasticsearch/data
COPY docker-entrypoint.sh /
EXPOSE 9200 9300
RUN chmod +x /docker-entrypoint.sh
ENTRYPOINT ["/docker-entrypoint.sh"]
CMD ["elasticsearch"]
COPY template.json /usr/share/elasticsearch/data/
RUN /bin/bash -c "source /docker-entrypoint.sh"
This is the docker-entrypoint.sh in which I added the line curl -XPOST http://localhost:9200/uniko-documents/document/978-1-60741-503-9 -d "/usr/share/elasticsearch/data/template.json":
#!/bin/bash
set -e
# Add elasticsearch as command if needed
if [ "${1:0:1}" = '-' ]; then
set -- elasticsearch "$#"
fi
# Drop root privileges if we are running elasticsearch
# allow the container to be started with `--user`
if [ "$1" = 'elasticsearch' -a "$(id -u)" = '0' ]; then
# Change the ownership of /usr/share/elasticsearch/data to elasticsearch
chown -R elasticsearch:elasticsearch /usr/share/elasticsearch/data
set -- gosu elasticsearch "$#"
#exec gosu elasticsearch "$BASH_SOURCE" "$#"
fi
curl -XPOST http://localhost:9200/tests/test/999 -d "/usr/share/elasticsearch/data/test.json"
# As argument is not related to elasticsearch,
# then assume that user wants to run his own process,
# for example a `bash` shell to explore this image
exec "$#"
Remove the following from your docker-entrypoint.sh:
curl -XPOST http://localhost:9200/tests/test/999 -d "/usr/share/elasticsearch/data/test.json"
It's running before you exec the service at the end.
In your Dockerfile, move the following after any commands that modify the directory:
VOLUME /usr/share/elasticsearch/data
Once you create a volume, future changes to the directory are typically ignored.
Lastly, in your Dockerfile, this line at the end likely doesn't do what you think, I'd remove it:
RUN /bin/bash -c "source /docker-entrypoint.sh"
The entrypoint.sh should be run when you start the container, not when you're building it.
#Klue in case you still need it.. you need to change the -d option on your curl command to --data-binary. -d strips the newlines. that's why you are getting the errors.