This question has been asked here in many ways, but none of them seemed to be exactly what I'm looking for. Note: I'm totally new to Docker.
I've currently got a Docker image ("padcrawler") on my Mac, which is an image built off of scrapinghub/splash.
I'm basically just trying to make it so that any changes I make on my host machine (Mac), are automatically updated within the container. I'm using docker-machine and NOT boot2docker.
This is my Dockerfile:
FROM scrapinghub/splash:latest
# Update UBUNTU base image and install deps
RUN apt-get update \
&& apt-get upgrade -y \
&& apt-get install -y \
build-essential \
ca-certificates \
gcc \
git \
libpq-dev \
make \
libxml2-dev \
libxslt-dev \
libssl-dev \
libffi-dev \
zlib1g-dev \
python-pip \
python \
python-dev \
ssh \
&& apt-get autoremove \
&& apt-get clean
RUN mkdir -p /usr/src/pad
WORKDIR /usr/src/pad
# Copy over everything to container directory
COPY . /usr/src/pad
# Install Python (2.7) deps
RUN /usr/bin/pip install -r /usr/src/pad/requirements.txt
# Launch pad-crawler upon docker container bootup
ENTRYPOINT /usr/src/pad/start
I'm also using docker-compose and this is my yml file:
version: '2'
services:
crawler:
build: .
volumes:
- ".:/usr/src/pad"
image: padcrawler
However, I'm still unable to get any host file changes to automatically sync with the mounted volume in the container. Obviously it's annoying and frustrating to have to manually copy over the changes every-time. Is there a bigger issue here? I'm using Docker totally wrong?
Related
Present are two files, Dockerfile.infra and docker-compose-infra.yml. Firstly, docker-compose-infra.yml is built via the following command:
docker-compose --file docker-compose-infra.yml build
This results in no errors and finishes as expected.
The problem arises when trying to deploy this to AWS. The following command:
docker-compose --file docker-compose-infra.yml run cdk
Produces this error:
bash: cdk: command not found
This appears to be triggered when the docker-compose-infra.yml attempts to run the cdk deploy bash command.
The command should run because within the Dockerfile.infra build, cdk is installed via npm install -g aws-cdk-lib.
Dockerfile.infra file:
FROM node:16-slim
ENV DEBIAN_FRONTEND=noninteractive
RUN npm install -g aws-cdk-lib \
&& apt-get update -y \
&& apt-get upgrade -y \
&& apt-get install -y --no-install-recommends \
# install Python
python3-pip \
# install Poetry via curl
curl \
&& curl -k https://install.python-poetry.org | python3 - \
&& apt-get remove curl -y \
&& apt-get autoremove -y \
&& rm -rf /var/lib/apt/lists/*
COPY pyproject.toml poetry.lock /
ENV PATH=/root/.local/bin:$PATH
RUN poetry config virtualenvs.create false \
&& poetry install --no-dev
WORKDIR /app/
COPY app.py cdk.json cdk.context.json /app/
COPY stacks/ /app/stacks/
docker-compose-infra.yml:
version: "3"
services:
cdk:
command: bash -c "cdk deploy --require-approval never --all --parameters my-app-${ENVIRONMENT}-service:MyServiceImageTag=${IMAGE_TAG}"
build:
context: ./
dockerfile: Dockerfile.infra
environment:
- AWS_PROFILE=${AWS_PROFILE}
- ENVIRONMENT=${ENVIRONMENT}
- DEPLOY_ACCOUNT=${DEPLOY_ACCOUNT}
volumes:
- ~/.aws/credentials:/root/.aws/credentials
You need to install aws-cdk not aws-cdk-lib
RUN npm install -g aws-cdk \
This might be a bit confusing because aws-cdk-lib is also the name of the required Python dependency when writing Python CDK apps and a valid npm package.
I am setting up a new Laravel project and automating the installation process using docker.
Everything works, but the problem here is that the PhpStorm doesn't recognize some namespaces because the vendor folder is a volume. I will show the files to understand them better.
docker-compose.yml
version: '3'
services:
laravel.test:
build:
dockerfile: Dockerfile
ports:
- '${APP_PORT:-8080}:80'
volumes:
- '.:/var/www/html'
- /var/www/html/vendor #HERE IS THE VOLUME FOR VENDOR
Dockerfile
FROM php:8.1.2-fpm
#INSTALL DEPENDENCIES
RUN apt-get update && apt-get install -y \
openssl \
build-essential \
libfreetype6-dev \
libjpeg62-turbo-dev \
libpng-dev \
locales \
apt-transport-https \
wget \
lsb-release \
libxml2-dev \
ca-certificates \
git \
nano \
libonig-dev \
zip
#PGSQL
RUN apt-get install -y libpq-dev \
&& docker-php-ext-configure pgsql -with-pgsql=/usr/local/pgsql \
&& docker-php-ext-install pdo pdo_pgsql pgsql
#Libraries required for Laravel
RUN docker-php-ext-install \
pdo_mysql \
bcmath \
ctype \
fileinfo \
iconv \
gd \
mbstring \
pdo_mysql \
xml \
pcntl
# Install composer
RUN php -r "readfile('https://getcomposer.org/installer');" | php -- --install-dir=/usr/bin/ --filename=composer
COPY ./ /var/www/html
# Copy code to $path
RUN composer install
EXPOSE 9000
CMD ["php-fpm"]
.dockerignore
vendor
So after building the image and running composer install, the vendor folder is created in the container with all dependencies and in the host machine but without subfolders, as we can see in the following image:
And when I try to import any namespace, I receive an error.
Nothing wrong in PHPStorm. Your vendor folder is empty.
I used to run docker-composer without mapping volume for vendor.
Remove the mapping and log in to service with bash and run the composer install
Steps
Remove the - /var/www/html/vendor volume mapping
Log in to service using docker exec -it laravel.test bash (FYI: It's better if you use container_name: search it.)
run the command composer install or composer update
Note: Before running remove the existing vendor folder.(delete or rename it _vendor for temp)
Wild guess: check you volums whether you defined volume.
I'm trying to build a docker image but it throws an error and I can't seem to figure out why.
It is stuck at RUN apt-get -y update with the following error messages:
4.436 E: Release file for http://security.debian.org/debian-security/dists/buster/updates/InRelease is not valid yet (invalid for another 2d 16h 26min 22s). Updates for this repository will not be applied.
4.436 E: Release file for http://deb.debian.org/debian/dists/buster-updates/InRelease is not valid yet (invalid for another 3d 10h 28min 24s). Updates for this repository will not be applied.
executor failed running [/bin/sh -c apt-get -y update]: exit code: 100
Here's my docker file:
FROM python:3.7
# Adding trusting keys to apt for repositories
RUN wget -q -O - https://dl-ssl.google.com/linux/linux_signing_key.pub | apt-key add -
# Adding Google Chrome to the repositories
RUN sh -c 'echo "deb [arch=amd64] http://dl.google.com/linux/chrome/deb/ stable main" >> /etc/apt/sources.list.d/google-chrome.list'
# Updating apt to see and install Google Chrome
RUN apt-get -y update
# Magic happens
RUN apt-get install -y google-chrome-stable
# Installing Unzip
RUN apt-get install -yqq unzip
# Download the Chrome Driver
RUN CHROMEDRIVER_RELEASE=$(curl http://chromedriver.storage.googleapis.com/LATEST_RELEASE) && \
echo "Chromedriver latest version: $CHROMEDRIVER_RELEASE" && \
wget --quiet "http://chromedriver.storage.googleapis.com/$CHROMEDRIVER_RELEASE/chromedriver_linux64.zip" && \
unzip chromedriver_linux64.zip && \
rm -rf chromedriver_linux64.zip && \
mv chromedriver /usr/local/bin/chromedriver && \
chmod +x /usr/local/bin/chromedriver && \
chromedriver --version
# Set display port as an environment variable
ENV DISPLAY=:99
WORKDIR /
COPY requirements.txt ./
RUN pip install --upgrade pip && pip install -r requirements.txt
COPY . .
RUN pip install -e .
What is happening here?
In my case, docker was still using the cached RUN apt update && apt upgrade command, thus not updating the package sources.
The solution was to build the docker image once with the --no-cache flag:
docker build --no-cache .
If you are using docker desktop, please check if enough resources are set in settings/preferences.
Eg. memory and disk requirement
It's answered here https://askubuntu.com/questions/1059217/getting-release-is-not-valid-yet-while-updating-ubuntu-docker-container
Correct your system clock. (in comments I also suggested checking for a mismatch between clock and your timezone too)
I get this ERROR: executor failed running [...]: exit code: 100 error message when I mistyped the name of a package.
This was in my Dockerfile:
RUN sudo apt-get update; \
sudo apt-get -y upgrade; \
sudo apt-get install -y gnupg2 wget lsb_release
instead of this:
RUN sudo apt-get update; \
sudo apt-get -y upgrade; \
sudo apt-get install -y gnupg2 wget lsb-release
(see the difference between the underscore and the dash.)
Fixing the package name solved the problem.
This happens specific to OS also.
I had same issues running MariaDB on my Windows 10.
Check for Docker Settings:
{
"registry-mirrors": [],
"insecure-registries": [],
"debug": false,
"experimental": false,
"features": {
"buildkit": true
},
"builder": {
"gc": {
"enabled": true,
"defaultKeepStorage": "20GB"
}
}
}
Remove below block, and it should work:
"features": {
"buildkit": true
},
I had this error and I think it was because I installed buildx but the version of the plugin didn't match my docker installation. Uninstalling buildx resolved the issue for me:
docker buildx uninstall
For me adding this to the Dockerfile did the job:
RUN apk add --update linux-headers;
I am using alpine linux as a base image, and I need to install an oracle client native library. I believe you can download from here:
https://www.oracle.com/technetwork/topics/linuxx86-64soft-092277.html
it looks like I have to login to download, does anyone know how to download a zip file of the client lib without login?
does anyone know how to install the client library properly in a bash script or dockerfile?
I have figure out some different way to install Oracle instant client in ubuntu Docker, it might help others
Follow these simple steps:
Download oracle instant client (.rpm file) from oracle official download center
Convert into .deb (you can use apt-get install alien ) and move somewhere in your working directory.
Now Update your Dockerfile and make build
RUN apt-get update
WORKDIR /opt
ADD ./ORACLE-INSTANT-CLIENT.deb /opt
#if libaio also required
RUN apt-get install libaio1
RUN dpkg -i oracle-instantclient.deb
Here is a working solution for the official PHP-FPM images based on Debian 10 (Buster). The following Dockerfile installs the Oracle Instant Client 18.5 (basiclite and devel) using the RPM packges and alien.
As Christopher Jones wrote the files can currently be downloaded without an Oracle account.
FROM php:7.2.32-fpm
# see https://help.ubuntu.com/community/Oracle%20Instant%20Client
RUN apt-get update && apt-get install -y --no-install-recommends alien libaio1 wget && \
wget https://download.oracle.com/otn_software/linux/instantclient/185000/oracle-instantclient18.5-basiclite-18.5.0.0.0-3.x86_64.rpm && \
wget https://download.oracle.com/otn_software/linux/instantclient/185000/oracle-instantclient18.5-devel-18.5.0.0.0-3.x86_64.rpm && \
alien -i oracle-instantclient18.5-basiclite-18.5.0.0.0-3.x86_64.rpm && \
alien -i oracle-instantclient18.5-devel-18.5.0.0.0-3.x86_64.rpm
ENV LD_LIBRARY_PATH="/usr/lib/oracle/18.5/client64/lib:${LD_LIBRARY_PATH}"
If you want to download oracle at runtime then you can run the below commands
FROM ruby:3.0
ENV LD_LIBRARY_PATH=/opt/oracle/instantclient_21_4
RUN apt-get update && \
apt-get install -y libpq-dev zlib1g-dev build-essential shared-mime-info libaio1 libaio-dev unzip wget --no-install-recommends && \
wget https://download.oracle.com/otn_software/linux/instantclient/214000/instantclient-sdk-linux.x64-21.4.0.0.0dbru.zip && \
wget https://download.oracle.com/otn_software/linux/instantclient/214000/instantclient-sqlplus-linux.x64-21.4.0.0.0dbru.zip && \
wget https://download.oracle.com/otn_software/linux/instantclient/214000/instantclient-basic-linux.x64-21.4.0.0.0dbru.zip && \
mkdir -p /opt/oracle && \
cp instantclient-* /opt/oracle/ && \
cd /opt/oracle/ && \
unzip instantclient-basic-linux.x64-21.4.0.0.0dbru.zip && \
unzip instantclient-sdk-linux.x64-21.4.0.0.0dbru.zip && \
unzip instantclient-sqlplus-linux.x64-21.4.0.0.0dbru.zip && \
rm -rf /var/lib/apt/lists/* instantclient-basic-linux.x64-21.4.0.0.0dbru.zip instantclient-sdk-linux.x64-21.4.0.0.0dbru.zip instantclient-sqlplus-linux.x64-21.4.0.0.0dbru.zip && \
apt -y clean && \
apt -y remove wget unzip && \
apt -y autoremove && \
rm -rf /var/cache/apt
You can download the specific version of instantclient by specifying the version above
These two packages are require for ruby-oci if you are using ruby on rails application
libaio1
libaio-dev
You don't want to use Alpine Linux, since you will need to hack it and it could become unstable. See https://stackoverflow.com/a/53291026/4799035 for more comments.
Also see https://github.com/oracle/docker-images/blob/master/OracleInstantClient/dockerfiles/19/Dockerfile which doesn't need any login.
In summary, on Oracle Linux 7:
yum -y install oracle-release-el7
yum -y install oracle-instantclient19.3-basic && rm -rf /var/cache/yum
Update: Oracle has Docker images at https://github.com/oracle/docker-images/pkgs/container/oraclelinux7-instantclient and https://github.com/oracle/docker-images/pkgs/container/oraclelinux8-instantclient which can be pulled like:
docker pull ghcr.io/oracle/oraclelinux7-instantclient:21
and
docker pull ghcr.io/oracle/oraclelinux8-instantclient:21
I'm using docker to put together a script and its get stuck when this package xorg asks the prompt for the country of origin for the keyboard, I'll attach the image.
I've tried piping in the command like this from other answers I've read
RUN echo "29" | apt-get install -y xorg
But it didn't seem to work. I'm not sure how I get this to auto answer 29. Any help is appreciated.
Reference to docker file
FROM ubuntu:16.04
MAINTAINER Joe Astrahan <email#hey.com>
VOLUME ["/var/www"]
#Install Apache & Basic Software
RUN apt-get update && \
apt-get install -y \
software-properties-common \
apache2 \
curl \
libcurl3 \
libcurl3-dev
#Install PHP 7.0 & mod apache
RUN apt-get install -y \
php7.0 \
php7.0-cli \
libapache2-mod-php7.0 \
php7.0-gd \
php7.0-json \
php7.0-ldap \
php7.0-mysqlnd \
php7.0-pgsql \
php7.0-curl \
php7.0-xml \
php7.0-xsl \
php7.0-zip \
php7.0-sqlite3 \
php7.0-ldap \
php7.0-json \
php7.0-mbstring \
php7.0-soap \
php7.0-intl \
php7.0-bcmath \
php7.0-gmp \
php7.0-mcrypt
#Install mysql, vim and openssl
RUN apt-get install -y \
mysql-client \
vim \
openssl
# Copy an initial PHP.ini file into the system with default settings
#COPY config/php.ini /etc/php5/apache2/php.ini
# Install php-5.5.30
#ADD config/install_php-5.5.30.sh /tmp/install_php-5.5.30.sh
#RUN /bin/bash /tmp/install_php-5.5.30.sh
# Set environment variables for Apache so we know its user and group names
ENV APACHE_RUN_USER www-data
ENV APACHE_RUN_GROUP www-data
# Configure Apache SSL and Standard Virtualhosts
COPY config/apache_default.conf /etc/apache2/sites-available/000-default.conf
COPY config/apache_default-ssl.conf /etc/apache2/sites-available/default-ssl.conf
COPY config/run /usr/local/bin/run
# Configure SSL Directories & Create Temporary SSL Keys
RUN mkdir /etc/apache2/ssl
RUN openssl req -x509 -nodes -days 365 -newkey rsa:2048 -keyout /etc/apache2/ssl/apache.key -out /etc/apache2/ssl/apache.crt -subj "/C=US/ST=Florida/L=Fort Lauderdale/O=Pool Service Software LLC/OU=IT Department/CN=dev.poolservice.software.local"
RUN chmod +x /usr/local/bin/run
RUN a2enmod rewrite
#Configure SSL On Apache2 & Headers Mod
RUN a2enmod ssl
RUN a2enmod headers
RUN service apache2 restart
RUN a2ensite default-ssl.conf
RUN service apache2 restart
#RUN apt-get install -y wkhtmltopdf
#Download and install composer and git
RUN apt-get install git -y
RUN curl -sS https://getcomposer.org/installer | php -- --install-dir=/usr/local/bin --filename=composer
#Install Zip & Unzip
RUN apt-get install zip unzip -y
#Install NodeJS
#RUN curl -sL https://deb.nodesource.com/setup_7.x | sudo -E bash -
RUN apt-get install -y nodejs
RUN apt-get install -y npm
#Install UglifyCSS and JS
RUN npm install -g uglify-js
RUN npm install -g uglifycss
RUN npm install -g less
# Download and install wkhtmltopdf
RUN apt-get install -y build-essential
RUN echo "29" | apt-get install -y xorg
RUN apt-get install -y libssl-dev
RUN apt-get install -y libxrender-dev
RUN apt-get install -y wget
RUN apt-get install -y gdebi
RUN apt-get install -y libxrender1
RUN apt-get install -y xfonts-utils
RUN apt-get install -y xfonts-base
RUN apt-get install -y xfonts-75dpi
RUN apt-get install -y libfontenc1
RUN apt-get install -y x11-common
RUN apt-get install -y xfonts-encodings
RUN apt-get install -y libxfont1
RUN apt-get install -y fontconfig
RUN wget http://download.gna.org/wkhtmltopdf/0.12/0.12.2.1/wkhtmltox-0.12.2.1_linux-trusty-amd64.deb
RUN gdebi --n wkhtmltox-0.12.2.1_linux-trusty-amd64.deb
RUN echo 'exec xvfb-run -a -s "-screen 0 640x480x16" wkhtmltopdf "$#"' | tee /usr/local/bin/wkhtmltopdf.sh >/dev/null && sudo chmod a+x /usr/local/bin/wkhtmltopdf.sh
EXPOSE 80
EXPOSE 443
CMD ["/usr/local/bin/run"]
There are quite a few issues I see with the Dockerfile provided.
Defining a volume inside the Dockerfile provides little to no value. It's much better to define this in your docker-compose.yml or as part of your run command. I've got a blog post going through the issues with this.
Splitting up the apt-get update from the later apt-get install commands can result in situations where the apt-get install will fail. There's a section in the best practices about this.
For your error message, I'd run apt-get in the noninteractive mode. You can also preconfigure debconf if you need a non-default value set during install.
Splitting each apt-get into a separate RUN command is creating lots of excessive layers, these should be merged where possible. This is also described in the best practices documentation.
Here's a sample of an install command that works for me taking the above into account:
FROM ubuntu:16.04
RUN apt-get update \
&& DEBIAN_FRONTEND=noninteractive apt-get install -y \
build-essential \
xorg \
libssl-dev \
libxrender-dev \
wget \
gdebi \
libxrender1 \
xfonts-utils \
xfonts-base \
xfonts-75dpi