Can't install nokogiri with bundle when build docker container in archlinux - ruby

Question

Never was, and here again.
ruby 2.5.1p57 (2018-03-29 revision 63029) [x86_64-linux]
Bundler version 1.16.2
Archlinux updated 4.17.4-1 x86_64
Pure ruby project
I can install nokogiri with gem install nokogiri --no-rdoc --no-ri
When I run bundle install --path=vendor inside my ruby project I need to add options to bundler before -- bundle config build.nokogiri --use-system-libraries --with-xml2-include=/usr/include/libxml2 for that installing have success. Without those options he can't.
It fails when I run Dockefile. I need hints where to look more.
FROM archlinux/base
ENV HOME="/srv"
ENV LANG=ru_RU.UTF-8
ENV APP_ENV=production
ENV LANG=ru_RU.UTF-8
ENV PATH=$HOME/sbin:$HOME/bin:$HOME/.gem/ruby/2.5.0/bin:/usr/lib/ruby/gems/2.5.0/bin:/usr/local/sbin:/usr/local/bin:/usr/bin:/usr/bin/site_perl:/usr/bin/vendor_perl:/usr/bin/core_perl
RUN pacman -Syu --noconfirm pkgconf procps-ng iproute2 iputils supervisor ruby make gcc grep postgresql-libs awk && \
gem install bundler --no-ri --no-rdoc --no-document --no-user-install && \
pacman -Scc --noconfirm
COPY ./ /srv/
RUN cd /srv && rm -rf vendor && \
bundle config build.nokogiri --use-system-libraries --with-xml2-include=/usr/include/libxml2 --with-xslt-include=/usr/include/libxslt --with-exslt-include=/usr/include/libexslt --with-xml2-lib=/usr/lib/libxml2.so && \
bundle install --path=vendor --jobs=4 --binstubs
...
$ docker build -t mprj:1.2.0 -f ./info/Dockerfile .
Sending build context to Docker daemon 2.931MB
Step 1/12 : FROM archlinux/base
---> 4702f86ec64f
Step 2/12 : ENV HOME="/srv"
---> Using cache
---> 25d2416a2261
Step 3/12 : ENV LANG=ru_RU.UTF-8
---> Using cache
---> a5f2bb965b81
Step 4/12 : ENV APP_ENV=production
---> Using cache
---> 0ea44026fe0a
Step 5/12 : ENV PATH=$HOME/sbin:$HOME/bin:$HOME/.gem/ruby/2.5.0/bin:/usr/lib/ruby/gems/2.5.0/bin:/usr/local/sbin:/usr/local/bin:/usr/bin:/usr/bin/site_perl:/usr/bin/vendor_perl:/usr/bin/core_perl
---> Running in e4fecc392bb2
Removing intermediate container e4fecc392bb2
---> d26d4a07b769
Step 6/12 : RUN pacman -Syu --noconfirm pkgconf procps-ng iproute2 iputils supervisor ruby make gcc grep postgresql-libs awk && gem install bundler --no-ri --no-rdoc --no-document --no-user-install && pacman -Scc --noconfirm
---> Running in b18a1c4585ee
:: Synchronizing package databases...
downloading core.db...
downloading extra.db...
downloading community.db...
:: Starting full system upgrade...
resolving dependencies...
looking for conflicting packages...
warning: dependency cycle detected:
warning: rubygems will be installed before its ruby dependency
Packages (36) binutils-2.30-5 gc-7.6.6-1 gdbm-1.16-1 guile-2.2.4-1 libatomic_ops-7.6.4-1 libmpc-1.1.0-1 libnsl-1.2.0-1 libsasl-2.1.26-13 libsystemd-239.0-2 libtool-2.4.6+40+g6ca5e224-7 libyaml-0.1.7-1 mpfr-4.0.1-1 perl-5.26.2-2 python2-2.7.15-2 python2-appdirs-1.4.3-1 python2-meld3-1.0.2-1 python2-packaging-17.1-1 python2-pyparsing-2.2.0-1 python2-setuptools-1:39.2.0-2 python2-six-1.11.0-1 rubygems-2.7.7-1 sysfsutils-2.1.0-10 systemd-239.0-2 tar-1.30-1 texinfo-6.5-1 gawk-4.2.1-1 gcc-8.1.1+20180531-1 grep-3.1-1 iproute2-4.17.0-1 iputils-20161105.1f2bb12-2 make-4.2.1-2 pkgconf-1.5.1-1 postgresql-libs-10.4-3 procps-ng-3.3.15-1 ruby-2.5.1-2 supervisor-3.3.4-1
Total Download Size: 83.64 MiB
Total Installed Size: 401.21 MiB
Net Upgrade Size: 327.94 MiB
:: Proceed with installation? [Y/n]
:: Retrieving packages...
downloading pkgconf-1.5.1-1-x86_64.pkg.tar.xz...
downloading libsystemd-239.0-2-x86_64.pkg.tar.xz...
downloading procps-ng-3.3.15-1-x86_64.pkg.tar.xz...
downloading iproute2-4.17.0-1-x86_64.pkg.tar.xz...
downloading sysfsutils-2.1.0-10-x86_64.pkg.tar.xz...
downloading gdbm-1.16-1-x86_64.pkg.tar.xz...
downloading perl-5.26.2-2-x86_64.pkg.tar.xz...
downloading iputils-20161105.1f2bb12-2-x86_64.pkg.tar.xz...
downloading libsasl-2.1.26-13-x86_64.pkg.tar.xz...
downloading libnsl-1.2.0-1-x86_64.pkg.tar.xz...
downloading tar-1.30-1-x86_64.pkg.tar.xz...
downloading libtool-2.4.6+40+g6ca5e224-7-x86_64.pkg.tar.xz...
downloading texinfo-6.5-1-x86_64.pkg.tar.xz...
downloading make-4.2.1-2-x86_64.pkg.tar.xz...
downloading binutils-2.30-5-x86_64.pkg.tar.xz...
downloading mpfr-4.0.1-1-x86_64.pkg.tar.xz...
downloading libmpc-1.1.0-1-x86_64.pkg.tar.xz...
downloading gcc-8.1.1+20180531-1-x86_64.pkg.tar.xz...
downloading grep-3.1-1-x86_64.pkg.tar.xz...
downloading gawk-4.2.1-1-x86_64.pkg.tar.xz...
downloading systemd-239.0-2-x86_64.pkg.tar.xz...
downloading python2-2.7.15-2-x86_64.pkg.tar.xz...
downloading python2-appdirs-1.4.3-1-any.pkg.tar.xz...
downloading python2-pyparsing-2.2.0-1-any.pkg.tar.xz...
downloading python2-six-1.11.0-1-any.pkg.tar.xz...
downloading python2-packaging-17.1-1-any.pkg.tar.xz...
downloading python2-setuptools-1:39.2.0-2-any.pkg.tar.xz...
downloading libyaml-0.1.7-1-x86_64.pkg.tar.xz...
downloading rubygems-2.7.7-1-any.pkg.tar.xz...
downloading ruby-2.5.1-2-x86_64.pkg.tar.xz...
downloading libatomic_ops-7.6.4-1-x86_64.pkg.tar.xz...
downloading gc-7.6.6-1-x86_64.pkg.tar.xz...
downloading guile-2.2.4-1-x86_64.pkg.tar.xz...
downloading postgresql-libs-10.4-3-x86_64.pkg.tar.xz...
downloading python2-meld3-1.0.2-1-any.pkg.tar.xz...
downloading supervisor-3.3.4-1-any.pkg.tar.xz...
checking keyring...
checking package integrity...
loading package files...
checking for file conflicts...
checking available disk space...
:: Processing package changes...
installing pkgconf...
upgrading libsystemd...
installing procps-ng...
installing iproute2...
Optional dependencies for iproute2
linux-atm: ATM support
installing sysfsutils...
upgrading gdbm...
upgrading perl...
installing iputils...
Optional dependencies for iputils
xinetd: for tftpd
upgrading libsasl...
installing libnsl...
installing python2...
Optional dependencies for python2
tk: for IDLE
python2-setuptools [pending]
python2-pip
installing python2-meld3...
installing python2-appdirs...
installing python2-pyparsing...
installing python2-six...
installing python2-packaging...
installing python2-setuptools...
installing supervisor...
installing libyaml...
installing rubygems...
installing ruby...
Optional dependencies for ruby
ruby-docs: Ruby documentation
tk: for Ruby/TK
installing tar...
installing libtool...
installing texinfo...
installing libatomic_ops...
installing gc...
installing guile...
installing make...
installing binutils...
installing mpfr...
installing libmpc...
installing gcc...
Optional dependencies for gcc
lib32-gcc-libs: for generating code for 32-bit ABI
installing grep...
installing postgresql-libs...
installing gawk...
upgrading systemd...
:: Running post-transaction hooks...
( 1/11) Warn about old perl modules
perl: warning: Setting locale failed.
perl: warning: Please check that your locale settings:
LANGUAGE = (unset),
LC_ALL = (unset),
LANG = "ru_RU.UTF-8"
are supported and installed on your system.
perl: warning: Falling back to the standard locale ("C").
( 2/11) Cleaning up package cache...
( 3/11) Updating journal message catalog...
( 4/11) Reloading system manager configuration...
Skipped: Current root is not booted.
( 5/11) Updating udev hardware database...
( 6/11) Applying kernel sysctl settings...
Skipped: Current root is not booted.
( 7/11) Creating system user accounts...
( 8/11) Creating temporary files...
[/usr/lib/tmpfiles.d/journal-nocow.conf:26] Failed to resolve specifier: uninitialized /etc detected, skipping
All rules containing unresolvable specifiers will be skipped.
( 9/11) Reloading device manager configuration...
Skipped: Device manager is not running.
(10/11) Arming ConditionNeedsUpdate...
(11/11) Updating the info directory file...
Successfully installed bundler-1.16.2
1 gem installed
Cache directory: /var/cache/pacman/pkg/
:: Do you want to remove ALL files from cache? [y/N]
Database directory: /var/lib/pacman/
:: Do you want to remove unused repositories? [Y/n]
removing unused sync repositories...
Removing intermediate container b18a1c4585ee
---> f6937487ce2c
Step 7/12 : COPY ./ /srv/
---> 6bb96498a4b5
Step 8/12 : RUN cd /srv && rm -rf vendor && bundle config build.nokogiri --use-system-libraries --with-xml2-include=/usr/include/libxml2 --with-xslt-include=/usr/include/libxslt --with-exslt-include=/usr/include/libexslt --with-xml2-lib=/usr/lib/libxml2.so && bundle install --path=vendor --jobs=1 --binstubs
---> Running in 187980e45717
Don't run Bundler as root. Bundler can ask for sudo if it is needed, and
installing your bundle as root will break this application for all non-root
users on this machine.
Fetching gem metadata from https://rubygems.org/.........
Fetching rake 12.3.1
Installing rake 12.3.1
Fetching public_suffix 3.0.2
Installing public_suffix 3.0.2
Fetching addressable 2.5.2
Installing addressable 2.5.2
Fetching amq-protocol 2.3.0
Installing amq-protocol 2.3.0
Fetching builder 3.2.3
Installing builder 3.2.3
Using bundler 1.16.2
Fetching bunny 2.11.0
Installing bunny 2.11.0
Fetching byebug 10.0.2
Installing byebug 10.0.2 with native extensions
Fetching coderay 1.1.2
Installing coderay 1.1.2
Fetching colored 1.2
Installing colored 1.2
Fetching command_line_reporter 4.0.0
Installing command_line_reporter 4.0.0
Fetching concurrent-ruby 1.0.5
Installing concurrent-ruby 1.0.5
Fetching connection_pool 2.2.2
Installing connection_pool 2.2.2
Fetching safe_yaml 1.0.4
Installing safe_yaml 1.0.4
Fetching crack 0.4.3
Installing crack 0.4.3
Fetching daemons 1.2.6
Installing daemons 1.2.6
Fetching diff-lcs 1.3
Installing diff-lcs 1.3
Fetching dotenv 2.5.0
Installing dotenv 2.5.0
Fetching eventmachine 1.2.7
Installing eventmachine 1.2.7 with native extensions
Fetching multipart-post 2.0.0
Installing multipart-post 2.0.0
Fetching faraday 0.15.2
Installing faraday 0.15.2
Fetching ffi 1.9.25
Installing ffi 1.9.25 with native extensions
Fetching gyoku 1.3.1
Installing gyoku 1.3.1
Fetching hashdiff 0.3.7
Installing hashdiff 0.3.7
Fetching hashie 3.5.7
Installing hashie 3.5.7
Fetching jwt 2.1.0
Installing jwt 2.1.0
Fetching memory_profiler 0.9.10
Installing memory_profiler 0.9.10
Fetching method_source 0.9.0
Installing method_source 0.9.0
Fetching mini_portile2 2.3.0
Installing mini_portile2 2.3.0
Fetching mustermann 1.0.2
Installing mustermann 1.0.2
Fetching nokogiri 1.8.4
Installing nokogiri 1.8.4 with native extensions
Gem::Ext::BuildError: ERROR: Failed to build gem native extension.
current directory: /srv/vendor/ruby/2.5.0/gems/nokogiri-1.8.4/ext/nokogiri
/usr/bin/ruby -r ./siteconf20180709-1-199bolx.rb extconf.rb
--use-system-libraries --with-xml2-include=/usr/include/libxml2
--with-xslt-include=/usr/include/libxslt
--with-exslt-include=/usr/include/libexslt --with-xml2-lib=/usr/lib/libxml2.so
checking if the C compiler accepts ... yes
Building nokogiri using system libraries.
ERROR: cannot discover where libxml2 is located on your system. please make sure
`pkg-config` is installed.
*** extconf.rb failed ***
Could not create Makefile due to some reason, probably lack of necessary
libraries and/or headers. Check the mkmf.log file for more details. You may
need configuration options.
Provided configuration options:
--with-opt-dir
--without-opt-dir
--with-opt-include
--without-opt-include=${opt-dir}/include
--with-opt-lib
--without-opt-lib=${opt-dir}/lib
--with-make-prog
--without-make-prog
--srcdir=.
--curdir
--ruby=/usr/bin/$(RUBY_BASE_NAME)
--help
--clean
--use-system-libraries
--with-zlib-dir
--without-zlib-dir
--with-zlib-include
--without-zlib-include=${zlib-dir}/include
--with-zlib-lib
--without-zlib-lib=${zlib-dir}/lib
--with-xml2-dir
--without-xml2-dir
--with-xml2-include=${xml2-dir}/include
--with-xml2-lib=${xml2-dir}/lib
--with-xslt-dir
--without-xslt-dir
--with-xslt-include=${xslt-dir}/include
--with-xslt-lib
--without-xslt-lib=${xslt-dir}/lib
--with-exslt-dir
--without-exslt-dir
--with-exslt-include=${exslt-dir}/include
--with-exslt-lib
--without-exslt-lib=${exslt-dir}/lib
To see why this extension failed to compile, please check the mkmf.log which can
be found here:
/srv/vendor/ruby/2.5.0/extensions/x86_64-linux/2.5.0/nokogiri-1.8.4/mkmf.log
extconf failed, exit code 1
Gem files will remain installed in /srv/vendor/ruby/2.5.0/gems/nokogiri-1.8.4
for inspection.
Results logged to
/srv/vendor/ruby/2.5.0/extensions/x86_64-linux/2.5.0/nokogiri-1.8.4/gem_make.out
An error occurred while installing nokogiri (1.8.4), and Bundler cannot
continue.
Make sure that `gem install nokogiri -v '1.8.4' --source
'https://rubygems.org/'` succeeds before bundling.
In Gemfile:
nokogiri
The command '/bin/sh -c cd /srv && rm -rf vendor && bundle config build.nokogiri --use-system-libraries --with-xml2-include=/usr/include/libxml2 --with-xslt-include=/usr/include/libxslt --with-exslt-include=/usr/include/libexslt --with-xml2-lib=/usr/lib/libxml2.so && bundle install --path=vendor --jobs=1 --binstubs' returned a non-zero code: 5

Answer 1

Here is a Dockerfile based on your example which have installed nokogiri from Gemfile.
Dockerfile:
FROM archlinux/base
ENV HOME="/srv"
ENV LANG=ru_RU.UTF-8
ENV APP_ENV=production
ENV LANG=ru_RU.UTF-8
ENV PATH=$HOME/sbin:$HOME/bin:$HOME/.gem/ruby/2.5.0/bin:/usr/lib/ruby/gems/2.5.0/bin:/usr/local/sbin:/usr/local/bin:/usr/bin:/usr/bin/site_perl:/usr/bin/vendor_perl:/usr/bin/core_perl
RUN pacman -Syu --noconfirm libxml2 libxslt pkg-config pkgconf procps-ng iproute2 iputils supervisor ruby make gcc grep postgresql-libs awk && \
gem install bundler --no-ri --no-rdoc --no-document --no-user-install && \
pacman -Scc --noconfirm
COPY Gemfile /srv/
RUN cd /srv && rm -rf vendor && bundle config build.nokogiri --use-system-libraries --with-xml2-include=/usr/include/libxml2 --with-xslt-include=/usr/include/libxslt && bundle install --path=vendor --jobs=4 --binstubs
Gemfile
source 'https://rubygems.org'
gem 'nokogiri'
Feel free to use it.

Answer 2

So, I decided to start from scratch and removed all options from bundler rm -f $HOME/.bundle/config. Then I started to build docker over and recevied the next message from nokogiri team, where the hint was found: Failed to complete patch task; patch(1) or git(1) is required.
So adding patch and git to pacman options was fired the goal.
...
Gem::Ext::BuildError: ERROR: Failed to build gem native extension.
current directory: /srv/vendor/ruby/2.5.0/gems/nokogiri-1.8.4/ext/nokogiri
/usr/bin/ruby -r ./siteconf20180709-1-dxdiny.rb extconf.rb
checking if the C compiler accepts ... yes
Building nokogiri using packaged libraries.
Using mini_portile version 2.3.0
checking for gzdopen() in -lz... yes
checking for iconv... yes
************************************************************************
IMPORTANT NOTICE:
Building Nokogiri with a packaged version of libxml2-2.9.8
with the following patches applied:
- 0001-Revert-Do-not-URI-escape-in-server-side-includes.patch
Team Nokogiri will keep on doing their best to provide security
updates in a timely manner, but if this is a concern for you and want
to use the system library instead; abort this installation process and
reinstall nokogiri as follows:
gem install nokogiri -- --use-system-libraries
[--with-xml2-config=/path/to/xml2-config]
[--with-xslt-config=/path/to/xslt-config]
If you are using Bundler, tell it to use the option:
bundle config build.nokogiri --use-system-libraries
bundle install
Note, however, that nokogiri is not fully compatible with arbitrary
versions of libxml2 provided by OS/package vendors.
************************************************************************
Extracting libxml2-2.9.8.tar.gz into
tmp/x86_64-pc-linux-gnu/ports/libxml2/2.9.8... OK
*** extconf.rb failed ***
Could not create Makefile due to some reason, probably lack of necessary
libraries and/or headers. Check the mkmf.log file for more details. You may
need configuration options.
Provided configuration options:
--with-opt-dir
--without-opt-dir
--with-opt-include
--without-opt-include=${opt-dir}/include
--with-opt-lib
--without-opt-lib=${opt-dir}/lib
--with-make-prog
--without-make-prog
--srcdir=.
--curdir
--ruby=/usr/bin/$(RUBY_BASE_NAME)
--help
--clean
--use-system-libraries
--enable-static
--disable-static
--with-zlib-dir
--without-zlib-dir
--with-zlib-include
--without-zlib-include=${zlib-dir}/include
--with-zlib-lib
--without-zlib-lib=${zlib-dir}/lib
--enable-cross-build
--disable-cross-build
/srv/vendor/ruby/2.5.0/gems/mini_portile2-2.3.0/lib/mini_portile2/mini_portile.rb:84:in
`apply_patch': Failed to complete patch task; patch(1) or git(1) is required.
(RuntimeError)
from
/srv/vendor/ruby/2.5.0/gems/mini_portile2-2.3.0/lib/mini_portile2/mini_portile.rb:92:in
`block in patch'
from
/srv/vendor/ruby/2.5.0/gems/mini_portile2-2.3.0/lib/mini_portile2/mini_portile.rb:90:in
`each'
from
/srv/vendor/ruby/2.5.0/gems/mini_portile2-2.3.0/lib/mini_portile2/mini_portile.rb:90:in
`patch'
from
/srv/vendor/ruby/2.5.0/gems/mini_portile2-2.3.0/lib/mini_portile2/mini_portile.rb:152:in
`cook'
from extconf.rb:365:in `block (2 levels) in process_recipe'
from extconf.rb:257:in `block in chdir_for_build'
from extconf.rb:256:in `chdir'
from extconf.rb:256:in `chdir_for_build'
from extconf.rb:364:in `block in process_recipe'
from extconf.rb:262:in `tap'
from extconf.rb:262:in `process_recipe'
from extconf.rb:551:in `<main>'
To see why this extension failed to compile, please check the mkmf.log which can
be found here:
/srv/vendor/ruby/2.5.0/extensions/x86_64-linux/2.5.0/nokogiri-1.8.4/mkmf.log
extconf failed, exit code 1
Gem files will remain installed in /srv/vendor/ruby/2.5.0/gems/nokogiri-1.8.4
for inspection.
Results logged to
/srv/vendor/ruby/2.5.0/extensions/x86_64-linux/2.5.0/nokogiri-1.8.4/gem_make.out
An error occurred while installing nokogiri (1.8.4), and Bundler cannot
continue.
Make sure that `gem install nokogiri -v '1.8.4' --source
'https://rubygems.org/'` succeeds before bundling.