How to run and interact with a script from within an RPM? - bash

I'm building an RPM which needs to run a bash script as root.
The %install stanza of the spec file is:
%install
cp %{SOURCE1} %{SOURCE2} %{_tmppath}/%{name}-%{version}-%{release}
cd %{_tmppath}/%{name}-%{version}-%{release}
chmod u+x %{installscript}
sudo ./%{installscript}
Where %{installscript} is the script that runs as root with sudo.
rpmbuild executes %{installscript} and creates the RPM (without problems).
However, when I install the RPM:
$ sudo rpm -Uvh $rpmpath
Preparing... ########################################### [100%]
1:tty-cap ########################################### [100%]
$
The %{installscript} script is not executed.
I tried to change the spec file by moving the script invocation to a %post stanza:
%install
cp %{SOURCE1} %{SOURCE2} %{_tmppath}/%{name}-%{version}-%{release}
cd %{_tmppath}/%{name}-%{version}-%{release}
%post
chmod u+x %{installscript}
sudo ./%{installscript}
But the %post doesn't seem to do anything.
How can I pack an RPM that will execute a script when installed?
Edit 1:
After reviewing the helpful comments below, here's a spec file with a %post stanza that actually gets executed during the RPM installation.
However, the script %{installscript} does not interact with the user (as it does when run from the shell), but seems to accept all its defaults without user interaction.
What should I change so that the script will interact with the rpm command user?
$ cat ~/RPMBUILD/SPECS/demo.spec
#
# %_topdir and %_tmppath are defined in ~/.rpmmacros
%define name tty-cap
%define version 5.2
%define release 1
%define buildroot %{_tmppath}/%{name}-%{version}-%{release}
%define tarversion tty-5.2.0-00-70270
%define tarfile %{tarversion}.tar
%define installscript tty.install.sh
Name: %{name}
Version: %{version}
Release: %{release}
BuildArch: noarch
Summary: Bla
License: Proprietary
Source1: %{installscript}
Source2: tty-5.2.0-00-70270.tar
Prefix: /opt/Intellinx/TTYCapture
BuildRoot: %{_builddir}/%{name}-root
%description
Demonstration RPM
%prep
%build
%install
cp %{SOURCE1} %{SOURCE2} %{_tmppath}/%{name}-%{version}-%{release}
cd %{_tmppath}/%{name}-%{version}-%{release}
%clean
[ ${RPM_BUILD_ROOT} != "/" ] && rm -rf ${RPM_BUILD_ROOT}
%post
echo ">>> Inside post <<<"
chmod u+x %{installscript}
./%{installscript}
%files
%define tmp /
%{tmp}/%{tarfile}
%{tmp}/%{installscript}
$ rpmbuild -v -bb ~/RPMBUILD/SPECS/demo.spec
Executing(%prep): /bin/sh -e /home/ronbarak/RPMBUILD/tmp/rpm-tmp.oEOM10
+ umask 022
+ cd /home/ronbarak/RPMBUILD/BUILD
+ LANG=C
+ export LANG
+ unset DISPLAY
+ exit 0
Executing(%build): /bin/sh -e /home/ronbarak/RPMBUILD/tmp/rpm-tmp.qQFuTA
+ umask 022
+ cd /home/ronbarak/RPMBUILD/BUILD
+ LANG=C
+ export LANG
+ unset DISPLAY
+ exit 0
Executing(%install): /bin/sh -e /home/ronbarak/RPMBUILD/tmp/rpm-tmp.8rTMLa
+ umask 022
+ cd /home/ronbarak/RPMBUILD/BUILD
+ '[' /home/ronbarak/RPMBUILD/tmp/tty-cap-5.2-1 '!=' / ']'
+ rm -rf /home/ronbarak/RPMBUILD/tmp/tty-cap-5.2-1
++ dirname /home/ronbarak/RPMBUILD/tmp/tty-cap-5.2-1
+ mkdir -p /home/ronbarak/RPMBUILD/tmp
+ mkdir /home/ronbarak/RPMBUILD/tmp/tty-cap-5.2-1
+ LANG=C
+ export LANG
+ unset DISPLAY
+ cp /home/ronbarak/RPMBUILD/SOURCES/tty.install.sh /home/ronbarak/RPMBUILD/SOURCES/tty-5.2.0-00-70270.tar /home/ronbarak/RPMBUILD/tmp/tty-cap-5.2-1
+ cd /home/ronbarak/RPMBUILD/tmp/tty-cap-5.2-1
+ /usr/lib/rpm/check-buildroot
+ /usr/lib/rpm/redhat/brp-compress
+ /usr/lib/rpm/redhat/brp-strip /usr/bin/strip
+ /usr/lib/rpm/redhat/brp-strip-static-archive /usr/bin/strip
+ /usr/lib/rpm/redhat/brp-strip-comment-note /usr/bin/strip /usr/bin/objdump
+ /usr/lib/rpm/brp-python-bytecompile
+ /usr/lib/rpm/redhat/brp-python-hardlink
+ /usr/lib/rpm/redhat/brp-java-repack-jars
Processing files: tty-cap-5.2-1.noarch
Requires(interp): /bin/sh
Requires(rpmlib): rpmlib(CompressedFileNames) <= 3.0.4-1 rpmlib(FileDigests) <= 4.6.0-1 rpmlib(PayloadFilesHavePrefix) <= 4.0-1
Requires(post): /bin/sh
Checking for unpackaged file(s): /usr/lib/rpm/check-files /home/ronbarak/RPMBUILD/tmp/tty-cap-5.2-1
Wrote: /home/ronbarak/RPMBUILD/RPMS/noarch/tty-cap-5.2-1.noarch.rpm
Executing(%clean): /bin/sh -e /home/ronbarak/RPMBUILD/tmp/rpm-tmp.Yag9bm
+ umask 022
+ cd /home/ronbarak/RPMBUILD/BUILD
+ '[' /home/ronbarak/RPMBUILD/tmp/tty-cap-5.2-1 '!=' / ']'
+ rm -rf /home/ronbarak/RPMBUILD/tmp/tty-cap-5.2-1
+ exit 0
$ sudo rpm -Uvh /home/ronbarak/RPMBUILD/RPMS/noarch/tty-cap-5.2-1.noarch.rpm
Preparing... ########################################### [100%]
1:tty-cap ########################################### [100%]
>>> Inside post <<<
###### Starting tty capturing installation ######
Specify installation directory full path [/opt/Intellinx/TTYCapture]:
/opt/Intellinx/TTYCapture already exists. continue installation? (y/n) [y]
Extracting archive tty-5*.* ...
Please specify TTY sensor owner user [ronbarak]:
TTY sensor will be accessible by user 'ronbarak' in group 'ronbarak'
Specify sensor working directory full path [/opt/Intellinx/TTYCapture/work]:
Specify sensor listening port [1024-65000] [8888]:
Do you want to register the sensor service? (y/n) [n]
Server gxttySensorService is not register as a daemon
Executing default user shell (y/n) [y]
TTY capturing has been installed successfully.

You are mixing several concepts here. So let make step back.
In %prep section you should unpack your %{SOURCE0} and apply patches if any. This usually do
%setup -q
However if you want you can extract it manually. For more info about this macro see http://www.rpm.org/max-rpm/s1-rpm-inside-macros.html
In %build section you usually compile source into binaries. Likely empty if you use interpreted language or your tar contains already compiled binaries.
In %install section you should copy the files into %{buildroot} and create there structure which will land in package. E.g. %{buildroot}/etc/yourconfig, %{buildroot}/usr/bin/yourcommand etc. You can run there any script you want, but keep in mind that it is run only in build time. I.e. only on your machine (or build system). This is intended for creating files which are automatically generated (e.g. documentation of libraries from source code).
Then you have section %post which is run on user machine after the package was installed. And all files are installed in final path. Not in buildroot. At the beginning you are changed to / so you need to specify full path on that user machine.
So in your case it should be probably look like:
%install
mkdir -p %{buildroot}%{_bindir}
cp -a %{installscript} %{buildroot}%{_bindir}/
chmod a+x %{buildroot}%{_bindir}/%{installscript}
%files
%{_bindir}/%{installscript}
%post
%{_bindir}%{installscript}
Sever notes:
%post section is executed under root, so sudo is not needed.
Running interactive script is strongly discouraged. RPM was designed as non-interactive and every utility around assume no interaction during package installation (e.g. PackageKit, Spacewalk etc.). So sooner then later you will get some compains. It is much safer to say user to run some command after installation manually (or automate it using Ansible or Puppet).

Related

Docker container unable to ignore the EntryPoint bash script failure

Bash script:
clonePath=/data/config/
git branch -r | fgrep -v 'origin/HEAD' | sed 's| origin/|git checkout |' > checkoutAllBranches.sh
chmod +x checkoutAllBranches.sh
echo "Fetch branch: `cat checkoutAllBranches.sh`"
./checkoutAllBranches.sh
git checkout master
git remote rm origin
rm checkoutAllBranches.sh
for config_dir in `ls -a`; do
cp -r $config_dir $clonePath/;
done
echo "API Config update complete..."
Dockerfile which issues this script execution
ENTRYPOINT ["sh","config-update-force.sh","|| true"]
The error below causes the container startup failure despite setting the command status to 0 manually using || true
ERROR:
Error:
cp: cannot create regular file '/data/./.git/objects/pack/pack-27a9d...fb5e368e4cf.pack': Permission denied
cp: cannot create regular file '/data/./.git/objects/pack/pack-27a9d...fbae25e368e4cf.idx': Permission denied
I am looking for 2 options here:
Change these file permissions and then store them in the remote with rwx permissions
Do something to the docker file to ignore this script failure error and start the container.
DOCKERFILE:
FROM docker.hub.com/java11-temurin:latest
USER root
RUN apt-get update
RUN apt-get -y upgrade
RUN apt-get install -y rsync telnet vim wget git
RUN mkdir -p /opt/config/clone/data
RUN chown -R 1001:1001 /opt/config
USER 1001
ADD build/libs/my-api-config-server.jar .
ADD config-update-force.sh .
USER root
RUN chmod +x config-update-force.sh
USER 1001
EXPOSE 8080
CMD java $BASE_JAVA_OPTS $JAVA_OPTS -jar my-api-config-server.jar
ENTRYPOINT ["sh","config-update-force.sh","|| true"]
BASH SCRIPT:
#!/bin/bash
set +e
set +x
clonePath=/opt/clone/data/data
#source Optumfile.properties
echo "properties loaded: example ${git_host}"
if [ -d my-api-config ]; then
rm -rf my-api-config;
echo "existing my-api-config dir deleted..."
fi
git_url=https://github.com/my-api-config-server
git clone https://github.com/my-api-config-server
cd my-api-config-server
git branch -r | fgrep -v 'origin/HEAD' | sed 's| origin/|git checkout |' > checkoutAllBranches.sh
chmod +x checkoutAllBranches.sh
echo "Fetch branch: `cat checkoutAllBranches.sh`"
./checkoutAllBranches.sh
git checkout master
git remote rm origin
rm checkoutAllBranches.sh
for config_dir in `ls -a`; do
cp -r $config_dir $clonePath/;
done
echo "My API Config update complete..."
When you do in the script...
chmod +x checkoutAllBranches.sh
...than why not before cp
chmod -R +rwx ${clonePath}
...or if the stderr message 'wont impact anything'...
cp -r $config_dir $clonePath/ 2>/dev/null;
...even cp dont copy -verbosly.
?
When your Dockerfile declares an ENTRYPOINT, that command is the only thing the container does. If it also declares a CMD, the CMD is passed as additional arguments to the ENTRYPOINT; it is not run on its own unless the ENTRYPOINT makes sure to execute it.
Shell errors are not normally fatal, and especially if you explicitly set +e, even if a shell command fails the shell script will keep running. You see this in your output where you get multiple cp errors; the first error does not terminate the script.
You need to do two things here. The first is to set the ENTRYPOINT to actually run the CMD; the simplest and most common way to do this is to end the script with
exec "$#"
The second is to remove the || true from the Dockerfile. As you have it written out currently, this is passed as the first argument to the entrypoint wrapper – it is not run through a shell and it is not interpreted as a "or" operator. If your script begins with a "shebang" line and is marked executable (both of these are correct in the question) the you do not explicitly need the sh interpreter.
# must be a JSON array; no additional "|| true" argument; no sh -c wrapper
ENTRYPOINT ["./config-update-force.sh"]
# any valid CMD will work with `exec "$#"
CMD java $BASE_JAVA_OPTS $JAVA_OPTS -jar my-api-config-server.jar

rpmbuild exiting with error: File not found

I'm trying to build my first RPM package and struggling with the rpmbuild process.
This is my environment I have to deal with:
cat /home/rpmadm/.rpmmacros
%_topdir /export/standardbuild
%_tmppath /export/standardbuild/tmp
%_sourcedir /export/standardbuild/SOURCES
%_rpmdir /export/standardbuild/RPMS
%_srcrpmdir /export/standardbuild/SRPMS
%_builddir /export/standardbuild/COMP_BUILDS/%{name}
My Makefile looks like this:
packagename = metricconf
%_sourcedir = /export/standardbuild/SOURCES
# version number of package: make sure you also update the version
# number in the spec file
version = 1.0
all:
#echo "usage: make dist | package"
install:
if [ ! -d $(DESTDIR)/tmp/$(packagename) ]; \
then \
mkdir -p $(DESTDIR)/tmp/$(packagename); \
fi
cp mb_system_betrieb.yml $(DESTDIR)/tmp/$(packagename)/mb_system_betrieb.yml
cp metricbeat $(DESTDIR)/tmp/$(packagename)/metricbeat
cp metricbeat.yml $(DESTDIR)/tmp/$(packagename)/metricbeat.yml
dist:
tar cvf $(packagename)-$(version).tar ../$(packagename)-$(version)
gzip $(packagename)-$(version).tar
cp $(packagename)-$(version).tar.gz $(%_sourcedir) && rm $(packagename)-$(version).tar.gz
package: rpm
rpm:
rpmbuild -ba $(packagename).spec
rm -rf $(packagename)-$(version).tar
And this is my .spec file:
Summary: configures Metricbeat for AIX
Name: metricconf
Version: 1.0
Release: 0
Vendor: Comp
Source: metricconf-1.0.tar.gz
BuildRoot: %{_builddir}/%{name}-{version}
Requires: metricbeat
%description
metricconf prepares AIX for metricbeat
%prep
%setup -c
%build
%install
%files
%defattr(644,root,system)
/tmp/metricbeat.yml
/tmp/mb_system_betrieb.yml
/tmp/metricbeat
make dist works well and the .tar.gz file will be created correctly.
But the make rpm command fails every time with following error:
make rpm
rpmbuild -ba metricconf.spec
Executing(%prep): /bin/sh -e /export/standardbuild/tmp/rpm-tmp.cIlqQa
+ umask 022
+ cd /export/standardbuild/COMP_BUILDS/metricconf
+ cd /export/standardbuild/COMP_BUILDS/metricconf
+ rm -rf metricconf-1.0
+ /usr/bin/mkdir -p metricconf-1.0
+ cd metricconf-1.0
+ /bin/gzip -dc /export/standardbuild/SOURCES/metricconf-1.0.tar.gz
+ /bin/tar -xvvof -
x ../COOPmetricbeat-1.0
x ../COOPmetricbeat-1.0/metricconf.spec, 2289 bytes, 5 tape blocks
x ../COOPmetricbeat-1.0/ChangeLog, 135 bytes, 1 tape blocks
x ../COOPmetricbeat-1.0/Makefile, 885 bytes, 2 tape blocks
x ../COOPmetricbeat-1.0/README, 74 bytes, 1 tape blocks
x ../COOPmetricbeat-1.0/mb_system_betrieb.yml, 1161 bytes, 3 tape blocks
x ../COOPmetricbeat-1.0/metricbeat, 1552 bytes, 4 tape blocks
x ../COOPmetricbeat-1.0/metricbeat.yml, 1240 bytes, 3 tape blocks
+ STATUS=0
+ [ 0 -ne 0 ]
+ /bin/chmod -Rf a+rX,u+w,g-w,o-w .
+ exit 0
Executing(%build): /bin/sh -e /export/standardbuild/tmp/rpm-tmp.dalqQb
+ umask 022
+ cd /export/standardbuild/COMP_BUILDS/metricconf
+ cd COOPmetricbeat-1.0
+ exit 0
Executing(%install): /bin/sh -e /export/standardbuild/tmp/rpm-tmp.delqQc
+ umask 022
+ cd /export/standardbuild/COMP_BUILDS/metricconf
+ cd metricconf-1.0
+ exit 0
Processing files: metricconf-1.0-0.ppc
error: File not found: /export/standardbuild/tmp/metricconf-1.0-0.ppc/tmp/metricbeat.yml
error: File not found: /export/standardbuild/tmp/metricconf-1.0-0.ppc/tmp/mb_system_betrieb.yml
error: File not found: /export/standardbuild/tmp/metricconf-1.0-0.ppc/tmp/metricbeat
RPM build errors:
File not found: /export/standardbuild/tmp/metricconf-1.0-0.ppc/tmp/metricbeat.yml
File not found: /export/standardbuild/tmp/metricconf-1.0-0.ppc/tmp/mb_system_betrieb.yml
File not found: /export/standardbuild/tmp/metricconf-1.0-0.ppc/tmp/metricbeat
make: 1254-004 The error code from the last command is 1.## Heading ##
Please, can anyone explain to me why rpmbuild is looking in "/export/standardbuild/tmp/metricconf-1.0-0.ppc" for the files and not in "/export/standardbuild/COMP_BUILDS/metricconf/metricconf-1.0" ???
I read a lot about building rpms especially at Fedora's doc pages, but it doesn't matter what I change in the Makefile or .spec file, the error remains.
Your %build and %install phases are both empty, so it's not doing anything. All you have is %prep which is untarring it into a temporary staging area.
Look around for examples, e.g. if you are using autoconf you probably want %configure.
ETA: It also looks like you are trying to put your final files into /tmp on the target machine, which won't work because they will disappear after a reboot. %files should list the files to be installed on the target without any leading temporary paths.

Script is running perfectly on co-workers devices but gives me 'Invalid cross-device link'

My script is running perfectly on co-workers devices (MacOSX with Docker Desktop same as me), but gives me every time the same error and it does not move or only half, the libraries in the deps directory:
OSError: [Errno 18] Invalid cross-device link: '/tmp/pip-target-dzwe_2kc/lib/python/numpy' ->
'/foo/python/numpy'
My script :
#!/bin/bash
export PKG_DIR='python'
export SIDE_DEPS_DIR='deps'
rm -rf ${PKG_DIR} && mkdir -p ${PKG_DIR}
rm -rf ${SIDE_DEPS_DIR} && mkdir -p ${SIDE_DEPS_DIR}
docker run --rm -v $(pwd):/foo -w /foo lambci/lambda:build-python3.8 \
pip3 install -r requirements.txt -t ${PKG_DIR}
# move stuff to deps
find /${PKG_DIR} -maxdepth 1 -type d \
\( -name "pandas*" -o -name "numpy*" -o -name "numpy.libs*" -o -name "scipy*" -o -name "scipy.libs*" \) -exec mv '{}' ${SIDE_DEPS_DIR} \;
# zip side dependencies
zip -r ge_deps.zip deps
# zip layer
zip -r layers-python38-great-expectations.zip python
It's a script which uses a public lambda docker image to create a lambda layer (basically a zip that contains libraries) and which removes unwanted libraries to put them in another folder deps.
The above code will use the public Docker image lambci / lambda and will install in the empty python directory, libraries which come from a python package which is called 'great-expectations' and which helps to test pipelines of data (which is specified in requirements.txt and is great-expectations==0.12.7)
I have been stuck with this problem for a while and have not found a solution.
Had this exact problem just now.
/tmp and /foo are different devices - /tmp is within the docker OS and /foo is mapped to your local OS.
pip seems to be using shutil.rename() to move the built package from tmp to the final output location (/foo). This fails because they are different devices. Ideally pip would use shutil.move() instead, which will deal with a cross-device move.
As a workaround, you can change the temp folder used by PIP by setting TMPDIR before invoking the pip command. i.e. export TMPDIR=/foo/tmp before calling pip in the docker image. So, the whole command might be something like
docker run --rm -v $(pwd):/foo -w /foo lambci/lambda:build-python3.8 \
/bin/bash -c "export TMPDIR=/foo/tmp && pip3 install -r requirements.txt -t ${PKG_DIR}"
(multiple commands soln taken from https://www.edureka.co/community/10736/how-to-run-multiple-commands-in-docker-at-once - open to better suggestions!)
This will likely be slower because it's using the local OS for temp files, but it avoids the attempted 'rename' across devices from the temp folder to the final output folder.

mkdir always creates a file instead a directory

First I want to say that I don't really know what I should look for, here in Stack Overflow and what could be a good query for my problem.
In simple words I want to create a new directory and than do some file operations in it. But with the script that I have crafted I got always a file instead of a directory. It seems to be absolutely regardless how I stick the code together there is always the same result. I hope tat masses can help me with their knowledge.
Here is the script:
#!/bin/bash
DLURL=http://drubuntu.googlecode.com/git'
d7dir=/var/www/d7/'
dfsettings=/var/www/d7/sites/default/default.settings.php
settings=/var/www/d7/sites/default/settings.php
#settiing up drush
drush -y dl drush --destination=/usr/share;
#Download and set up drupal
cd /var/www/;
drush -y dl drupal;
mkdir "$d7dir"; #this is the line that always produces a file instead a directory
# regardless if it is replaced by the variable or entered as
# /var/www/d7
cd /var/www/drup*;
cp .htaccess .gitignore "$d7dir";
cp -r * "$d7dir";
cd "$d7dir";
rm -r /var/www/drup*;
mkdir "$d7dir"sites/default/files;
chmod 777 "$d7dir"sites/default/files;
cp "$dfsettings" "$settings";
chmod 777 "$settings";
chown $username:www-data /var/www/d7/.htaccess;
wget -O $d7dir"setupsite $DLURL/scripts/setupsite.sh; > /dev/null 2>&1
chmod +x /var/www/setupsite;
echo "Login Details following...";
read -sn 1 -p "Press any key to continue...";
bash "$d7dir"setupsite;
chown -Rh $username:www-data /var/www;
chmod 644 $d7dir".htaccess;
chmod 644"$settings";
chmod 644"$dfsettings";
exit
I hope someone got the reason for that.
There are many way to debug a shell-scripting.
Add set -x in your beginning script
Get the return value.
mkdir 'the-directory'
ret=$?
if test $ret -eq 0; then
echo 'Create success.'
else
echo 'Failed to create.'
fi
Set to verbose mode $ mkdir -v 'the-directory'
Try this command $ type mkdir, to checking mkdir command.

Getting `make install` to source your bash_completion

This is the install part of my Makefile:
install:
for e in $(EXEC); do \
sudo cp --remove-destination ${CURDIR}/$$e /usr/local/bin; done
sudo cp ${CURDIR}/bin/stage2.d /etc/bash_completion.d/stage2
. /etc/bash_completion
Where "stage2" is the name of my executable.
The last line is what provides the issue. After adding a file into bash_completion.d directory I want to source the bash_completion. But calling source or . yields:
. /etc/bash_completion
/etc/bash_completion: 32: [[: not found
/etc/bash_completion: 38: [[: not found
/etc/bash_completion: 50: Bad substitution
make: *** [install] Error 2
I see two issues:
Using sudo in a Makefile rule is a bad idea. Avoid that, and call sudo make install instead.
Why would you want to source the bash_completion file in the non-interactive shell which is the make rule? It makes no sense.
As to solving them:
install:
$(INSTALL) -m 0755 -d $(DESTDIR)$(bindir)
$(INSTALL) -m 0755 -p $(EXEC) $(DESTDIR)$(bindir)/
$(INSTALL) -m 0755 -d $(DESTDIR)$(sysconfdir)/bash_completion.d
$(INSTALL) -m 0644 -p ${CURDIR}/bin/stage2.d $(DESTDIR)$(sysconfdir)/bash_completion.d/stage2
for the make rule part and
sudo make install && . /etc/bash_completion.d/stage2
for the actual running of that command. You will want to document the latter in a README, or in a line which the install: target prints when finished.
Make uses /bin/sh by default. you have to force it to use bash since [[ is not supported by normal sh.
gnu make lets you set SHELL variable [in the makefile] to force it to use bash. So you would need to add a line
SHELL=/bin/bash
at the top of your makefile

Resources