Using a GCP Ubuntu 18.04 image, plain.
I use startup scripts on vm to automate the deployment of features and changes to the base image. I have one that start with the following :
#! /bin/bash
add-apt-repository 'deb https://cloud.r-project.org/bin/linux/ubuntu bionic-cran35/'
apt update
apt -y upgrade
export DEBIAN_FRONTEND=noninteractive
apt -y install libpam-dev libpam-ldap r-base
Unfortunatly on some machine that haven't ran in a while, I get the following error:
From /var/log/syslog :
startup-script: INFO startup-script: dpkg: error: dpkg frontend is locked by another process
From my investigation I can see that the process unattended upgrades is locking the file because it is doing the automatic security updates to the system. If I let the automatic upgrades complete and launch the script manually, everything runs just fine.
Is there a way to delay the execution of the startup script so that it starts after unattended upgrades is done? I mean, something that is more reliable than a simple wait command.
The startup script is configured via the metadata of the vm per GCP documentation.
Thanks a lot and have a great day.
You can have your script wait until the dpkg lock is no longer held. This AskUbuntu answer suggests a solution using fuser to see if the lock file is in use by another process:
#!/bin/bash
while fuser /var/lib/dpkg/lock >& /dev/null; do
echo "waiting for other package installs to complete..."
sleep 1
done
add-apt-repository 'deb https://cloud.r-project.org/bin/linux/ubuntu bionic-cran35/'
apt update
apt -y upgrade
export DEBIAN_FRONTEND=noninteractive
apt -y install libpam-dev libpam-ldap r-base
Related
I was trying to setup a build environment on WSL. After starting it up and running sudo apt update -y && sudo apt upgrade -y It started doing its thing. But then got super slow (20kb/s) So i deleted the whole WSL and redownloaded it... Same issue. I tried disabling IPV6 in my sysctl that also didnt work.
Any ideas?
Check the local closet mirror for you from here and update sources.list file
sudo sed -i "s/archive.ubuntu.com/us.archive.ubuntu.com/" /etc/apt/sources.list
Found this at source
After upgrade from ubuntu 20.04 LTS to 22.04.1 LTS, I got a very persistent error:
(Reading database ... 350976 files and directories currently installed.)
Preparing to unpack .../firefox_1%3a1snap1-0ubuntu2_amd64.deb ...
=> Installing the firefox snap
==> Checking connectivity with the snap store
==> Installing the firefox snap
error: cannot perform the following tasks:
- Run hook connect-plug-host-hunspell of snap "firefox" (run hook "connect-plug-
host-hunspell": cannot perform operation: mount --rbind /var/log /tmp/snap.rootf
s_hE2Zj1//var/log: Permission denied)
dpkg: error processing archive /var/cache/apt/archives/firefox_1%3a1snap1-0ubunt
u2_amd64.deb (--unpack):
new firefox package pre-installation script subprocess returned error exit stat
us 1
Please restart all running instances of firefox, or you will experience problems
.
Errors were encountered while processing:
/var/cache/apt/archives/firefox_1%3a1snap1-0ubuntu2_amd64.deb
E: Sub-process /usr/bin/dpkg returned an error code (1)
This
- Run hook connect-plug-host-hunspell of snap "firefox" (run hook "connect-plug-
host-hunspell": cannot perform operation: mount --rbind /var/log /tmp/snap.rootf
s_hE2Zj1//var/log: Permission denied)
was very persistent and was hindering any apt-involved installations.
Thus, no apt install nor apt upgrade was working.
After long search and trying around,
where I did:
sudo apt --fix-broken install
sudo rm /var/lib/dpkg/lock
sudo rm /var/lib/dpkg/lock-frontend
sudo rm /var/lib/apt/lists/lock
sudo rm /var/cache/apt/archives/lock
sudo dpkg --configure -a
And then, cave(!) this removes firefox from your installed package list
and thus after this command you cannot use firefox any more on your computer
until you install it - so I did that - but I had a second computer
where I could google around while I had no firefox on that machine.
I couldn't install chromium browser or other browsers, because apt was not working! So run this command only when you have a second computer or at least your mobile to surf for instructions!
sudo dpkg --force depends -P firefox
I found a hint in
https://forums.mozillazine.org/viewtopic.php?f=38&t=3097766
My solution was:
# Add Mozilla Team PPA
sudo add-apt-repository ppa:mozillateam/ppa
# Set PPA priority
sudo gedit /etc/apt/preferences.d/mozillateamppa
# The command creates and opens empty config file in Gedit text editor.
# When it opens, add the lines below and save it:
Package: firefox*
Pin: release o=LP-PPA-mozillateam
Pin-Priority: 501
save and close that file.
after that, I could do finally:
sudo apt --fix-broken install
# and then:
sudo apt update && sudo apt upgrade
and then, all apt or snap commands were working again flawlessly.
note:
Now, I encounter
sudo apt install chromium-browser
Reading package lists... Done
Building dependency tree... Done
Reading state information... Done
The following NEW packages will be installed:
chromium-browser
0 upgraded, 1 newly installed, 0 to remove and 4 not upgraded.
Need to get 0 B/48,4 kB of archives.
After this operation, 164 kB of additional disk space will be used.
Preconfiguring packages ...
(Reading database ... 313313 files and directories currently installed.)
Preparing to unpack .../chromium-browser_1%3a85.0.4183.83-0ubuntu2_amd64.deb ...
=> Installing the chromium snap
==> Checking connectivity with the snap store
==> Installing the chromium snap
error: cannot perform the following tasks:
- Run configure hook of "chromium" snap if present (run hook "configure": cannot perform operation: mount --rbind /var/log /
tmp/snap.rootfs_Gg42mE//var/log: Permission denied)
dpkg: error processing archive /var/cache/apt/archives/chromium-browser_1%3a85.0.4183.83-0ubuntu2_amd64.deb (--unpack):
new chromium-browser package pre-installation script subprocess returned error exit status 1
Errors were encountered while processing:
/var/cache/apt/archives/chromium-browser_1%3a85.0.4183.83-0ubuntu2_amd64.deb
E: Sub-process /usr/bin/dpkg returned an error code (1)
I tried:
sudo add-apt-repository ppa:xtradeb/apps
sudo gedit /etc/apt/preferences.d/xtradebppa
# content:
Package: chromium*
Pin: release o=LP-PPA-xtradeb
Pin-Priority: 501
But this didn't help.
Finally, I found out the solution!
In former days, when my / containing system partition was too full, I sym-linked /var/log. I linked it to somehwere in my home folder. But then, I moved snap back.
- Run hook connect-plug-host-hunspell of snap "firefox" (run hook "connect-plug-
host-hunspell": cannot perform operation: mount --rbind /var/log /tmp/snap.rootf
s_hE2Zj1//var/log: Permission denied)
The permission was denied, because it tried to mount to a symlink.
All I had to do was:
sudo rm /var/log
sudo mkdir -p /var/log
Now, it is not a symlink any more. So it can actually mount to it.
I have a cloudformation template that I have created in hopes to spin up an ec2 instance with the necessary dependencies (where these dependencies are installed as bash in UserData) to leverage GPU hardware within a docker container. The main dependencies are: 1) nvidia drivers, 2) docker, and 3) nvidia-docker2.
The first two dependencies install as expected and after several moments of running can be verified by 1) nvidia-smi, and docker --version. The third dependency however consistently does not install.
For reference here are the relevant parts of my UserData bash:
# install gpu stuff
apt-get install linux-headers-$(uname -r)
distribution=$(. /etc/os-release;echo $ID$VERSION_ID | sed -e 's/\.//g')
wget https://developer.download.nvidia.com/compute/cuda/repos/$distribution/x86_64/cuda-$distribution.pin
mv cuda-$distribution.pin /etc/apt/preferences.d/cuda-repository-pin-600
apt-key adv --fetch-keys https://developer.download.nvidia.com/compute/cuda/repos/$distribution/x86_64/7fa2af80.pub
echo "deb http://developer.download.nvidia.com/compute/cuda/repos/$distribution/x86_64 /" | tee /etc/apt/sources.list.d/cuda.list
apt-get update
apt-get -y install cuda-drivers
# install docker on system
curl https://get.docker.com | sh
systemctl start docker && systemctl enable docker
distribution=$(. /etc/os-release;echo $ID$VERSION_ID)
curl -s -L https://nvidia.github.io/nvidia-docker/gpgkey | apt-key add -
curl -s -L https://nvidia.github.io/nvidia-docker/$distribution/nvidia-docker.list | tee /etc/apt/sources.list.d/nvidia-docker.list
apt-get -y install nvidia-docker2 > /var/log/mason
# add nvidia runtime stuff
# echo "{ \"runtimes\": { \"nvidia\": { \"path\": \"/usr/bin/nvidia-container-runtime\", \"runtimeArgs\": [] } } }" >> /etc/docker/daemon.json
systemctl restart docker
I have tried to pipe the stdout from apt-get -y install nvidia-docker2 to a log file but the logs only show:
Reading package lists...
Building dependency tree...
Reading state information...
and seems to be stuck there.
Other potential helpful bits:
AMI: ubuntu 18.04 image
I will also note that I am able to SSH into the instance and install the apt-get -y install nvidia-docker2 in the command terminal without a hitch (or any user prompt or anything).
Can anyone help me figure out how to trouble shoot this issue or does anyone see any potential problems in what I have shared above? The stdout pipe to file is about the only trick I know to debug such an issue as this. Please let me know if I can update/edit this post to make this issue easier to debug.
Based on the comments.
The issue was caused by not updating ubuntu's repositories after adding nvidia-docker2 repo.
The solution was to run apt-get update after the addition of the repo.
replace:
distribution=$(. /etc/os-release;echo $ID$VERSION_ID | sed -e 's/\.//g')
with:
distribution = ubuntu18.04
I'm writing a basic script in bash to automatically install a few packages on new machines (basically, the first script to run on a new machine before doing anything else). I'm downloading openssh-server and python for example.
My goal is just to launch the script in sudo mode, wait a few minutes and get started. The only problem I'm facing is the machine requires the user to input "y" when needed and I can't find a way to bypass this.
This is just a sample of my script :
#!/bin/bash
sudo apt update
sudo apt install openssh-server
sudo apt-get install python3
what I expect is just run this command and let it roll without the user to have and put "y" when needed
sudo bash start-script.sh
Use the -y option of apt-get. From the apt-get man page:
-y, --yes, --assume-yes
Automatic yes to prompts; assume "yes" as answer to all prompts and
run non-interactively. If an undesirable situation, such as
changing a held package, trying to install a unauthenticated
package or removing an essential package occurs then apt-get will
abort. Configuration Item: APT::Get::Assume-Yes.
I am using AWS Elastic Beanstalk on a Linux machine and need to install some fonts in .ebextensions:
container_commands:
01_getfont:
command: sudo yum -y install http://somesite.com/rpm/webcore-fonts-3.0-1.noarch.rpm
That works well the 1st time, with the fonts installed.
The 2nd time when I deploy the EB again, it now gave me this error:
Application update failed at 2019-01-28T23:44:14Z with exit status 1 and error: container_command 01_getfont in .ebextensions/fonts.config failed.
Loaded plugins: priorities, update-motd, upgrade-helper
Examining /var/tmp/yum-root-0Yx1DY/webcore-fonts-3.0-1.noarch.rpm: webcore-fonts-3.0-1.noarch
/var/tmp/yum-root-0Yx1DY/webcore-fonts-3.0-1.noarch.rpm: does not update installed package.
Error: Nothing to do.
How do I avoid getting that errors when that package has been installed on the same EC2 instance the 2nd time?
I found out the answer to this problem later, posting it here for the benefits of others with similar issue.
I use reinstall instead:
sudo yum -y reinstall http://somesite.com/rpm/webcore-fonts-3.0-1.noarch.rpm
This will work the 1st time and all other times of deployment.
Edit:
The above does not work as well as reinstall will fail if package is not installed. I ended up detecting if the package has been installed, if not, install it else re-install:
command: sudo yum -q list installed webcore-fonts.noarch &>/dev/null && sudo yum -y reinstall http://somesite.com/rpm/webcore-fonts-3.0-1.noarch.rpm || sudo yum -y install http://somesite.com/rpm/webcore-fonts-3.0-1.noarch.rpm
Use the packages directive:
packages:
rpm:
webcore-fonts: http://somesite.com/rpm/webcore-fonts-3.0-1.noarch.rpm
This will handle not installed and already installed scenarios.
Yum can return a non-zero exit status for things that are not really errors, causing higher-level systems such as Elastic Beanstalk to think the script has failed.
In particular, yum sometimes says "Nothing to do" with exit status of 1 - this can mean various things but includes the case where required packages are already installed.
The way I work around this for scripts using yum is:
yum -y install somepackage
if [ $? -ne 1 ]; then # Exit on any any error except 'nothing to do'
exit 0
fi
A simpler way is just to ignore all errors by appending a true or exit 0 command - however, this is eventually going to bite you when the Yum repo is unreachable, or Yum has out of date metadata, etc.
Advanced tip
If you have several yum commands, or more error codes to handle, you might want to read up on the shell trap command, specifically on EXIT or ERR which lets you handle these cases in a single place, and potentially not exit on unwanted errors. See this stack for more on this.
Alternative for local installs
See this answer for more, including a simple alternative when installing RPMs that you have downloaded.