So I'm trying to write some init.d scripts satisfying LSB so they run appropriately at startup and shutdown. Unfortunately, I'm having issues with LSB/update-rc.d to satisfy dependencies.
# Required-Start: $network $local_fs hadoop-namenode hadoop-datanode zookeeper-server
# Required-Stop: $network $local_fs hadoop-namenode hadoop-datanode zookeeper-server
# Default-Start: 2 3 4 5
# Default-Stop: 0 1 6
however, when I run update-rc.d defaults, it merely generates the defaults involved, with the exact same start time as the already dependencies, which breaks things horribly. Am I missing something with update-rc.d or LSB to get this configured?
In case it's relevant, this is Ubuntu 12.04
To make your script run later in the boot sequence, just add a sequence number, like 98 in the following command. Then most dependencies will be satisfied.
Once, I had to use a boot script which depended on other boot scripts. This brought a lot troubles. In the end I used this command to solve the problem:
cd /etc/init.d
sudo update-rc.d my_script defaults 98
The 98 means my_script gets sequence number 98 at boot, it looks like it's range from 1 to 99, and most other boot scripts have smaller numbers, this ensures my script has all its dependencies ready.
BTW, to remove the old script, this can be used:
sudo update-rc.d -f my_old_script remove
# -f means 'force'
Hope this helps.
This approach doesn't work anymore on some systems (Debian Squeeze for example). The number is just ignored with no explanation. The preferred method is this: http://refspecs.linuxbase.org/LSB_3.1.1/LSB-Core-generic/LSB-Core-generic/initscrcomconv.html
Look at "Required-Start:" script declaration.
Related
This question already has answers here:
Using conda activate or specifying python path in bash script?
(2 answers)
Closed 6 months ago.
I am trying to submit a SLURM job on a computing cluster CentOS7. The job is a python file (cifar100-vgg16.py) which needs tensorflow-gpu 2.8.1, which I've installed in a conda environment (tf_gpu). The bash script I'm submitting to SLURM (our job scheduler) is copied below. The SLURM output file shows that the environment being used is the base conda environment Python/3.6.4-foss-2018a (with tensorflow 1.10.1), not tf_gpu. Please advise on how to solve.
Bash script:
#!/bin/bash --login
########## SBATCH Lines for Resource Request ##########
#SBATCH --time=00:10:00 # limit of wall clock time - how long the job will run (same as -t)
#SBATCH --nodes=1 # the number of node requested.
#SBATCH --ntasks=1 # the number of task to run
#SBATCH --cpus-per-task=1 # the number of CPUs (or cores) per task (same as -c)
#SBATCH --mem-per-cpu=2G # memory required per allocated CPU (or core) - amount of memory (in bytes)
#SBATCH --job-name test2 # you can give your job a name for easier identification (same as -J)
########## Command Lines to Run ##########
conda activate tf_gpu
python cifar100-vgg16.py
SLURM output file:
> /opt/software/Python/3.6.4-foss-2018a/lib/python3.6/site-packages/tensorflow/python/framework/dtypes.py:523: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
_np_qint8 = np.dtype([("qint8", np.int8, 1)])
/opt/software/Python/3.6.4-foss-2018a/lib/python3.6/site-packages/tensorflow/python/framework/dtypes.py:524: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
_np_quint8 = np.dtype([("quint8", np.uint8, 1)])
/opt/software/Python/3.6.4-foss-2018a/lib/python3.6/site-packages/tensorflow/python/framework/dtypes.py:525: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
_np_qint16 = np.dtype([("qint16", np.int16, 1)])
/opt/software/Python/3.6.4-foss-2018a/lib/python3.6/site-packages/tensorflow/python/framework/dtypes.py:526: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
_np_quint16 = np.dtype([("quint16", np.uint16, 1)])
/opt/software/Python/3.6.4-foss-2018a/lib/python3.6/site-packages/tensorflow/python/framework/dtypes.py:527: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
_np_qint32 = np.dtype([("qint32", np.int32, 1)])
/opt/software/Python/3.6.4-foss-2018a/lib/python3.6/site-packages/tensorflow/python/framework/dtypes.py:532: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
np_resource = np.dtype([("resource", np.ubyte, 1)])
Tensorflow version 1.10.1
Keras version 2.1.6-tf
Scikit learn version 0.20.0
Traceback (most recent call last):
File "cifar100-vgg16.py", line 39, in <module>
print("Number of GPUs Available:", len(tensorflow.config.experimental.list_physical_devices('GPU')))
AttributeError: module 'tensorflow' has no attribute 'config'
There is a mistake in your job script. Replace conda activate tf_gpu with source activate tf_gpu.
Also, I guess you need to load the module so that you can use it. It would be something like module load anaconda check module avail for the list of available modules.
But looks like you HPC doesn't need a module load as it identifies conda without module load.
EDIT: FlytingTeller said that source activate is replaced with conda activate in 2017. I know this.
I don't know if this works on HPCs. To prove my point here is the output of Swansea SUNBIRD, when I use conda activate.
(base) hell#Dell-Precision-T7910:~$ ssh sunbird
Last login: Wed Aug 10 15:30:29 2022 from en003013.swan.ac.uk
====================== Supercomputing Wales - Sunbird ========================
This system is for authorised users, if you do not have authorised access
please disconnect immediately, and contact Technical Support.
-----------------------------------------------------------------------------
For user guides, documentation and technical support:
Web: http://portal.supercomputing.wales
-------------------------- Message of the Day -------------------------------
SUNBIRD has returned to service unchanged. Further information on
the maintenance outage and future work will be distributed soon.
===============================================================================
[s.1915438#sl2 ~]$ module load anaconda/3
[s.1915438#sl2 ~]$ conda activate base
CommandNotFoundError: Your shell has not been properly configured to use 'conda activate'.
If your shell is Bash or a Bourne variant, enable conda for the current user with
$ echo ". /apps/languages/anaconda3/etc/profile.d/conda.sh" >> ~/.bashrc
or, for all users, enable conda with
$ sudo ln -s /apps/languages/anaconda3/etc/profile.d/conda.sh /etc/profile.d/conda.sh
The options above will permanently enable the 'conda' command, but they do NOT
put conda's base (root) environment on PATH. To do so, run
$ conda activate
in your terminal, or to put the base environment on PATH permanently, run
$ echo "conda activate" >> ~/.bashrc
Previous to conda 4.4, the recommended way to activate conda was to modify PATH in
your ~/.bashrc file. You should manually remove the line that looks like
export PATH="/apps/languages/anaconda3/bin:$PATH"
^^^ The above line should NO LONGER be in your ~/.bashrc file! ^^^
[s.1915438#sl2 ~]$ source activate base
(base) [s.1915438#sl2 ~]$
Here is the output of Cardiff HAWK when I use conda activate.
(base) hell#Dell-Precision-T7910:~$ ssh cardiff
Last login: Tue Aug 2 09:32:44 2022 from en003013.swan.ac.uk
======================== Supercomputing Wales - Hawk ==========================
This system is for authorised users, if you do not have authorised access
please disconnect immediately, and contact Technical Support.
-----------------------------------------------------------------------------
For user guides, documentation and technical support:
Web: http://portal.supercomputing.wales
-------------------------- Message of the Day -------------------------------
- WGP Gluster mounts are now RO on main login nodes.
- WGP RW access is via Ser Cymru system or dedicated access VM.
===============================================================================
[s.1915438#cl1 ~]$ module load anaconda/
anaconda/2019.03 anaconda/2020.02 anaconda/3
anaconda/2019.07 anaconda/2021.11
[s.1915438#cl1 ~]$ module load anaconda/2021.11
INFO: To setup environment run:
eval "$(/apps/languages/anaconda/2021.11/bin/conda shell.bash hook)"
or just:
source activate
[s.1915438#cl1 ~]$ conda activate
CommandNotFoundError: Your shell has not been properly configured to use 'conda activate'.
To initialize your shell, run
$ conda init <SHELL_NAME>
Currently supported shells are:
- bash
- fish
- tcsh
- xonsh
- zsh
- powershell
See 'conda init --help' for more information and options.
IMPORTANT: You may need to close and restart your shell after running 'conda init'.
[s.1915438#cl1 ~]$ conda activate base
CommandNotFoundError: Your shell has not been properly configured to use 'conda activate'.
To initialize your shell, run
$ conda init <SHELL_NAME>
Currently supported shells are:
- bash
- fish
- tcsh
- xonsh
- zsh
- powershell
See 'conda init --help' for more information and options.
IMPORTANT: You may need to close and restart your shell after running 'conda init'.
[s.1915438#cl1 ~]$ source activate base
(2021.11)[s.1915438#cl1 ~]$
The conda versions are certainly after 2020 not 2017. SInce, the question was about a HPC cluster, that I why said replace conda activate with source activate, to activate the first conda environment.
Anyone with a possible explanation?
EDIT 2: I think I have an explanation.
[s.1915438#sl2 ~]$ cat ~/.bashrc
# .bashrc
# Dynamically generated by: genconfig (Do not edit!)
# Source global definitions
if [ -f /etc/bashrc ]; then
. /etc/bashrc
fi
# Load saved modules
module load null
# Personal settings file
if [ -f $HOME/.myenv ]
then
source $HOME/.myenv
fi
The ~/.bashrc does not contain the path to conda.sh. I think this is true for many HPCs.
So I am making a script that can run these commands whenever a server boot/reboot:
sudo bash
su - erp
cd frappe-bench/
bench start >/tmp/bench_log &
I found guides here and there about how can I change user in script I came out with the following script:
#! /bin/sh
sudo -u erp bash
cd /home/erp/frappe-bench/
bench start >/tmp/bench_log &
And, I have created a service at /etc/systemd/system/ and set it to run automatically when the server boots up.
The problem is, whenever I run sudo systemctl start erpnextd.service and checked the status, it came up with this
May 24 17:10:05 appbsystem2 systemd[1]: Started ERPNext | Auto Restart.
May 24 17:10:05 appbsystem2 sudo[18814]: root : TTY=unknown ; PWD=/ ; USER=>erp ; COMMAND=/bin/bash
May 24 17:10:05 appbsystem2 systemd[1]: erpnextd.service: Succeeded.
But it still doesn't start up ERPNext.
All I wanted to do is make a script that will start erpnext automatically everytime a server reboot.
Note: I only install frappe-bench on user erp only
Because you are using systemd, you already have all the features from your script available, and better. So you don't even need the script anymore:
[Unit]
Description=...
[Service]
# Run as user erp.
User=erp
# You probably also want to run as group erp, if it exists.
Group=erp
# Change to this directory before executing.
WorkingDirectory=/home/erp/frappe-bench
# Redirect standard output to the given log file.
StandardOutput=file:/tmp/bench_log
# Redirect standard error to the same log file.
StandardError=file:/tmp/bench_log
# Command line for starting the program. Make sure to use an absolute path!
ExecStart=/full/path/to/bench start
[Install]
WantedBy=multi-user.target
Using crontab (the script will start after every reboot/startup)
#crontab -e
#reboot sh /full/path/to/bench start >/tmp/bench_log
The answer provide by Thomas is very helpful.
However, I found another workaround by adding the path of my script file into the bottom of /etc/rc.local file.
Both method works, just a matter of preference ;)
TL;DR: update your bin/qgtunnel.
I've recently noticed an increase in my web dyno's memory usage. After digging a bit, I could see that the LD_PRELOAD variable that should be set with heroku-buildpack-jemalloc was not set correctly. I used a tiny script (bin/show_preload) that helped me debug that and trace which program was overriding LD_PRELOAD.
#!/usr/bin/env bash
echo "buildpack=foo preload='$LD_PRELOAD' at=start-app cmd='$#'"
$#
I introduced that in our Procfile:
web: bin/show_preload bin/qgtunnel bin/show_preload bin/start-nginx bin/show_preload bin/start-pgbouncer bin/show_preload bundle exec puma -C config/puma.rb
And when lauching on heroku I can see that bin/qgtunnel overrides our LD_PRELOAD configuration.
I created a tiny helper for the time being which makes sure I keep original value as well as what is added by bin/qgtunnel:
#!/usr/bin/env bash
after_qgtunnel_script=$(mktemp)
echo <<-BASH > $after_qgtunnel_script
# Retrieve previous LD_PRELOAD value
export LD_PRELOAD="\$LD_PRELOAD $LD_PRELOAD"
# Clean after usage
rm $after_qgtunnel_script
# Start following commands
$#
BASH
chmod +x $after_qgtunnel_script
bin/qgtunnel $after_qgtunnel_script $#
If you ever need this script use it in place of bin/qgtunnel
After reaching out to Quotaguard, they patched the qgtunnel binary and there is no error anymore:
curl https://quotaguard.s3.amazonaws.com/qgtunnel-2.4.1.tar.gz | tar xz
git add bin/qgtunnel vendor/nss_wrapper/libnss_wrapper.so
git commit -m "Update qgtunnel to fix LD_PRELOAD"
NOTE: new versions may occur since that one, see the related documentation
This is my first post.
I searched a lot and could not find all these pieces in one place. I gathered them from various posts on SO and managed to create a working script. All the credits go to their respective creators.
It's a bash script rather than a question.Hope it helps someone. Please feel free to suggest improvements.
#!/bin/bash
# Author: krishna - and thanks to all the helpful folks on SO.
# date: apr-03-2018
# script
# Notes: unattented script to install percona server. Pieced together from all over SO.
# see versions below next to each item. they are current as on this script creation dates.
# 2. percona server 5.7 or later
# 4. ubuntu 64bit 16.4.3
#
# This script works for me but may not be the best bash script, Please help, to make it better.
#
# you have to run this script as sudo
# if you saved this script as let's say install-percona.sh
# then do this
# chmod +x install-percona.sh
# sudo ./install-percona.sh
#
# do everything in downloads
mkdir -p ~/downloads
cd ~/downloads
# Variables
# Passwords are like under garments.
# 1. Dont leave them lying around
# 2. Change Often
# 3. Do not share it with others
# 4. Dont be chaep. Invest in STRONG AND high quality one's.
# So I hope you know what to do with the next 3 lines.
MYSQL_ROOT_PASSWORD="SecretRootPassword123"
DB_USERNAME="MickeyMouse"
DB_PASSWORD="SecretDBPassword123"
export DEBIAN_FRONTEND=noninteractive
sudo apt-get install -y debconf-utils
# Install software
# We are going to install Percona Server.
#
# By default, percona-server is going to ask for the root password and we automate that with debconf-set-selections
# begin percona stuff
# you need to change the next 2 lines, visit the percona server site to get latest values.
wget https://repo.percona.com/apt/percona-release_0.1-4.$(lsb_release -sc)_all.deb
dpkg -i percona-release_0.1-4.$(lsb_release -sc)_all.deb
#
apt-get update
# also change the version number accordingly. the -5.7 part
# Note. its root-pass and re-root-pass unlike root_password and root_password_again for other flavors of mysql.
#
# !!!!! CAUTION !!!!
# Be sure to check /var/cache/debconf/passwords.dat file for these 2 entries. After installation is completed.
# The value fields were clear in my case but check anyway.
# Y.M.M.V - dont want to leave passwords hanging in the clear.
#
#
echo "percona-server-server-5.7 percona-server-server-5.7/root-pass password ${MYSQL_ROOT_PASSWORD}" | debconf-set-selections
echo "percona-server-server-5.7 percona-server-server-5.7/re-root-pass password ${MYSQL_ROOT_PASSWORD}" | debconf-set-selections
apt install -y percona-server-server-5.7
# Configure MySQL ie Percona
# We are going to create a user also and the 3 CREATE FUNCTION lines are what percona suggests.
# these may happen silently
echo "lets configure percona."
mysql -u root -p${MYSQL_ROOT_PASSWORD} <<EOF
CREATE FUNCTION fnv1a_64 RETURNS INTEGER SONAME 'libfnv1a_udf.so';
CREATE FUNCTION fnv_64 RETURNS INTEGER SONAME 'libfnv_udf.so';
CREATE FUNCTION murmur_hash RETURNS INTEGER SONAME 'libmurmur_udf.so';
CREATE USER '${DB_USERNAME}'#'localhost' IDENTIFIED BY '${DB_PASSWORD}';
GRANT ALL ON *.* TO '${DB_USERNAME}'#'localhost' with grant option;
FLUSH Privileges;
EOF
echo "make sure you run mysql_secure_installation after install"
# run the secure installation script
mysql_secure_installation
echo "done."
This is the contents of my /etc/rc.local file. It is supposed to run on login on my raspberry pi, yet it just logs in in (as I'm using auto login) and then does nothing, i.e. it just sits there with pi#raspberrypi ~$_ waiting for a command. I have no idea why it's not working nor any experience with bash scripts.
It should mount a usb then run a file on said usb but it doesn't.
#!/bin/sh -e
#
# rc.local
#
# This script is executed at the end of each multiuser runlevel.
# Make sure that the script will "exit 0" on success or any other
# value on error.
#
# In order to enable or disable this script just change the execution
# bits.
#
# By default this script does nothing.
sudo /bin/mount /dev/sda1 /media/robousb
sudo python media/robousb/Robopython/usercode_old.py
exit 0
I assuming you're running Raspbian, which is pretty much Debian.
rc.local runs as root before login, so you don't need or want sudo; it may be causing an error, hence nothing happening.
User-level commands that run for any user when they log in (unlike rc.local, which runs before login) can be put into /etc/bash.bashrc. That may be more applicable to your situation, at least the second command.
Login commands for the pi user only can be put into /home/pi/.bashrc
I don't know raspberry-pi but you could try to write something into a file to see if the file is running or not. For example :
touch /tmp/test.txt
echo "$(date) => It's running" > /tmp/test.txt
If it doesn't work, I know that on some OS (fedora, rhel, centos for example), the path of that file is /etc/init.d/rc.local. It doesn't cost anything to try this path ;)
I have the exact same problem with RPi3/Jessie.
I suggest you to launch your script in the bashrc by doing
sudo emacs /home/pi/.bashrc
In my case i wrote at the EOF:
bash /home/pi/jarvis/jarvis.sh -b &
And that works well at each startup.
I have the same problem. In Raspbian forum is the solution:
Just change the first row from #!/bin/sh -e to
#!/bin/bash
Ivan X is right. You don´t need sudo command.