I am in the process of setting up a Ubuntu 14.04 server to automate hybrid Android app builds with Phonegap CLI. Having written up all the relevant scripts I ran into a rather strange problem - when I SSH in to my server I can run the script and run all Phonegap commands successfully in my interactive shell session. However, every attempt to run those same commands in an automated script that gets triggered by some other - visitor generated - event fails. To pin down the problem I reduced it down to a simple experiment which I outline below.
Step 1 - write a startup script, pgtest in /etc/init.d
#! /bin/bash
source ~/.nvm/nvm.sh;
nvm use stable;
cd /home;
ls >> /tmp/ls;
which node >> /tmp/node;
which git >> /tmp/git;
which phonegap >> /tmp/pgp;
phonegap -v >> /tmp/pgpv 2>/tmp/pgpe;
Explanations
I use NVM to manage Node so I am making sure that the system knows where to find nvm.sh
I am firing up NVM to use the stable (4.1.1.) version of Node + NPM
I want to make sure that my batch file is actually being executed so I do ls /home and pipe its output to the /tmp/ls file.
I want to be sure that node, git and phonegap are actually available so I do pipe the output from which node|git|phonegap to files in the /tmp folder.
Little point in complicating things so I am issuing the simplest of Phonegap commands, phonegap -v to report the current version number. Any errors that might happen when doing this are being piped into the file /tmp/pgpe.
Step 2 - Make sure that pgtest is run last
ln -s /etc/init.d/pgtest /etc/rc2.d/S04PGTest
Explanation - I only want this script to be run after everything else on my server has had a chance to startup
With all of this in place I rebooted by server and examined the contents of the /tmp folder. My findings
ls - the folder listing for the /home folder present and correct.
node, git and pgp point to the locations of Node, Git and Phonegap
pgpv, which should contain the Phonegap version number, is EMPTY
pgpe is present and NOT empty
That last implies that the system encountered an error whilst attempting to execute phonegap -v. Here are the contents of pgpe.
path.js:8
throw new TypeError('Path must be a string. Received ' +
^
TypeError: Path must be a string. Received undefined
at assertPath (path.js:8:11)
at Object.posix.join (path.js:477:5)
at Object.
(/root/.nvm/versions/node/v4.1.1/lib/node_modules/phonegap/node_modules/phonegap-build/lib/common/config/global.js:17:28)
at Module._compile (module.js:434:26)
at Object.Module._extensions..js (module.js:452:10)
at Module.load (module.js:355:32)
at Function.Module._load (module.js:310:12)
at Module.require (module.js:365:17)
at require (module.js:384:17)
at Object.
(/root/.nvm/versions/node/v4.1.1/lib/node_modules/phonegap/node_modules/phonegap-build/lib/common/config.js:9:13)
Now here is the curious thing. If I clear out the /tmp folder and issue a /etc/init.d/pgtest in an interactive shell session I get the following results
/tmp/ls present and populated with the /home folder listing as before
/tmp/node, /tmp/git /tmp/pgp present and correct
/tmp/pgpvreports 5.3.6 - the current Phonegap version number
/tmp/pgpe is EMPTY , i.e, no errors are reported
Clearly, the interactive bash shell environment has something that is not present when I run an automated script - at startup in this case but it also happens when I trigger the process via an automated script in any other way.
With all of this I am moving closer to pinning down the cause of the problem. However, there my knowledge of how these systems work is letting me down. What is the difference between the interactive shell environment and the one that is encountered by my automated script? Just how do I interpret the errors reported in /tmp/pgpe? What do I do to fix them?
I'd be most grateful to anyone who might be able to put me on the right track here.
Edit in light of #Eduardo's suggestions. I grabbed the two sets of environments (interactive & init.d). The results of doing a DIFF (interactive vs init.d) can be found in this fiddle. A somewhat less accessible dump of the DIFF result is shown below
--- /home/env.inter 2015-11-11 08:30:40.314172560 +0000
+++ /home/env.auto 2015-11-11 08:32:55.240906000 +0000
## -1,48 +1,38 ##
BASH=/bin/bash
BASHOPTS=cmdhist:complete_fullquote:extquote:force_fignore:hostcomplete:interactive_comments:login_shell:progcomp:promptvars:sourcepath
BASH_ALIASES=()
-BASH_ARGC=()
-BASH_ARGV=()
+BASH_ARGC=([0]="1")
+BASH_ARGV=([0]="start")
BASH_CMDS=()
BASH_LINENO=([0]="0")
-BASH_SOURCE=([0]="/etc/init.d/pgtest")
+BASH_SOURCE=([0]="/etc/rc2.d/S04pgtest")
BASH_VERSINFO=([0]="4" 1="3" [2]="11" [3]="1" [4]="release" [5]="x86_64-pc-linux-gnu")
BASH_VERSION='4.3.11(1)-release'
DIRSTACK=()
EUID=0
GROUPS=()
-HOME=/root
HOSTNAME=example.com
HOSTTYPE=x86_64
IFS=$' \t\n'
-LANG=en_US.UTF-8
-LESSCLOSE='/usr/bin/lesspipe %s %s'
-LESSOPEN='| /usr/bin/lesspipe %s'
-LOGNAME=root
-LS_COLORS='rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:.tar=01;31:.tgz=01;31:.arj=01;31:.taz=01;31:.lzh=01;31:.lzma=01;31:.tlz=01;31:.txz=01;31:.zip=01;31:.z=01;31:.Z=01;31:.dz=01;31:.gz=01;31:.lz=01;31:.xz=01;31:.bz2=01;31:.bz=01;31:.tbz=01;31:.tbz2=01;31:.tz=01;31:.deb=01;31:.rpm=01;31:.jar=01;31:.war=01;31:.ear=01;31:.sar=01;31:.rar=01;31:.ace=01;31:.zoo=01;31:.cpio=01;31:.7z=01;31:.rz=01;31:.jpg=01;35:.jpeg=01;35:.gif=01;35:.bmp=01;35:.pbm=01;35:.pgm=01;35:.ppm=01;35:.tga=01;35:.xbm=01;35:.xpm=01;35:.tif=01;35:.tiff=01;35:.png=01;35:.svg=01;35:.svgz=01;35:.mng=01;35:.pcx=01;35:.mov=01;35:.mpg=01;35:.mpeg=01;35:.m2v=01;35:.mkv=01;35:.webm=01;35:.ogm=01;35:.mp4=01;35:.m4v=01;35:.mp4v=01;35:.vob=01;35:.qt=01;35:.nuv=01;35:.wmv=01;35:.asf=01;35:.rm=01;35:.rmvb=01;35:.flc=01;35:.avi=01;35:.fli=01;35:.flv=01;35:.gl=01;35:.dl=01;35:.xcf=01;35:.xwd=01;35:.yuv=01;35:.cgm=01;35:.emf=01;35:.axv=01;35:.anx=01;35:.ogv=01;35:.ogx=01;35:.aac=00;36:.au=00;36:.flac=00;36:.mid=00;36:.midi=00;36:.mka=00;36:.mp3=00;36:.mpc=00;36:.ogg=00;36:.ra=00;36:.wav=00;36:.axa=00;36:.oga=00;36:.spx=00;36:.xspf=00;36:'
MACHTYPE=x86_64-pc-linux-gnu
-MAIL=/var/mail/root
-NVM_DIR=/root/.nvm
-NVM_IOJS_ORG_MIRROR=https://iojs.org/dist
-NVM_NODEJS_ORG_MIRROR=https://nodejs.org/dist
-NVM_RC_VERSION=
OPTERR=1
OPTIND=1
OSTYPE=linux-gnu
-PATH=/opt/android/platform-tools:/opt/android/tools:/opt/android:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games
+PATH=/sbin:/usr/sbin:/bin:/usr/bin
PIPESTATUS=([0]="0")
-PPID=4801
+PPID=911
+PREVLEVEL=N
PS4='+ '
-PWD=/etc/init.d
+PWD=/
+RUNLEVEL=2
SHELL=/bin/bash
SHELLOPTS=braceexpand:hashall:interactive-comments
-SHLVL=3
-SSH_CLIENT='nn.nn.nn.nn nnnn nnnn'
-SSH_CONNECTION='nn.nn.nn.nn nnnn nn.nn.nn.nn nnnn'
-SSH_TTY=/dev/pts/0
-TERM=xterm
+SHLVL=1
+TERM=linux
UID=0
-USER=root
-XDG_RUNTIME_DIR=/run/user/1000
-XDG_SESSION_ID=5
+UPSTART_EVENTS=runlevel
+UPSTART_INSTANCE=
+UPSTART_JOB=rc
_=n
+previous=N
+runlevel=2
The only things I have changed here - masked the Host name and the SSH client IP address.
I am pretty sure I had tried this in my own experiments prior to posting this question but following #Eduardo's suggestion below I tried sticking in a
EXPORT PATH=/opt/android/platform-tools:/opt/android/tools:/opt/android:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games
at the top of the script - just below the source ~/.nvm... line. A reboot later the result was still the same: an empty /tmp/pgpv and the same errors reported in /tmp/pgpe.
Just make sure to set the PATH variable inside your script to the same, longer one, you see on the diff, then retry automated.
I would probably also define the HOME and NVM* variables. As a test I'd create a test.sh script at the same directory of phonegap, with this content:
#!/bin/bash
set > /tmp/env_vars.log
And have your script as:
#!/bin/bash
export TERM=linux
export USER=root
export HOME=/root
export NVM_DIR=/root/.nvm
export NVM_IOJS_ORG_MIRROR=https://iojs.org/dist
export NVM_NODEJS_ORG_MIRROR=https://nodejs.org/dist
export NVM_RC_VERSION=
export PATH=/opt/android/platform-tools:/opt/android/tools:/opt/android:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games
source ~/.nvm/nvm.sh
nvm use stable
cd /home
ls > /tmp/ls
which node > /tmp/node
which git > /tmp/git
which phonegap > /tmp/pgp
phonegap -v > /tmp/pgpv 2>/tmp/pgpe;
test.sh
Related
I have a bash script on my test server that will export my wordpress db, rsync the db to the prod server, and git push all of my files to prod sever.
Within the prod server's git repo I have a git post-receive hook correctly configured.
#!/bin/bash
#Receive Git Push from Test
git --work-tree=/home/username/public_html --git-dir=/home/username/public_html/git/production-site.git checkout -f
Within the working tree directory (WordPress directory) on the prod server I also have a bash script that will import the newly uploaded db. /home/username/public_html/db-import-script.sh
#!/bin/bash
#bunch of commands
...
...
...
Question:
How can I automatically execute the db import script immediately following a git push?
troubleshooting:
inside of post-receive, I have tried using an absolute paths to execute the script, no luck
#!/bin/bash
#Receive Git Push from Test
git --work-tree=/home/username/public_html --git-dir=/home/username/public_html/git/production-site.git checkout -f
#execute script with absolute path
/home/username/public_html/db-import-script.sh
db-import-script.sh does not execute. NOTE: this script must remain located in the Wordpress directory b/c it uses wp-cli commands for various actions.
any tips?
I use e.g. gitea and on server one has to simply copy a script in post-receive.d/ folder. The post-receive hook (see below and you may use it as a template) will scan this folder and execute scripts in it.
#!/usr/bin/env bash
# AUTO GENERATED BY GITEA, DO NOT MODIFY
data=$(cat)
exitcodes=""
hookname=$(basename $0)
GIT_DIR=${GIT_DIR:-$(dirname $0)/..}
for hook in ${GIT_DIR}/hooks/${hookname}.d/*; do
test -x "${hook}" && test -f "${hook}" || continue
echo "${data}" | "${hook}"
exitcodes="${exitcodes} $?"
done
for i in ${exitcodes}; do
[ ${i} -eq 0 ] || exit ${i}
done
kiss rule... (keep it simple stupid)
rather than spending days trying to learn sysdig well enough to trace a process that I have never previously heard of and it subprocesses. (no offence intended Charles, just need to actually get tasks done. Your bash debug-log snippet highly useful)
and rather than creating some git / gitea hybrid (no offence #m19v, I did try your solution, but didn't work)
knowing that production server db-import.sh worked properly and that my test server git push / db upload push.sh worked properly.
My final solution was to leave the production server's post-receive properly configured and to.... remotely execute my db-import.sh script via ssh directly within the directory in which it needs to be executed.
In a nutshell, I added this to the end of push.sh script on my test server:
#Remotely execute db import
ssh -p22 -i /home/username/.ssh/id_rsa username#1233.456.789.12 'cd public_html && bash' << EOF
./db-import.sh
EOF
Bang problem solved...
I have a WSL Ubuntu distro that I've set up so that when I login 4 services start working, including a web API that I can test via Swagger to verify it is up and working.
I'm at the point where what I want to do now is start WSL via a script - that is, launch my distro, have all of the services start, and do it from Python. The problem is I cannot even figure out the correct syntax to get WSL to start from PowerShell in a manner where my services start.
Side note: "services" != systemctl (or similar) calls, but just executing bash CLI commands from either my .bashrc or .profile at login.
I've put the commands to execute in .profile & .bashrc. I've configured it both for root execution and non-root user execution. I've taken the commands out of those 2 files and put it into a script in the Windows file system that I pass in on the start of wsl. And I've put that shell script in the WSL file system as well. Nothing seems to work, and sometimes the distro starts and then stops after about 30 seconds.
Some of the PS CLI commands I've tried:
Start-Job -ScriptBlock{ wsl -d distro -u root }
Start-Job -ScriptBlock{ wsl -d distro -u root 'bash -i -l -c /root/bin/start.sh'
Start-Job -ScriptBlock{ wsl -d distro -u root 'bash -i -l -c .\start.sh'
wsl -d distro -u root -- bash -i -l -c /root/bin/start.sh
wsl -d distro -u root -- bash -i -l -c .\start.sh
wsl -d distro -u root -- /root/bin/start.sh
Permutations of the above that I've tried: replace root with my default login, and turning all of the Start-Job bash options into a comma-separated list of single-quoted strings (Ex: 'bash', '-i', '-l', ... ). Nothing I launch from the CLI will allow me access to the web API that is supposed to be hosted on my distro.
Any advice on what to try next?
Not necessarily an answer here as much as troubleshooting tips which will hopefully lead to an answer:
First, most of the forms that you are using seem to be correct. The only ones that absolutely shouldn't work are those that attempt to run the script from the Windows filesystem.
Make sure that you have a shebang line starting your script. I'm assuming you do, but other readers may come across this as well. For the moment, try this form:
#!/usr/bin/env -S bash -li
That's going to have the same effect as the bash -li you tried -- It will source both both interactive startup files such as ~/.bashrc as well as login profiles such as ~/.bash_profile (and /etc/profile.d/*, etc.).
Note that preferably, you won't need the -li. Best practice would be to move anything necessary for the services over from the startup scripts to your start.sh script, and avoid parsing the profile and rc. I need to go update some of my answers, since I just realized I've been guilty of giving some potentially bad advice ...
Specifically, though, I'm wondering if your interactive Bash config has something truly, well, "interactive" in it that might be preventing the automatic running of the script itself. Again, best practice would be for ~/.bashrc to only hold configuration that is needed for interactive shell sessions.
Make sure the script is set as executable (chmod +x start.sh). Again, I'm assuming this is the case for you.
With a shebang line and an executable script, use something like:
wsl -d distro -u root -e /root/bin/start.sh
The -e tells WSL to launch the script directly. Since it has a shebang line, it will be parsed by Bash. Most of the other forms you use above actually run Bash twice - Once when launching WSL and another when it finds the shebang line in the script.
Try some basic troubleshooting for your script like:
Add set -x to the top (right under the shebang line) to turn on script debugging.
Add a ps -efH at the end to show the processes that are running when the script completes
If needed, resort to quick-and-dirty echo statements to show where things have progressed in the script.
I'm hopeful that the above will at least show you the problem, but if not, add the debugging info that you gain from this to your question, and we can troubleshoot further.
UPDATE:
The proposed link was the solution to this question. On a Mac you have to make symlink:
sudo ln -s /usr/local/bin/node /usr/bin/node
I wanted to automate some processes on boot of a mac mini.
The script should fire up some node.js scripts (forever). The script itself is fired by a launchd process. I have several scripts that work like that. But that one I have a problem:
#!/bin/bash
export PATH=/usr/local/bin
export NODE_ENV=production
cd /Volumes/Services/Proxy
forever -a start /Volumes/Services/Proxy/app.js
This script works pretty fine if executed in the terminal. It makes what it does.
But this script throws errors on std.err when launched on boot.
env: node: no such file or directory
First I thought it's a problem because the PATH to "forever" ist not known on startup so I added the export PATH explicitly.
It throws the same error when I give the absolute path to "forever":
#!/bin/bash
export NODE_ENV=production
cd /Volumes/Services/Proxy
/usr/local/bin/forever -a start /Volumes/Services/Proxy/app.js
There is no error in the launched.plist file. If I add other content to this script like for example mkdir FOO it does this without problems.
... im not so firm with shell scripting... :(
Anyone knows what I'm doin wrong?
This is the contents of my /etc/rc.local file. It is supposed to run on login on my raspberry pi, yet it just logs in in (as I'm using auto login) and then does nothing, i.e. it just sits there with pi#raspberrypi ~$_ waiting for a command. I have no idea why it's not working nor any experience with bash scripts.
It should mount a usb then run a file on said usb but it doesn't.
#!/bin/sh -e
#
# rc.local
#
# This script is executed at the end of each multiuser runlevel.
# Make sure that the script will "exit 0" on success or any other
# value on error.
#
# In order to enable or disable this script just change the execution
# bits.
#
# By default this script does nothing.
sudo /bin/mount /dev/sda1 /media/robousb
sudo python media/robousb/Robopython/usercode_old.py
exit 0
I assuming you're running Raspbian, which is pretty much Debian.
rc.local runs as root before login, so you don't need or want sudo; it may be causing an error, hence nothing happening.
User-level commands that run for any user when they log in (unlike rc.local, which runs before login) can be put into /etc/bash.bashrc. That may be more applicable to your situation, at least the second command.
Login commands for the pi user only can be put into /home/pi/.bashrc
I don't know raspberry-pi but you could try to write something into a file to see if the file is running or not. For example :
touch /tmp/test.txt
echo "$(date) => It's running" > /tmp/test.txt
If it doesn't work, I know that on some OS (fedora, rhel, centos for example), the path of that file is /etc/init.d/rc.local. It doesn't cost anything to try this path ;)
I have the exact same problem with RPi3/Jessie.
I suggest you to launch your script in the bashrc by doing
sudo emacs /home/pi/.bashrc
In my case i wrote at the EOF:
bash /home/pi/jarvis/jarvis.sh -b &
And that works well at each startup.
I have the same problem. In Raspbian forum is the solution:
Just change the first row from #!/bin/sh -e to
#!/bin/bash
Ivan X is right. You don´t need sudo command.
I want to use rvm (or rbenv/chruby for that matter) to select different ruby versions from within my Jenkins jobs.
By default, Jenkins will use /bin/sh, which on Ubuntu, is dash.
For this to change, I can add
#!/bin/bash -l
To the top of every single shell execute function everywhere. Seeing as that's a lot of annoying work, I'd like to be able to set that somewhere central.
Using the "Shell executable" configuration setting, I can get it to run bash, adding parameters like '-l' however will fail with
"/bin/bash -l" -xe /tmp/hudson5660076222778817826.sh FATAL:
command execution failed java.io.IOException: Cannot run program
"/bin/bash -l" (in directory
"/home/jenkins/jobs/workspace/rvm-test"): error=2, No such file or
directory
I tried using the rvm plugin for jenkins, but that doesn't even install on the current release version.
Any ideas? :)
You could work around by creating a wrapper around bash:
#!/bin/sh
# for ex.: /usr/local/bin/login-bash
exec /bin/bash -l "$#"
If you want to use the default ruby just use the rvm-shell, which comes with rvm.
Login as the jenkins user and type:
$ which rvm-shell
/home/jenkins/.rvm/bin/rvm-shell
to get the path of the rvm-shell.
Use this path for the "Shell executable" option.