I'm wondering if there a way of capturing a list of the processes executed on a non-interactive shell?
Basically I have a script which calls some variables from other sources and I want to see what the values of said variables are. However, the script executes and finishes very quickly so I can't capture the values using ps.
Is there a way to log processes and what arguments were used?
TIA
Huskie
EDIT:
I'm using Solaris in this instance. I even thought about about having a quick looping script to capture the values being passed - but this doesn't seem very accurate and I'm sure executions aren't always being captured.
I tried this:
#!/bin/ksh
while [ true ]
do
ps -ef | grep $SCRIPT_NAME |egrep -v 'shl|lis|grep' >> grep_out.txt
done
I'd use sleep but I can't specify any precision as all my sleep executables want an integer value rather than any fractional value.
On Solaris:
truss -s!all -daDf -t exec yourCommand 2>&1 | grep -v ENOENT
On AIX and possibly other System V based OSes:
truss -s!all -daDf -t execve yourCommand 2>&1 | grep -v ENOENT
On Linux and other OSes supporting strace, you can use this command:
strace -ff -etrace=execve yourCommand 2>&1 >/dev/tty | grep -v ENOENT
In case the command you want to trace is already running, you can replace yourCommand by -p pid with pid being the process to be traced process id.
EDIT:
Here is a way to trace your running script(s) under Solaris:
for pid in $(pgrep -f $SCRIPT_NAME); do
truss -s!all -daDf -t exec -p $pid 2>&1 | grep -v ENOENT > log.$pid.out &
done
Note that with Solaris, you might also use dtrace to get the same (and more).
Most shells can be invoked in debug mode, where each statement being executed is printed to stdout (or stderr) after variable substitution and expansion.
For Bourne like shells (sh, bash), debug is enabled with the -x option (as in bash -x myscript) or using the set -x statement within the script itself.
However, debugging only works for the 'current' script. If the script calls other scripts, these other scripts will not execute in debug mode. Furthermore, the code inside functions may not be executed in debug mode either - depends on the specific shell - although you can use set -x within a function to enable debug explicitly.
A very much more verbose (at least by default) option is to use something like strace for this.
strace -f -o trace.out script.sh
will give you huge amounts of information about what the script is doing. For your specific usage you will likely want to limit the output a bit with the -e trace=.... option to control which system calls are traced.
Use truss instead of strace on Solaris. Use dtruss on OS X (I believe). With appropriate command line argument changes as well.
Related
I want to write a script that checks periodically if a specific executable has a live process, something like this:
psping [-c ###] [-t ###] [-u user-name] exe-name
-c - limit amount of pings, Default is infinite
-t - define alternative timeout in seconds, Default is 1 sec
-u - define user to check process for. The default is ANY user.
For example, psping java will list all processes that are currently invoked by the java command.
The main goal is to count and echo the number of live processes for a user, whose executable file is exe-name, java in the above example.
I wrote a function:
perform_ping(){
ps aux | grep "${EXE_NAME}" | awk '{print $2}' | while read PID
do
echo $PID # -> This will echo the correct PID
# How to find if this PID was executed by ${EXE_NAME}
done
fi
sleep 1
}
I'm having a hard time figuring out how to check if a specific executable file has a live process.
To list all processes that opens a file, we can use the lsof command. Because an executable must be opened in order to be run, we may just use lsof for this purpose.
The next problem is that when we run a java file, we simply type java some_file, and if we issue lsof java it will coldly says that lsof: status error on java: No such file or directory because the java is actually /usr/bin/java.
To convert from java to /usr/bin/java we can use which java, so the command would be:
lsof $(which $EXE_FILE)
The output may looks like this:
lsof: WARNING: can't stat() tracefs file system /sys/kernel/debug/tracing
Output information may be incomplete.
COMMAND PID USER FD TYPE DEVICE SIZE/OFF NODE NAME
python3 26969 user txt REG 8,1 4526456 15409 /usr/bin/python3.6
In this case I searched python3 as lsof $(which python3). It will report the PID in the second field. But when there's another user that invokes python3 too, lsof will issue the warning on stderr like the first two lines because it cannot read other users info. Therefore, we modify the command as:
lsof $(which python3) 2> /dev/null
to suppress the warning. Then we're almost there:
lsof $(which python3) 2> /dev/null | awk 'NR > 1 { print $2 }'
Then you can use read to catch the PID.
Edit: how to list all processes for all users?
By default lsof doesn't read process for a specific file, but after further reading man lsof I found that there are options that meet your needs.
-a causes list selection options to be ANDed.
-c c selects the listing of files for processes executing the command that begins with the characters of c. Multiple commands may be specified, using multiple -c options.
-u s selects the listing of files for the user whose login names or user ID numbers are in the comma-separated set s.
Therefore, you can use
lsof -c java
to list all commands that are run by java. And to see a specific user, add -u option as
lsof -a -c java -u user
-a is needed for the AND operation. If you run this command you will see multiple entry for a process, to unique them, run
lsof -c java 2> /dev/null | sed 1d | sort -uk2,2
Also please notice that users may run their own java in their path and therefore you have to decide which one to monitor: java or /usr/bin/java.
Here is a test:
$ bash -c "pgrep -f novalidname"
$ sh -c "pgrep -f novalidname"
11202
Why is pgrep giving output when run from sh? (As far as I can see, there are no processes on my computer that is named novalidname)
It's probably a timing issue and pgrep finds itself, as you're issuing it with -f and novalidname is present in the command line. Try with -l to confirm.
The actual explanation:
Regardless of flags, pgrep never returns its own PID.
If you execute bash -c with a simple command, then bash will exec the command rather than creating a redundant subshell to execute it in. Consequently, bash -c "pgrep -f blah" will replace the bash process with a pgrep process. If that pgrep process is the only process whose command line includes blah, then pgrep will not display any PIDs (as per 1).
dash does not perform the above optimization. (zsh and ksh do.) So if on your system, sh is implemented with dash, then sh -c "pgrep -f blah" will result in two processes being executed -- the sh process and the pgrep child -- both of which contain blah in their command lines. pgrep will not report itself, but it will report its parent.
That's one thing (finding itself because of delay) see also:
$ ps ax | grep novalidname
Here it usually shows as well. (on Ubuntu does for me. (under bash)
The other thing is what is /bin/sh bound to?
On most Linux distros /bin/sh is a soft link to default shell which is usually actually bash, but can be any other shell.
The time difference that causes grep/pgrep to show itself may be introduced by finding a soft link location (hm, odd) or some other shell is bound to /bin/sh which executes slightly different than bash, thus causing the delay needed for process to show in pgrep.
Also, bash will firstly try to source ~/.bashrc and load its history, while /bin/sh will do what will do. In .bashrc can be pgrep defined as alias in another way which may also affect the difference.
To see where /bin/sh points to do:
$ readlink -e /bin/sh
Or just run sh to see what will show up. :D
I want an advanced shell or command line in Unix which has the following features:
output to err and out are in different colours.
I should be able to highlight (or find) keywords in the output of the executing command.
indicator in the OS task-bar/title as a command is running or completed.
I am looking at an advanced shell that boosts productivity. Is there any alternative?
Re: output to err and out are in different colours ... can be done in Bash.
# colourize stderr in current shell
# note: use sed in line-buffering mode
(
exec 2> >(sed -l -e $'s/.*/\033[31m&\033[m/')
ls -ld / xxxxx
)
# colourize stderr & stdout in current shell
(
exec 1> >(sed -l -e $'s/.*/\033[32m&\033[m/') 2> >(sed -l -e $'s/.*/\033[31m&\033[m/')
ls -ld / xxxxx
)
That isn't a trivial proposition.
There are shells that work with the terminal to echo the currently executing command in the title bar, such as bash on MacOS X.
The commands are autonomous and do not, in general, colour-code their output. So, to get colour-coded output, the shell will have to capture the error outputs of the commands it runs, and arrange to display that information appropriately colour-coded.
Searching the output requires the terminal program to keep the output it displays in a searchable form, and some program (probably the terminal program or possibly the shell) will have to respond to searching operations.
Emacs allows you to run interactive shells such as bash or zsh.
rc works similarly when run in Plan9 -- I'm not sure about its Unix ports.
I have this user crontab (accessed via the command crontab -e):
# m h dom mon dow command
*/3 * * * * sh /home/FRAPS/Desktop/cronCheck.sh
The script cronCheck.sh looks like that:
#!/bin/sh
SERVICE='Script'
if ps ax | grep -v grep | grep -i "$SERVICE" > /dev/null
then
echo "######## $SERVICE service running, everything is fine ##################\n" >> CronReport.txt
else
echo "$SERVICE is not running. Launching it now\n" >> CronReport.txt
perl Script.pl
fi
When I launch the script (cronCheck.sh) from its own directory, it works like a charm, but when cron launches it, it always "# $SERVICE service running, everything is fine ###"
despite 'Script' is not running.
Thanks,
Here's an even better way to write that conditional:
services=$(ps -e -o comm | grep -cFi "$SERVICE")
case "$services" in
(0)
# restart service
;;
(1)
# everything is fine
;;
(*)
# more than one copy is running
;;
esac
By using ps -e -o comm you avoid having to do the silly grep -v grep thing, because only the actual process name appears in the ps output, not the arguments. And grep -cFi counts up the matches and gives you a number, so you don't have to deal with the exit status of a pipeline.
Also, as other posters have implied, you should lead off this script by setting the PATH variable.
PATH=/bin:/usr/bin:/sbin:/usr/sbin
export PATH
You might or might not want to put /usr/local/bin at the beginning of that list, depending on your system. Don't do it if you don't need anything from there.
Final piece of advice: When writing scripts that will execute without user supervision (such as cron jobs), it's a good idea to put set -e at the beginning. That makes them exit unsuccessfully if any command fails.
You need to put the grep -v grep after the grep -i "$SERVICE". The way you have it now it's guaranteed to be true.
Checking the return status of a pipe like that could be problematic. You should either check the $PIPESTATUS array, or you can pipe the final grep into wc -l to count the number of lines.
cron typically does not set up a lot of the environment like a user account does. You may need to modify your script to get things setup properly.
Cron jobs don't get the same environment settings that you get at a shell prompt - those are generally set up by your shell on login - so you want to use absolute rather than relative paths throughout. (i.e. don't assume the PATH environment variable will exist or be set up the same as it is for you at a shell prompt, and don't assume the script will run with PWD set to your home directory, etc.) So:
in your crontab entry replace sh with /bin/sh (or remove it if cronCheck.sh is executable, the shebang line will do).
in cronCheck.sh add paths to the log file and the perl script.
cronCheck.sh should end up looking something like:
#!/bin/sh
SERVICE='Script'
if ps ax | grep -v grep | grep -i "$SERVICE" > /dev/null
then
echo "######## $SERVICE service running, everything is fine ##################\n" >> CronReport.txt
else
# Specify absolute path to a log file that's writeable for the user the
# cron runs as (probably you). Example: /tmp/CronReport.txt
echo "$SERVICE is not running. Launching it now\n" >> /tmp/CronReport.txt
# Specify absolute path to both perl and the script. Example: /usr/bin/perl
# and /home/FRAPS/scripts/Script.pl
/usr/bin/perl /home/FRAPS/scripts/Script.pl
fi
(Again you can get rid of the /usr/bin/perl bit if Script.pl is executable and has the path to the right perl in the shebang line.)
Is there any way to set the process name of a shell script? This is needed for killing this script with the killall command.
Here's a way to do it, it is a hack/workaround but it works pretty good. Feel free to tweak it to your needs, it certainly needs some checks on the symbolic link creation or using a tmp folder to avoid possible race conditions (if they are problematic in your case).
Demonstration
wrapper
#!/bin/bash
script="./dummy"
newname="./killme"
rm -iv "$newname"
ln -s "$script" "$newname"
exec "$newname" "$#"
dummy
#!/bin/bash
echo "I am $0"
echo "my params: $#"
ps aux | grep bash
echo "sleeping 10s... Kill me!"
sleep 10
Test it using:
chmod +x dummy wrapper
./wrapper some params
In another terminal, kill it using:
killall killme
Notes
Make sure you can write in your current folder (current working directory).
If your current command is:
/path/to/file -q --params somefile1 somefile2
Set the script variable in wrapper to /path/to/file (instead of ./dummy) and call wrapper like this:
./wrapper -q --params somefile1 somefile2
You can use the kill command on a PID so what you can do is run something in the background, get its ID and kill it
PID of last job run in background can be obtained using $!.
echo test & echo $!
You cannot do this reliably and portably, as far as I know. On some flavors of Unix, changing what's in argv[0] will do the job. I don't believe there's a way to do that in most shells, though.
Here are some references on the topic.
Howto change a UNIX process and child process name by modifying argv0
Is there a way to change the effective process name in Python?
This is an extremely old post. Pretty sure the original poster got his/her answer long ago. But for newcomers, thought I'd explain my own experience (after playing with bash for a half hour). If you start a script by script name w/ something like:
./script.sh
the process name listed by ps will be "bash" (on my system). However if you start a script by calling bash directly:
/bin/bash script.sh
/bin/sh script.sh
bash script.sh
you will end up with a process name that contains the name of the script. e.g.:
/bin/bash script.sh
results in a process name of the same name. This can be used to mark pids with a specific script name. And, this can be useful to (for example) use the kill command to stop all processes (by pid) that have a process name containing said script name.
You can all use the -f flag to pgrep/pkill which will search the entire command line rather than just the process name. E.g.
./script &
pkill -f script
Include
#![path to shell]
Example for path to shell -
/usr/bin/bash
/bin/bash
/bin/sh
Full example
#!/usr/bin/bash
On Linux at least, killall dvb works even though dvb is a shell script labelled with #!. The only trick is to make the script executable and invoke it by name, e.g.,
dvb watch abc write game7 from 9pm for 3:30
Running ps shows a process named
/usr/bin/lua5.1 dvb watch ...
but killall dvb takes it down.
%1, %2... also do an adequate job:
#!/bin/bash
# set -ex
sleep 101 &
FIRSTPID=$!
sleep 102 &
SECONDPID=$!
echo $(ps ax|grep "^\(${FIRSTPID}\|${SECONDPID}\) ")
kill %2
echo $(ps ax|grep "^\(${FIRSTPID}\|${SECONDPID}\) ")
sleep 1
kill %1
echo $(ps ax|grep "^\(${FIRSTPID}\|${SECONDPID}\) ")
I put these two lines at the start of my scripts so I do not have to retype the script name each time I revise the script. It won't take $0 of you put it after the first shebang. Maybe someone who actually knows can correct me but I believe this is because the script hasn't started until the second line so $0 doesn't exist until then:
#!/bin/bash
#!/bin/bash ./$0
This should do it.
My solution uses a trivial python script, and the setproctitle package. For what it's worth:
#!/usr/bin/env python3
from sys import argv
from setproctitle import setproctitle
from subprocess import run
setproctitle(argv[1])
run(argv[2:])
Call it e.g. run-with-title and stick it in your path somewhere. Then use via
run-with-title <desired-title> <script-name> [<arg>...]
Run bash script with explicit call to bash (not just like ./test.sh). Process name will contain script in this case and can be found by script name. Or by explicit call to bash with full path as
suggested in display_name_11011's answer:
bash test.sh # explicit bash mentioning
/bin/bash test.sh # or with full path to bash
ps aux | grep test.sh | grep -v grep # searching PID by script name
If the first line in script (test.sh) explicitly specifies interpreter:
#!/bin/bash
echo 'test script'
then it can be called without explicit bash mentioning to create process with name '/bin/bash test.sh':
./test.sh
ps aux | grep test.sh | grep -v grep
Also as dirty workaround it is possible to copy and use bash with custom name:
sudo cp /usr/bin/bash /usr/bin/bash_with_other_name
/usr/bin/bash_with_other_name test.sh
ps aux | grep bash_with_other_name | grep -v grep
Erm... unless I'm misunderstanding the question, the name of a shell script is whatever you've named the file. If your script is named foo then killall foo will kill it.
We won't be able to find pid of the shell script using "ps -ef | grep {scriptName}" unless the name of script is overridden using shebang. Although all the running shell scripts come in response of "ps -ef | grep bash". But this will become trickier to identify the running process as there will be multiple bash processing running simultaneously.
So a better approach is to give an appropriate name to the shell script.
Edit the shell script file and use shebang (the very first line) to name the process e.g. #!/bin/bash /scriptName.sh
In this way we would be able to grep the process id of scriptName using
"ps -ef | grep {scriptName}"