Monitoring life time of a process - shell

Question

I have a python script called hdsr_writer.py. I can launch this script in shell by calling
"python hdsr_writer.py 1234"
where 1234 is a parameter.
I made a shell script to increase the number and execute the python script with the number every 1 second
for param from 1 to 100000
python hdsr_writer.py $param &
sleep (1)
Usually, the python script executes its task within 0.5 second. However, there are times at which the python script gets stuck and resides in the system for longer than 30 seconds. I don't want that. So I would like to monitor life time of each python process executed. If it has stayed for longer than 2 second it would be killed and re-executed 2 times at most.
Note: I would like do this in the shell script not python script because I could not change the python script.
Update: More explainations about my question
Please note that: launching a new python process and monitoring python processes are independent jobs. Launching job doesn't care how many python processes are running and how "old" they are, just calls "python hdsr_writer.py $param &" every 1 second after increasing param. On the other hand, monitoring job periodically checks life time of all hdsr_writer python processes. If one has resided more than 2 second in memory, kills it, and re-runs it at most of 2 times.

Answer 1

Not so short answer
#/bin/bash
param=1
while [[ $param -lt 100000 ]]; do
echo "param=$param"
chances=3
while [[ $chances -gt 0 ]]; do
python tst.py $param &
sleep 2
if [[ "$(jobs | grep 'Running')" == "" ]]; then
chances=0
else
kill -9 $(jobs -l | awk '{print $2}')
chances=$(($chances-1))
if [[ $chances -gt 0 ]]; then
echo "one more chance for parameter $param"
fi
fi
done
param=$(($param+1))
done
UPD
This is another answer as requested by OP.
Here is still 2 scripts in one. But they can be spitted in two files.
Please pay attention that $() & is used to run sub-shells in background
#!/bin/bash
# Script launcher
pscript='rand.py'
for param in {1..10}
do
# start background sub-shell, where python with $param is started
echo $(
left=3
error_on_exit=1
# go if any chances left and previous run exits not with code 0
while [[ ( ( $left -gt 0 ) && ( $error_on_exit -ne 0 ) ) ]]; do
left=$(($left-1))
echo "param=$param; chances left $left "
# run python and grab python exit code (=0 if ok)
python $pscript $param
error_on_exit=$?
done
) &
done
# Script controller
# just kills python processes older than 2 seconds
# exits after no python left
# $(...) & can be removed if this code goes to separate script
$(while [[ $(ps | grep -v 'grep' | grep -c python ) != "0" ]]
do
sleep 0.5
killall -9 -q --older-than 2s python
done) &

Answer 2

Use a combination of sleep and nohup commands. After sleep time use kill to finish the execution of python script. You can check if the process is running with ps command.
#!/usr/bin/ksh
for param from {1..100000}
nohup python hdsr_writer.py $param &
pid=$!
sleep(2)
if [ ps -p $pid ]
then
kill -9 $pid
fi
done
Re-answer:
I'd use two scripts, the first one (script1.ksh):
#!/usr/bin/ksh
for param from {1..1000000}
nohup script2.sh $param &
done
And the second (script2.ksh):
#!/usr/bin/ksh
for i from {1..3}
python hsdr_write.py $1 &
pid=$!
sleep(2)
if [ ps -p $pid ]
then
kill -9 $pid
else
echo 'Finalizado'$1 >> log.txt
return
fi
done
The first script will launch all yours processes one after the other. The second one will check his own python process.