I have a shell script in which in the first line I ask the user to input how many minutes they want the script to run for:
#!/usr/bin/ksh
echo "How long do you want the script to run for in minutes?:\c"
read scriptduration
loopcnt=0
interval=1
date2=$(date +%H:%M%S)
(( intervalsec = $interval * 1 ))
totalmin=${1:-$scriptduration}
(( loopmax = ${totalmin} * 60 ))
ofile=/home2/s499929/test.log
echo "$date2 total runtime is $totalmin minutes at 2 sec intervals"
while(( $loopmax > $loopcnt ))
do
date1=$(date +%H:%M:%S)
pid=`/usr/local/bin/lsof | grep 16752 | grep LISTEN |awk '{print $2}'` > /dev/null 2>&1
count=$(netstat -an|grep 16752|grep ESTABLISHED|wc -l| sed "s/ //g")
process=$(ps -ef | grep $pid | wc -l | sed "s/ //g")
port=$(netstat -an | grep 16752 | grep LISTEN | wc -l| sed "s/ //g")
echo "$date1 activeTCPcount:$count activePID:$pid activePIDcount=$process listen=$port" >> ${ofile}
sleep $intervalsec
(( loopcnt = loopcnt + 1 ))
done
It works great if I kick it off an input the values manually. But if I want to run this for 3 hours I need to kick off the script to run in the background.
I have tried just running ./scriptname & and I get this:
$ How long do you want the test to run for in minutes:360
ksh: 360: not found.
[2] + Stopped (SIGTTIN) ./test.sh &
And the script dies. Is this possible, any suggestions on how I can accept this one input and then run in the background?? Thanks!!!
You could do something like this:
test.sh arg1 arg2 &
Just refer to arg1 and arg2 as $1 and $2, respectively, in the bash script. ($0 is the name of the script)
So,
test.sh 360 &
will pass 360 as the first argument to the bash or ksh script which can be referred to as $1 in the script.
So the first few lines of your script would now be:
#!/usr/bin/ksh
scriptduration=$1
loopcnt=0
...
...
With bash you can start the script in the foreground and after you finished with the user input, interrupt it by hitting Ctrl-Z.
Then type
$ bg %
and the script will continue to run in the background.
Why You're Getting What You're Getting
When you run the script in the background, it can't take any user input. In fact, the program will freeze if it expects user input until its put back in the foreground. However, output has to go somewhere. Thus, the output goes to the screen (even though the program is running in the background. Thus, you see the prompt.
The prompt you see your program displaying is meaningless because you can't input at the prompt. Instead, you type in 360 and your shell is interpreting it as a command you want because you're not putting it in the program, you're putting it in the command prompt.
You want your program to be in the foreground for the input, but run in the background. You can't do both at once.
Solutions To Your Dilemma
You can have two programs. The first takes the input, and the second runs the actual program in the background.
Something like this:
#! /bin/ksh
read time?"How long in seconds do you want to run the job? "
my_actual_job.ksh $time &
In fact, you could even have a mechanism to run the job in the background if the time is over a certain limit, but otherwise run the job in the foreground.
#! /bin/ksh
readonly MAX_FOREGROUND_TIME=30
read time?"How long in seconds do you want to run the job? "
if [ $time -gt $MAX_FOREGROUND_TIME ]
then
my_actual_job.ksh $time &
else
my_actual_job.ksh $time
fi
Also remember if your job is in the background, it cannot print to the screen. You can redirect the output elsewhere, but if you don't, it'll print to the screen at inopportune times. For example, you could be in VI editing a file, and suddenly have the output appear smack in the middle of your VI session.
I believe there's an easy way to tell if your job is in the background, but I can't remember it offhand. You could find your current process ID by looking at $$, then looking at the output of jobs -p and see if that process ID is in the list. However, I'm sure someone will come up with an easy way to tell.
It is also possible that a program could throw itself into the background via the bg $$ command.
Some Hints
If you're running Kornshell, you might consider taking advantage of many of Kornshell's special features:
print: The print command is more flexible and robust than echo. Take a look at the manpage for Kornshell and see all of its features.
read: You notice that you can use the read var?"prompt" form of the read command.
readonly: Use readonly to declare constants. That way, you don't accidentally change the value of that variable later. Besides, it's good programming technique.
typeset: Take a look at typeset in the ksh manpage. The typeset command can help you declare particular variables as floating point vs. real, and can automatically do things like zero fill, right or left justify, etc.
Some things not specific to Kornshell:
The awk and sed commands can also do what grep does, so there's no reason to filter something through grep and then through awk or sed.
You can combine greps by using the -e parameter. grep foo | grep bar is the same as grep -e foo -e bar.
Hope this helps.
I've tested this with ksh and it worked. The trick is to let the script call itself with the time to wait as parameter:
if [ -z "$1" ]; then
echo "How long do you want the test to run for in minutes:\c"
read scriptduration
echo "running task in background"
$0 $scriptduration &
exit 0
else
scriptduration=$1
fi
loopcnt=0
interval=1
# ... and so on
So are you using bash or ksh? In bash, you can do this:
{ echo 360 | ./test.sh ; } &
It could work for ksh also.
Related
I want to run a background set of commands from the terminal as a script or function that waits until a certain command is entered in the foreground. I don't want to wait for just a keypress, I need the word, string or character + enter. I've tried stuff like reading the last command in bash history but I am getting nowhere.
sh -c "until [ "$(history | tail -n 1 | awk '{print $2}')" = donut ];\
do : ; done; echo 'Why did you just type donut you doughnut? \
That is not even a word let alone a command.' &" &
sh -c "until [ "$(history | tail -n 1 | awk '{print $2}')" = ":" ]; do sleep 1 ; done; echo gotcha" &
Ideally I dont want any output like 'command not found' if the word is not a command and I don't want the word to execute if it is a command. That's why I tried the : command it is both and neither.
I also tried looking for ":\n" in various ways. I'm probably barking up the wrong tree here with the history command and suspect something to do with cat /dev/tty is required.
Any thoughts?
It depends on your operating system. For Mac, you need to create an executable file in your /usr/local/bin or /usr/bin. That is the directory where all the bash commands are executing from. You'll find things like "time", or "python", or "whoami" stored there. The actual executable can be anything, but you're trying to keep things simple, use a python script with a shebang. Then make the file into a unix executable. The actual program can be very simple, just force quit the process id of the "background process". I'm not sure how Windows organizes their "bin", but it's probably something similar.
Take the following example:
ls -l | grep -i readme | ./myscript.sh
What I am trying to do is get ls -l | grep -i readme as a string variable in myscript.sh. So essentially I am trying to get the whole command before the last pipe to use inside myscript.sh.
Is this possible?
No, it's not possible.
At the OS level, pipelines are implemented with the mkfifo(), dup2(), fork() and execve() syscalls. This doesn't provide a way to tell a program what the commands connected to its stdin are. Indeed, there's not guaranteed to be a string representing a pipeline of programs being used to generate stdin at all, even if your stdin really is a FIFO connected to another program's stdout; it could be that that pipeline was generated by programs calling execve() and friends directly.
The best available workaround is to invert your process flow.
It's not what you asked for, but it's what you can get.
#!/usr/bin/env bash
printf -v cmd_str '%q ' "$#" # generate a shell command representing our arguments
while IFS= read -r line; do
printf 'Output from %s: %s\n' "$cmd_str" "$line"
done < <("$#") # actually run those arguments as a command, and read from it
...and then have your script start the things it reads input from, rather than receiving them on stdin.
...thereafter, ./yourscript ls -l, or ./yourscript sh -c 'ls -l | grep -i readme'. (Of course, never use this except as an example; see ParsingLs).
It can't be done generally, but using the history command in bash it can maybe sort of be done, provided certain conditions are met:
history has to be turned on.
Only one shell has been running, or accepting new commands, (or failing that, running myscript.sh), since the start of myscript.sh.
Since command lines with leading spaces are, by default, not saved to the history, the invoking command for myscript.sh must have no leading spaces; or that default must be changed -- see Get bash history to remember only the commands run with space prefixed.
The invoking command needs to end with a &, because without it the new command line wouldn't be added to the history until after myscript.sh was completed.
The script needs to be a bash script, (it won't work with /bin/dash), and the calling shell needs a little prep work. Sometime before the script is run first do:
shopt -s histappend
PROMPT_COMMAND="history -a; history -n"
...this makes the bash history heritable. (Code swiped from unutbu's answer to a related question.)
Then myscript.sh might go:
#!/bin/bash
history -w
printf 'calling command was: %s\n' \
"$(history | rev |
grep "$0" ~/.bash_history | tail -1)"
Test run:
echo googa | ./myscript.sh &
Output, (minus the "&" associated cruft):
calling command was: echo googa | ./myscript.sh &
The cruft can be halved by changing "&" to "& fg", but the resulting output won't include the "fg" suffix.
I think you should pass it as one string parameter like this
./myscript.sh "$(ls -l | grep -i readme)"
I think that it is possible, have a look at this example:
#!/bin/bash
result=""
while read line; do
result=$result"${line}"
done
echo $result
Now run this script using a pipe, for example:
ls -l /etc | ./script.sh
I hope that will be helpful for you :)
I am trying to create an init script for a program in bash. (rhel6)
It checks for the processes first. If processes are found it will echo that program is already online and if not it'll move on to to start the program as a certain user by using launch script. After doing that it should tail the log file of the program and check for a string of words together. If the words are found it should kill tail and echo that program is online.
Here's the start segment.
prog=someProg
user=someUser
threadCount=$(ps -ef | grep $prog |grep -v 'grep' |awk '{ print $2 }'| wc -l)
startb() {
if [ "$threadCount" -eq 2 ]; then
echo "$prog already online."
else
echo "Bringing $prog online."
su $user -c "/path/to/start/script.sh"
tail -f /path/to/$prog/log/file |
while IFS=$'\n' read line
do
if [[ $line == *started\ up\ and\ registered\ in* ]]; then
pkill tail
echo "$prog now online."
fi
done
fi
}
My problems:
The variable $prog doesn't get picked in $threadcount no
matter how I try. (with single and double quotes)
The logic about tailing the log file works randomly. Some times it
just works perfect. It tails and waits till the string is found
before echoing program is online and at times it just starts script
and then echoes that program is online without the tail or wait.
It's unpredictable. I implemented the same logic in stop segment too to monitor log and then echo but even that works the same way as start. Just random.
I am sure that this might look dumb and broken. This is made by picking pieces here and there with my beginner bash skills.
Thanks in advance for suggestions and help.
I can't reproduce the error you are experiencing with the "grep $prog"...sorry.
But for the other part.
I will assume that the script starting your program, the line with su, is starting something in background and that the script end by itself. If not, your example will wait indefinitely.
Could be a personal preference, but when I'm using something like tail to verify lines, I use a named pipe (mkfifo).
That would give something like :
# Getting the tail in background
tail -f /path/to/$prog/log/file > some_fifo &
# Getting the tail PID
tailPID=$!
while read line; do #You don't need to modify/use IFS here.
if [[ $line == *started\ up\ and\ registered\ in* ]]; then
kill -15 $tailPID #since you know the PID you won't kill another tail
echo "$prog now online."
break # don't like the possibility, even remote, of an infinite loop :)
fi
done < some_fifo #reading from the named pipe
Hope it can help you
I need to write the time taken to execute this command in a txt file:
time ./program.exe
How can I do in bash script?
I try with >> time.txt but that doesn't work (the output does not go to file and does go to the screen).
Getting time in bash to write to a file is hard work. It is a bash built-in command. (On Mac OS X, there's an external command, /usr/bin/time, that does a similar job but with a different output format and less recalcitrance.)
You need to use:
(time ./program.exe) 2> time.txt
It writes to standard error (hence the 2> notation). However, if you don't use the sub-shell (the parentheses), it doesn't work; the output still comes to the screen.
Alternatively, and without a sub-shell, you can use:
{ time ./program.exe; } 2> time.txt
Note the space after the open brace and the semi-colon; both are necessary on a single line. The braces must appear where a command could appear, and must be standalone symbols. (If you struggle hard enough, you'll come up with ...;}|something or ...;}2>&1. Both of these identify the brace as a standalone symbol, though. If you try ...;}xyz, the shell will (probably) fail to find a command called }xyz, though.)
I need to run more command in more terminal. If I do this:
xterm -xrm '*hold: true' -e (time ./Program.exe) >> time.exe & sleep 2
it doesn't work and tells me Syntax error: "(" unexpected. How do I fix this?
You would need to do something like:
xterm -xrm '*hold: true' -e sh -c "(time ./Program.exe) 2> time.txt & sleep 2"
The key change is to run the shell with the script coming from the argument to the -c option; you can replace sh with /bin/bash or an equivalent name. That should get around any 'Syntax error' issues. I'm not quite sure what triggers that error, though, so there may be a simpler and better way to deal with it. It's also conceivable that xterm's -e option only takes a single string argument, in which case, I suppose you'd use:
xterm -xrm '*hold: true' -e 'sh -c "(time ./Program.exe) 2> time.txt & sleep 2"'
You can manual bash xterm as well as I can.
I'm not sure why you run the timed program in background mode, but that's your problem, not mine. Similarly, the sleep 2 is not obviously necessary if the hold: true keeps the terminal open.
time_elapsed=(time sh -c "./program.exe") 2>&1 | grep "real" | awk '{print $(NF)}'
echo time_elapsed > file.txt
This command should give you the exact time consumed in bash in a desired file..
You can also redirect this to a file usng 2 > file.txt as explained in another reply.
It's not easy to redirect the output of the bash builtin time.
One solution is to use the external time program:
/bin/time --append -o time.txt ./program.exe
(on most systems it's a GNU program, so use info time rather than man to get its documentation).
Just enclose the command to time in a { .. }:
{ time ./program.exe; } 2>&1
Of course, the output of builtin time goes to stderr, thus the needed redirection 2>&1.
Then, it may appear to be tricky to capture the output, let's use a second { .. } to read the command more easily, this works:
{ { time ./program.exe; } 2>&1; } >> time.txt # This works.
However, the correct construct should simply have the capture reversed, as this:
{ time ./program.exe; } >> time.txt 2>&1; # Correct.
To close any possible output from the command, redirect it's output to /dev/null, as this:
{ time ./program.exe >/dev/null 2>&1; } >> time.txt 2>&1 # Better.
And, as now there is only output on stderr, we could simply capture just it:
{ time ./program.exe >/dev/null 2>&1; } 2>> time.txt # Best.
The output from ./program should be redirected, or it may well end inside time.txt.
I'm currently using the following to capture everything that goes to the terminal and throw it into a log file
exec 4<&1 5<&2 1>&2>&>(tee -a $LOG_FILE)
however, I don't want color escape codes/clutter going into the log file. so i have something like this that sorta works
exec 4<&1 5<&2 1>&2>&>(
while read -u 0; do
#to terminal
echo "$REPLY"
#to log file (color removed)
echo "$REPLY" | sed -r 's/\x1B\[([0-9]{1,2}(;[0-9]{1,2})?)?[m|K]//g' >> $LOG_FILE
done
unset REPLY #tidy
)
except read waits for carriage return which isn't ideal for some portions of the script (e.g. echo -n "..." or printf without \n).
Follow-up to Jonathan Leffler's answer:
Given the example script test.sh:
#!/bin/bash
LOG_FILE="./test.log"
echo -n >$LOG_FILE
exec 4<&1 5<&2 1>&2>&>(tee -a >(sed -r 's/\x1B\[([0-9]{1,2}(;[0-9]{1,2})?)?[m|K]//g' > $LOG_FILE))
##### ##### #####
# Main
echo "starting execution"
printf "\n\n"
echo "color test:"
echo -e "\033[0;31mhello \033[0;32mworld\033[0m!"
printf "\n\n"
echo -e "\033[0;36mEnvironment:\033[0m\n foo: cat\n bar: dog\n your wife: hot\n fix: A/C"
echo -n "Before we get started. Is the above information correct? "
read YES
echo -e "\n[READ] $YES" >> $LOG_FILE
YES=$(echo "$YES" | sed 's/^\s*//;s/\s*$//')
test ! "$(echo "$YES" | grep -iE '^y(es)?$')" && echo -e "\nExiting... :(" && exit
printf "\n\n"
#...some hundreds of lines of code later...
echo "Done!"
##### ##### #####
# End
exec 1<&4 4>&- 2<&5 5>&-
echo "Log File: $LOG_FILE"
The output to the terminal is as expected and there is no color escape codes/clutter in the log file as desired. However upon examining test.log, I do not see the [READ] ... (see line 21 of test.sh).
The log file [of my actual bash script] contains the line Log File: ... at the end of it even after closing the 4 and 5 fds. I was able to resolve the issue by putting a sleep 1 before the second exec - I assume there's a race condition or fd shenanigans to blame for it. Unfortunately for you guys, I am not able to reproduce this issue with test.sh but I'd be interested in any speculation anyone may have.
Consider using the pee program discussed in Is it possible to distribute stdin over parallel processes. It would allow you to send the log data through your sed script, while continuing to send the colours to the actual output.
One major advantage of this is that it would remove the 'execute sed once per line of log output'; that is really diabolical for performance (in terms of number of processes executed, if nothing else).
I know it's not a perfect solution, but cat -v will make non visible chars like \x1B to be converted into visible form like ^[[1;34m. The output will be messy, but it will be ascii text at least.
I use to do stuff like this by setting TERM=dumb before running my command. That pretty much removed any control characters except for tab, CR, and LF. I have no idea if this works for your situation, but it's worth a try. The problem is that you won't see color encodings on your terminal either since it's a dumb terminal.
You can also try either vis or cat (especially the -v parameter) and see if these do something for you. You'd simply put them in your pipeline like this:
exec 4<&1 5<&2 1>&2>&>(tee -a | cat -v | $LOG_FILE)
By the way, almost all terminal programs have an option to capture the input, and most clean it up for you. What platform are you on, and what type of terminal program are you using?
You could attempt to use the -n option for read. It reads in n characters instead of waiting for a new line. You could set it to one. This would increase the number of iteration the code runs, but it would not wait for newlines.
From the man:
-n NCHARS read returns after reading NCHARS characters rather than waiting for a complete line of input.
Note: I have not tested this
You can use ANSIFilter to strip or transform console output with ANSI escape sequences.
See http://www.andre-simon.de/zip/download.html#ansifilter
Might not screen -L or the script commands be viable options instead of this exec loop?