Specifics:
I'm trying to build a bash script which needs to do a couple of things.
Firstly, it needs to run a third party script that I cannot manipulate. This script will build a project and then start a node server which outputs data to the terminal continually. This process needs to continue indefinitely so I can't have any exit codes.
Secondly, I need to wait for a specific line of output from the first script, namely 'Started your app.'.
Once that line has been output to the terminal, I need to launch a separate set of commands, either from another subscript or from an if or while block, which will change a few lines of code in the project that was built by the first script to resolve some dependencies for a later step.
So, how can I capture the output of the first subscript and use that to run another set of commands when a particular line is output to the terminal, all while allowing the first script to run in the terminal, and without using timers and without creating a huge file from the output of subscript1 as it will run indefinitely?
Pseudo-code:
#!/usr/bin/env bash
# This script needs to stay running & will output to the terminal (at some point)
# a string that we need to wait/watch for to launch subscript2
sh subscript1
# This can't run until subscript1 has output a particular string to the terminal
# This could be another script, or an if or while block
sh subscript2
I have been beating my head against my desk for hours trying to get this to work. Any help would be appreciated!
I think this is a bad idea — much better to have subscript1 changed to be automation-friendly — but in theory you can write:
sh subscript1 \
| {
while IFS= read -r line ; do
printf '%s\n' "$line"
if [[ "$line" = 'Started your app.' ]] ; then
sh subscript2 &
break
fi
done
cat
}
Related
I'm attempting to add to my ~/.zshrc file a command to record the command line inputs and outputs. I have the script (https://www.tecmint.com/record-and-replay-linux-terminal-session-commands-using-script/) command required running but I'm having an issue with it seeming to infinitely attempt to record my session. I believe this is due to the fact that when you run the script command it starts a new bash session which in turn runs the ~/.zshrc which then tries to record the session which restarts the session (etc. etc.). This causes it to get into an infinite loop of attempting to record the session.
Attempt 1
Relevant piece of my ~/.zshrc
# RECORD COMMAND INPUT AND OUTPUT
LOG_PATH=/var/log/terminal/$(date +'%Y%m%d')
LOG_FILE=${LOG_PATH}/$(date +'%H%M%S').log
mkdir -p ${LOG_PATH}
script ${LOG_FILE}
This results in a session which constantly prints the following:
Script started, output file is /var/log/terminal/20200109/141849.log
Script started, output file is /var/log/terminal/20200109/141850.log
Script started, output file is /var/log/terminal/20200109/141851.log
Script started, output file is /var/log/terminal/20200109/141852.log
Script started, output file is /var/log/terminal/20200109/141853.log
Script started, output file is /var/log/terminal/20200109/141854.log
Script started, output file is /var/log/terminal/20200109/141855.log
... repeat infinitely ...
Attempt 2
One attempt at working around this was to check if a file already exists and then go on with recording (which works somewhat but sometimes it creates 2 files or sometimes the recording doesn't start say if I open 2 command sessions in rapid succession).
Upgraded script
# RECORD COMMAND INPUT AND OUTPUT
# Check if the recording of a file for the given time has already started
# as it seems that once you start the script recording it re-starts the session
# which in turn re-runs this file which attempts to script again running into an infinite loop
LOG_PATH=/var/log/terminal/$(date +'%Y%m%d')
LOG_FILE=${LOG_PATH}/$(date +'%H%M%S').log
if [[ ! -f $LOG_FILE ]]; then
mkdir -p ${LOG_PATH}
script ${LOG_FILE}
fi
Again this either yields no recording (in the case of opening 2 sessions quickly) or it results in the recording being done twice
Script started, output file is /var/log/terminal/20200109/141903.log
Script started, output file is /var/log/terminal/20200109/141904.log
Attempt 3
Another attempt was to check the bash history and see if the last command contained the word script. Problem with this attempt was that the bash session doesn't seem to have history when starting and hence gave the following error (and then infinitely tried to start the recording session like the first attempt):
# RECORD COMMAND INPUT AND OUTPUT
# Check what the last command was to make sure that it wasn't the script starting
# as it seems that once you start the script recording it re-starts the session
# which in turn re-runs this file which attempts to script again running into an infinite loop
LAST_HISTORY=$(history -1)
if [[ "$LAST_HISTORY" != *"script"* ]]; then
LOG_PATH=/var/log/terminal/$(date +'%Y%m%d')
LOG_FILE=${LOG_PATH}/$(date +'%H%M%S').log
mkdir -p ${LOG_PATH}
script ${LOG_FILE}
fi
Output
Last login: Thu Jan 9 14:27:30 on ttys009
Script started, output file is /var/log/terminal/20200109/142754.log
Script started, output file is /var/log/terminal/20200109/142755.log
omz_history:fc:13: no such event: 0
omz_history:fc:13: no events in that range
Script started, output file is /var/log/terminal/20200109/142755.log
omz_history:fc:13: no such event: 0
omz_history:fc:13: no events in that range
Script started, output file is /var/log/terminal/20200109/142755.log
Script started, output file is /var/log/terminal/20200109/142756.log
^C% danielcarmo#Daniels-MacBook-Pro-2 git %
Any suggestions or thoughts on this would be much appreciated.
Instead of trying to figure out whether or not script is running, simply leave a breadcrumb for yourself:
if [ -z "$BEING_LOGGED" ]
then
export BEING_LOGGED="yes"
LOG_PATH="/var/log/terminal/$(date +'%Y%m%d')"
LOG_FILE="${LOG_PATH}/$(date +'%H%M%S').log"
mkdir -p "${LOG_PATH}"
exec script "${LOG_FILE}"
fi
The first time the file is sourced, BEING_LOGGED will be unset. When script is invoked and the file is sourced again, it'll be set, and you can skip logging.
script adds the variable SCRIPT to the environment of the command it runs. You can check for that to decide whether script needs to run.
# RECORD COMMAND INPUT AND OUTPUT
log_path=/var/log/terminal/$(date +'%Y%m%d')
log_file=${log_path}/$(date +'%H%M%S').log
mkdir -p "${log_path}"
[ -z "$SCRIPT" ] && script "${log_file}"
The value of SCRIPT is the name of the file being logged to.
I am quite new to bash (barely any experience at all) and I need some help with a bash script.
I am using docker-compose to create multiple containers - for this example let's say 2 containers. The 2nd container will execute a bash command, but before that, I need to check that the 1st container is operational and fully configured. Instead of using a sleep command I want to create a bash script that will be located in the 2nd container and once executed do the following:
Execute a command and log the console output in a file
Read that file and check if a String is present. The command that I will execute in the previous step will take a few seconds (5 - 10) seconds to complete and I need to read the file after it has finished executing. I suppose i can add sleep to make sure the command is finished executing or is there a better way to do this?
If the string is not present I want to execute the same command again until I find the String I am looking for
Once I find the string I am looking for I want to exit the loop and execute a different command
I found out how to do this in Java, but if I need to do this in a bash script.
The docker-containers have alpine as an operating system, but I updated the Dockerfile to install bash.
I tried this solution, but it does not work.
#!/bin/bash
[command to be executed] > allout.txt 2>&1
until
tail -n 0 -F /path/to/file | \
while read LINE
do
if echo "$LINE" | grep -q $string
then
echo -e "$string found in the console output"
fi
done
do
echo "String is not present. Executing command again"
sleep 5
[command to be executed] > allout.txt 2>&1
done
echo -e "String is found"
In your docker-compose file make use of depends_on option.
depends_on will take care of startup and shutdown sequence of your multiple containers.
But it does not check whether a container is ready before moving to another container startup. To handle this scenario check this out.
As described in this link,
You can use tools such as wait-for-it, dockerize, or sh-compatible wait-for. These are small wrapper scripts which you can include in your application’s image to poll a given host and port until it’s accepting TCP connections.
OR
Alternatively, write your own wrapper script to perform a more application-specific health check.
In case you don't want to make use of above tools then check this out. Here they use a combination of HEALTHCHECK and service_healthy condition as shown here. For complete example check this.
Just:
while :; do
# 1. Execute a command and log the console output in a file
command > output.log
# TODO: handle errors, etc.
# 2. Read that file and check if a String is present.
if grep -q "searched_string" output.log; then
# Once I find the string I am looking for I want to exit the loop
break;
fi
# 3. If the string is not present I want to execute the same command again until I find the String I am looking for
# add ex. sleep 0.1 for the loop to delay a little bit, not to use 100% cpu
done
# ...and execute a different command
different_command
You can timeout a command with timeout.
Notes:
colon is a utility that returns a zero exit status, much like true, I prefer while : instead of while true, they mean the same.
The code presented should work in any posix shell.
I am trying to start a number of jobs in different screens from a shell script. Each job will read in a different value of a parameter from a premade input file and run a simulation based on that value, then tee or > the output to a differently named file. So in a do loop around all the jobs, job 40 on screen "session40" will read in line 40 of the input file, run the simulation, and output to output40.dat, for example. (I am basically trying to run a few jobs in parallel in a very elementary way; it appears my computer has plenty of RAM for this).
I am encountering the issue that the > and | tee commands do not seem to work when I use "exec" to run a command on the remote screen, despite having attempted to start a bash shell there; when I use these commands, it just prints to standard output. Although these commands do work with the command "stuff," I do not know how to pass the job number to stuff, as it appears to only work with string inputs.
The current attempted script is as follows. I have replaced the simulation script with echo and > for a simpler example of the problem. Neither of the last two screen lines work.
for i in 1:10; do
screen -Sdm session$i bash
screen -S session$i -X exec echo $i > runnumber$i.output (method 1)
screen -S session$i -X stuff $'echo $i > runnumber$i.output\r' (method 2)
done
Might there be an easy fix?
When I'm looking at bash script code, I sometimes see | and sometimes see ||, but I don't know which is preferable.
I'm trying to do something like ..
set -e;
ret=0 && { which ansible || ret=$?; }
if [[ ${ret} -ne 0 ]]; then
# install ansible here
fi
Please advise which OR operator is preferred in this scenario.
| isn't an OR operator at all. You could use ||, though:
which ansible || {
true # put your code to install ansible here
}
This is equivalent to an if:
if ! which ansible; then
true # put your code to install ansible here
fi
By the way -- consider making a habit of using type (a shell builtin) rather than which (an external command). type is both faster and has a better understanding of shell behavior: If you have an ansible command that's provided by, say, a shell function invoking the real command, which won't know that it's there, but type will correctly detect it as available.
There is a big difference between using a single pipe (pipe output from one command to be used as input for the next command) and a process control OR (double pipe).
cat /etc/issue | less
This runs the cat command on the /etc/issue file, and instead of immediately sending the output to stdout it is piped to be the input for the less command. Yes, this isn't a great example, since you could instead simply do less /etc/issue - but at least you can see how it works
touch /etc/testing || echo Did not work
For this one, the touch command is run, or attempted to run. If it has a non-zero exit status, then the double pipe OR kicks in, and tries to execute the echo command. If the touch command worked, then whatever the other choice is (our echo command in this case) is never attempted...
I am trying to write a task-runner for command line. No rationale. Just wanted to do it. Basically it just runs a command, stores the output in a file (instead of stdout) and meanwhile prints a progress indicator of sorts on stdout and when its all done, prints Completed ($TIME_HERE).
Here's the code:
#!/bin/bash
task() {
TIMEFORMAT="%E"
COMMAND=$1
printf "\033[0;33m${2:-$COMMAND}\033[0m\n"
while true
do
for i in 1 2 3 4 5
do
printf '.'
sleep 0.5
done
printf "\b\b\b\b\b \b\b\b\b\b"
sleep 0.5
done &
WHILE=$!
EXECTIME=$({ TIMEFORMAT='%E';time $COMMAND >log; } 2>&1)
kill -9 $WHILE
echo $EXECTIME
#printf "\rCompleted (${EXECTIME}s)\n"
}
There are some unnecessarily fancy bits in there I admit. But I went through tons of StackOverflow questions to do different kinds of fancy stuff just to try it out. If it were to be applied anywhere, a lot of fat could be cut off. But it's not.
It is to be called like:
task "ping google.com -c 4" "Pinging google.com 4 times"
What it'll do is print Pinging google.com 4 times in yellow color, then on the next line, print a period. Then print another period every .5 seconds. After five periods, start from the beginning of the same line and repeat this until the command is complete. Then it's supposed to print Complete ($TIME_HERE) with (obviously) the time it took to execute the command in place of $TIME_HERE. (I've commented that part out, the current version would just print the time).
The Issue
The issue is that that instead of the execution time, something very weird gets printed. It's probably something stupid I'm doing. But I don't know where that problem originates from. Here's the output.
$ sh taskrunner.sh
Pinging google.com 4 times
..0.00user 0.00system 0:03.51elapsed 0%CPU (0avgtext+0avgdata 996maxresident)k 0inputs+16outputs (0major+338minor)pagefaults 0swaps
Running COMMAND='ping google.com -c 4';EXECTIME=$({ TIMEFORMAT='%E';time $COMMAND >log; } 2>&1);echo $EXECTIME in a terminal works as expected, i.e. prints out the time (3.559s in my case.)
I have checked and /bin/sh is a symlink to dash. (However that shouldn't be a problem because my script runs in /bin/bash as per the shebang on the top.)
I'm looking to learn while solving this issue so a solution with explanation will be cool. T. Hanks. :)
When you invoke a script with:
sh scriptname
the script is passed to sh (dash in your case), which will ignore the shebang line. (In a shell script, a shebang is a comment, since it starts with a #. That's not a coincidence.)
Shebang lines are only interpreted for commands started as commands, since they are interpreted by the system's command launcher, not by the shell.
By the way, your invocation of time does not correctly separate the output of the time builtin from any output the timed command might sent to stderr. I think you'd be better with:
EXECTIME=$({ TIMEFORMAT=%E; time $COMMAND >log.out 2>log.err; } 2>&1)
but that isn't sufficient. You will continue to run into the standard problems with trying to put commands into string variables, which is that it only works with very simple commands. See the Bash FAQ. Or look at some of these answers:
How to escape a variable in bash when passing to command line argument
bash quotes in variable treated different when expanded to command
Preserve argument splitting when storing command with whitespaces in variable
find command fusses on -exec arg
Using an environment variable to pass arguments to a command
(Or probably hundreds of other similar answers.)