I have a wrapper script for a CI pipeline which works great, but it always returns with 0 even though subcommands in a for loop fails. Here is an example:
#!/bin/bash
file_list=("file1 file2 file_nonexistant file3")
for file in $file_list
do
cat $file
done
>./listfiles.sh
file1 contents
file2 contents
cat: file_nonexistant: No such file or directory
file3 contents
>echo $?
>0
Since the last iteration of the loop is successfull the entire script exits with 0.
What i want is for the loop to continue on fail and for the script to exit 1 if any of the loop iterations returned errors.
What i have tried so far:
set -e but it halts the loop and exits when an iteration fails
replaced done with done || exit 1 - no effect
replaced cat $file with cat $file || continue - no effect
Alternative 1
#!/bin/bash
for i in `seq 1 6`; do
if test $i == 4; then
z=1
fi
done
if [[ $z == 1 ]]; then
exit 1
fi
With files
#!/bin/bash
touch ab c d e
for i in a b c d e; do
cat $i
if [[ $? -ne 0 ]]; then
fail=1
fi
done
if [[ $fail == 1 ]]; then
exit 1
fi
The special parameter $? holds the exit value of the last command. A value above 0 represents a failure. So just store that in a variable and check for it after the loop.
The $? parameter actually holds the exit status of the previous pipeline, if present. If the command is killed with a signal then the value of $? will be 128+SIGNAL. For example 128+2 in case of SIGINT (ctrl+c).
Overkill solution with trap
#!/bin/bash
trap ' echo X $FAIL; [[ $FAIL -eq 1 ]] && exit 22 ' EXIT
touch ab c d e
for i in c d e a b; do
cat $i || export FAIL=1
echo F $FAIL
done
I have a script that must be able to accept both by files and stdin on the first argument. Then if more or less than 1 arguments, reject them
The goal that I'm trying to accomplish is able to accpet using this format
./myscript myfile
AND
./myscript < myfile
What I have so far is
if [ "$#" -eq 1 ]; then #check argument
if [ -t 0 ]; then #check whether input from keyboard (read from github)
VAR=${1:-/dev/stdin} #get value to VAR
#then do stuff here!!
else #if not input from keyboard
VAR=$1
if [ ! -f "$VAR" ]; then #check whether file readable
echo "ERROR!"
else
#do stuff heree!!!
fi
fi
fi
The PROBLEM is when I tried to say
./myscript < myfile
it prints
ERROR!
I dont know whether this is the correct way to do this, I really appreciate for suggestion or the correct code for my problem. Thank you
#!/bin/bash
# if nothing passed in command line pass "/dev/stdin" to myself
# so all below code can be made branch-free
[[ ${#} -gt 0 ]] || set -- /dev/stdin
# loop through the command line arguments, treating them as file names
for f in "$#"; do
echo $f
[[ -r $f ]] && while read line; do echo 'echo:' $line; done < $f
done
Examples:
$ args.sh < input.txt
$ args.sh input.txt
$ cat input.txt | args.sh
in bash i am trying to make a script that goes:
echo hi | while read line; do echo $line; done
&
echo $?
would return 0
lets say the first script messed up somehow:
ech hi | while read line; do echo $line; done
&
echo $?
would still return 0
How does one go about returning that error?
The Bash internal variable $PIPESTATUS does this. It is an array containing the exit status(es) of the commands in the last executed pipe. The first command in the pipe is $PIPESTATUS[0], etc:
$ ech hi | while read line; do echo $line; done
-bash: ech: command not found
$ echo ${PIPESTATUS[0]} ${PIPESTATUS[1]}
127 0
$ echo hi | while read line; do ech $line; done
-bash: ech: command not found
$ echo ${PIPESTATUS[0]} ${PIPESTATUS[1]}
0 127
$ echo hi | while read line; do echo $line; done
hi
$ echo ${PIPESTATUS[0]} ${PIPESTATUS[1]}
0 0
I'm trying to implement a REPL (read-eval-print loop) in bash. If such a thing already exists, please ignore the following and answer this question with a pointer to it.
Let's use this script as an example (name it test.sh):
if true
then
echo a
else
echo b
fi
echo c
What I want to do is to read this script line by line, check if what I have read so far is a complete bash expression; if it is complete, eval it; otherwise keep on reading the next line. The script below illustrates my idea hopefully (it does not quite work, though).
x=""
while read -r line
do
x=$x$'\n'$line # concatenate by \n
# the line below is certainly a bad way to go
if eval $x 2>/dev/null; then
eval $x # code seems to be working, so eval it
x="" # empty x, and start collecting code again
else
echo 'incomplete expression'
fi
done < test.sh
Motivation
For a bash script, I want to parse it into syntactically complete expressions, evaluate each expression, capture the output, and finally mark up the source code and output (say, using Markdown/HTML/LaTeX/...). For example, for a script
echo a
echo b
What I want to achieve is the output like this:
```bash
echo a
```
```
a
```
```bash
echo b
```
```
b
```
instead of evaluating the whole script and capture all the output:
```bash
echo a
echo b
```
```
a
b
```
bash -n -c "$command_text"
...will determine whether your $command_text is a syntactically valid script without actually executing it.
Note that there's a huge breadth of space between "syntactically valid" and "correct". Consider adopting something like http://shellcheck.net/ if you want to properly parse the language.
The following scripts should generate the Markdown output you expect.
eval "set -n; $x" is used to verify if the command is complete, by checking for syntax errors in the command. Only a command that has no syntax errors will be considered complete, executed, and shown in the output Markdown.
Please note that the input script that is to be processed is executed in a sub-shell and therefore will not interfere with the processing script itself (i.e. the input script can use the same variable names as the processing script and cannot change the values of variables in the processing script). The only exception are the special variables called ___internal__variable___.
There are two approaches to how to achieve that, which I present below. In Version 1, whenever a new complete command is processed, all the statements before it are executed to create a "context" for the command. This effectively runs the input script multiple times.
In Version 2, the environment of the sub-shell is stored in a variable after each complete command is executed. Then, before the next command is executed, the previous environment is restored in the sub-shell.
Version 1
#!/bin/bash
x="" # Current
y="" # Context
while IFS= read -r line # Keep indentation
do
[ -z "$line" ] && continue # Skip empty lines
x=$x$'\n'$line # Build a complete command
# Check current command for syntax errors
if (eval "set -n; $x" 2> /dev/null)
then
# Run the input script up to the current command
# Run context first and ignore the output
___internal_variable___="$x"
out=$(eval "$y" &>/dev/null; eval "$___internal_variable___")
# Generate command markdown
echo "=================="
echo
echo "\`\`\`bash$x"
echo "\`\`\`"
echo
# Generate output markdown
if [ -n "$out" ]
then
echo "Output:"
echo
echo "\`\`\`"
echo "$out"
echo "\`\`\`"
echo
fi
y=$y$'\n'$line # Build context
x="" # Clear command
fi
done < input.sh
Version 2
#!/bin/bash
x="" # Current command
y="true" # Saved environment
while IFS= read -r line # Keep indentation
do
[ -z "$line" ] && continue # Skip empty lines
x=$x$'\n'$line # Build a complete command
# Check current command for syntax errors
if (eval "set -n; $x" 2> /dev/null)
then
# Run the current command in the previously saved environment
# Then store the output of the command as well as the new environment
___internal_variable_1___="$x" # The current command
___internal_variable_2___="$y" # Previously saved environment
out=$(bash -c "${___internal_variable_2___}; printf '<<<BEGIN>>>'; ${___internal_variable_1___}; printf '<<<END>>>'; declare -p" 2>&1)
# Separate the environment description from the command output
y="${out#*<<<END>>>}"
out="${out%%<<<END>>>*}"
out="${out#*<<<BEGIN>>>}"
# Generate command markdown
echo "=================="
echo
echo "\`\`\`bash$x"
echo "\`\`\`"
echo
# Generate output markdown
if [ -n "$out" ]
then
echo "Output:"
echo
echo "\`\`\`"
echo "$out"
echo "\`\`\`"
echo
fi
x="" # Clear command
fi
done < input.sh
Example
For input script input.sh:
x=10
echo "$x"
y=$(($x+1))
echo "$y"
while [ "$y" -gt "0" ]
do
echo $y
y=$(($y-1))
done
The output will be:
==================
```bash
x=10
```
==================
```bash
echo "$x"
```
Output:
```
10
```
==================
```bash
y=$(($x+1))
```
==================
```bash
echo "$y"
```
Output:
```
11
```
==================
```bash
while [ "$y" -gt "0" ]
do
echo $y
y=$(($y-1))
done
```
Output:
```
11
10
9
8
7
6
5
4
3
2
1
```
Assume your test commands are stored in a file called "example". That is, using same commands than in previous answer:
$ cat example
x=3
echo "$x"
y=$(($x+1))
echo "$y"
while [ "$y" -gt "0" ]
do
echo $y
y=$(($y-1))
done
the command:
$ (echo 'PS1=; PROMPT_COMMAND="echo -n =====; echo"'; cat example2 ) | bash -i
produces:
=====
x=3
=====
echo "$x"
3
=====
y=$(($x+1))
=====
echo "$y"
4
=====
=====
=====
while [ "$y" -gt "0" ]
> do
> echo $y
> y=$(($y-1))
> done
4
3
2
1
=====
exit
if you are interested also in the intermediate results of a loop, the command:
$ ( echo 'trap '"'"'echo; echo command: $BASH_COMMAND; echo answer:'"'"' DEBUG'; cat example ) | bash
results in:
command: x=3
answer:
command: echo "$x"
answer:
3
command: y=$(($x+1))
answer:
command: echo "$y"
answer:
4
command: [ "$y" -gt "0" ]
answer:
command: echo $y
answer:
4
command: y=$(($y-1))
answer:
command: [ "$y" -gt "0" ]
answer:
command: echo $y
answer:
3
command: y=$(($y-1))
answer:
command: [ "$y" -gt "0" ]
answer:
command: echo $y
answer:
2
command: y=$(($y-1))
answer:
command: [ "$y" -gt "0" ]
answer:
command: echo $y
answer:
1
command: y=$(($y-1))
answer:
command: [ "$y" -gt "0" ]
answer:
Addendum 1
It is not difficult to change the previous results to some other format. By example, this small perl script:
$ cat formatter.pl
#!/usr/bin/perl
#
$state=4; # 0: answer, 1: first line command, 2: more command, 4: unknown
while(<>) {
# print $state;
if( /^===COMMAND===/ ) {
print "===\n";
$state=1;
next;
}
if( $state == 1 ) {
print;
$state=2;
next;
}
if( $state == 2 && /^>+ (.*)/ ) {
print "$1\n";
next;
}
if( $state == 2 ) {
print "---\n";
$state=0;
redo;
}
if( $state == 0 ) {
print;
next;
}
}
when used in command:
( echo 'PS1="===COMMAND===\n"'; cat example ) | bash -i 2>&1 | ./formatter.pl
gives this result:
===
x=3
===
echo "$x"
---
3
===
y=$(($x+1))
===
echo "$y"
---
4
===
===
===
while [ "$y" -gt "0" ]
do
echo $y
y=$(($y-1))
done
---
4
3
2
1
===
exit
In lieu of pidfiles, as long as your script has a uniquely identifiable name you can do something like this:
#!/bin/bash
COMMAND=$0
# exit if I am already running
RUNNING=`ps --no-headers -C${COMMAND} | wc -l`
if [ ${RUNNING} -gt 1 ]; then
echo "Previous ${COMMAND} is still running."
exit 1
fi
... rest of script ...
In the below code, I am trying to check if the command within the if condition completed successfully and that the data was pushed into the target file temp.txt.
Consider:
#!/usr/bin/ksh
A=4
B=1
$(tail -n $(( $A - $B )) sample.txt > temp.txt)
echo "1. Exit status:"$?
if [[ $( tail -n $(( $A - $B )) sample.txt > temp.txt ) ]]; then
echo "2. Exit status:"$?
echo "Command completed successfully"
else
echo "3. Exit status:"$?
echo "Command was unsuccessfully"
fi
Output:
$ sh sample.sh
1. Exit status:0
3. Exit status:1
Now I can't get why the exit status changes above.. when the output of both the instances of the tail commands are identical. Where am I going wrong here..?
In the first case, you're getting the exit status of a call to the tail command (the subshell you spawned with $() preserves the last exit status)
In the second case, you're getting the exit status of a call to the [[ ]] Bash built-in. But this is actually testing the output of your tail command, which is a completely different operation. And since that output is empty, the test fails.
Consider :
$ [[ "" ]] # Testing an empty string
$ echo $? # exit status 1, because empty strings are considered equivalent to FALSE
1
$ echo # Empty output
$ echo $? # exit status 0, the echo command executed without errors
0
$ [[ $(echo) ]] # Testing the output of the echo command
$ echo $? # exit status 1, just like the first example.
1
$ echo foo
foo
$ echo $? # No errors here either
0
$ [[ $(echo foo) ]]
$ echo $? # Exit status 0, but this is **NOT** the exit status of the echo command.
0