According the bash(1) man pages, when I run the following:
set -e
x=2
echo Start $x
while [ $((x--)) -gt 0 ]; do echo Loop $x; done | cat
echo End $x
The output will be:
Start 2
Loop 1
Loop 0
End 2
After the loop (runs as a subshell) the variable x reset to 2. But if I remove the pipe the x will be updated:
Start 2
Loop 1
Loop 0
End -1
I need to change the x but, I need the pipe too.
Any idea how to get around this problem?
bash always (at least as of 4.2) runs all non-rightmost parts of a pipeline in a subshell. If the value of x needs to change in the calling shell, you must rewrite your code to avoid the pipeline.
One horrible-looking example:
# If you commit to one bash feature, may as well commit to them all:
# Arithmetic compound: (( x-- > 0 ))
# Process substitution: > >( cat )
while (( x-- > 0 )); do echo Loop $x; done > >( cat )
Related
I want to break from my nested while loops.
Below is what i have tried in my code,
while [itretative condition]
do
...
...
cat test1.json | while read line
do
...
...
cat test2.json | while read line
do
...
...
if [ "$taskstatus" = "RUNNING" ]
#when my task status reach running, i want to stop the script execution and end it.
break 3 #break 3 or exit is not working for me.
fi
done
done
done
Please suggest me how i can achieve this?
Whenever you use a pipe, you create a sub shell. Neiter break nor exit works over sub shell boundaries.
Even trap does not work over sub shell boundaries. But the option set -E tells Bash to inherit the error handler of the parent shell. By this you can reserve a special exit code to implement the break logic.
The following code is divided in an outer shell and a sub shell, which runs your code using pipes and creating further sub shells.
The error handler of the outer shell is not inherited by the sub shells, because trap handler inheritance is disabled by default. The error handler of the outer shell just checks for the reserved exit code (42 in the example) and treats this as no error.
In the first sub shell error handler inheritance is enabled by set -E. This means that all sub shells share the same error handler. The sub shell error handler just passes the error code through and terminates the shell. By this all sub shells are terminated.
#! /bin/bash
err()
{
local err=$?
if (( err == 42 )); then
exit
else
exit $err
fi
}
trap err ERR
(
set -E
err()
{
local err=$?
exit $err
}
trap err ERR
printf "%s\n" a b c | while read i; do
echo $i
printf "%s\n" x y z | while read j; do
echo $j
exit 42
done
done
)
As mentioned in the comments, break statement will not work across sub-shell created from a pipe. For the simple case that the pipe is simply cat file | while, it is possible to 'de-pipe' the expression with while read ... done <somefile (Gordon Davisson comment above).
For the more general case, where the pipeline can contain arbitrary commands (grep x file1, and grep y file2, in the example below), possible to de-pipe the command (and allow the break to work over multiple levels) using one of the following: (1) here documents (2) process substitution.
With here documents: <<<"$(commands)"
while read x ; do
while read y ; do
echo "$x/$y" ;
[[ "$x" = d* ]] && echo BREAK && break 2
done <<< $(grep y file2);
done <<< "$(grep x file1)"
With process substitution < <(commands)
while read x ; do
while read y ; do
echo "$x/$y" ;
[[ "$x" = d* ]] && echo FOO && break 2 ;
done < <(grep y file2);
done < <(grep x file1)
I have the while loop in ksh which reads the file and loop through each line.
Here is the same file contents (TestCases.txt)
TEST_PROC_1(1)/TEST_1,TEST_2,TEST_3/N/P
TEST_PROC_1(1)/TEST_1,TEST_2,TEST_3/N/N
TEST_PROC_2('CICD_DEMO.txt')/TEST_1,TEST_2,TEST_3/N/N
TEST_FUNC_1(100)/TEST_1,TEST_2,TEST_3/N/P
TEST_FUNC_2/TEST_1,TEST_2,TEST_3/N/N
TEST_PROC_4/TEST_1,TEST_2/N/N
TEST_FUNC_3(3)//N/P
The scripts which reads the document
swd=$(pwd)
export swd
file=${swd}/TestCases.txt
export testCaseIndex=1
export validateTblIndex=1
cat ${file} | while IFS=\/ read procname tablelist hold_data testcase_type
do
echo "$procname $tablelist $hold_data $testcase_type"
ksh ${swd}/assets/sh/main.sh "${procname}" "${tablelist}" "${hold_data}" "${testcase_type}" "${testCaseIndex}" "${validateTblIndex}"
ret=$?
echo $ret
(( testCaseIndex+=1 ))
(( validateTblIndex+=1 ))
done
Here is the problem
If I comment the ksh call it iterates till last line.
TEST_PROC_1(1) TEST_1,TEST_2,TEST_3 N P
0
TEST_PROC_1(1) TEST_1,TEST_2,TEST_3 N N
0
TEST_PROC_2('CICD_DEMO.txt') TEST_1,TEST_2,TEST_3 N N
0
TEST_FUNC_1(100) TEST_1,TEST_2,TEST_3 N P
0
TEST_FUNC_2 TEST_1,TEST_2,TEST_3 N N
0
TEST_PROC_4 TEST_1,TEST_2 N N
0
TEST_FUNC_3(3) N P
0
If I uncomment it stops with first line of the file.
TEST_PROC_1(1) TEST_1,TEST_2,TEST_3 N P
0
Kindly help out what could be possible issues. ksh call works fine even I run separately.
I have ksh93 version.
main.sh is also reading from standard input, which it inherits from the loop, so it is consuming data meant for the read command. Given that this surprises you, you may simply be able to redirect the script's standard input from /dev/null.
(Also, unless cat ${file} is just filling in for some other process that produces the data, use input redirection instead of a pipe.)
while IFS=/ read procname tablelist hold_data testcase_type
do
echo "$procname $tablelist $hold_data $testcase_type"
ksh ${swd}/assets/sh/main.sh "${procname}" \
"${tablelist}" "${hold_data}" "${testcase_type}" \
"${testCaseIndex}" "${validateTblIndex}" < /dev/null
ret=$?
echo $ret
(( testCaseIndex+=1 ))
(( validateTblIndex+=1 ))
done < $file
If main.sh does need to read from standard input, use a different file descriptor for read command.
while IFS=/ read procname tablelist hold_data testcase_type <&3
do
echo "$procname $tablelist $hold_data $testcase_type"
ksh ${swd}/assets/sh/main.sh "${procname}" \
"${tablelist}" "${hold_data}" "${testcase_type}" \
"${testCaseIndex}" "${validateTblIndex}" < /dev/null
ret=$?
echo $ret
(( testCaseIndex+=1 ))
(( validateTblIndex+=1 ))
done 3< $file
to loop through a continous list of numbers in bash I can do
for s in $(seq 1 5);do
echo ${s}
done
to loop through a continous list of numbers leaving a given number out in python I can do:
list = [s2 for s2 in range(6)[1:] if s2 != s1]
for s1 in list:
print s1
where list contains all numbers in range except s1
How do I do the same in bash?
Just use continue to skip this step:
for s in {1..5} # note there is no need to use $(seq...)
do
[ "$s" -eq 3 ] && continue # if var is for example 3, jump to next loop
echo "$s"
done
This returns:
1
2
4 # <--- 3 is skipped
5
From Bash Reference Manual → 4.1 Bourne Shell Builtins:
continue
continue [n]
Resume the next iteration of an enclosing for, while, until, or select
loop. If n is supplied, the execution of the nth enclosing loop is
resumed. n must be greater than or equal to 1. The return status is
zero unless n is not greater than or equal to 1.
Add a short circuit evaluation, || (logical OR) :
for s in $(seq 1 5); do
(( s == 3 )) || echo "$s"
done
(( s == 3 )) checks if $s is equal to 3, if not (||) echo the number.
With the reverse check ($s not equal to 3) and logical AND (&&):
for s in $(seq 1 5); do
(( s != 3 )) && echo "$s"
done
The classic way, if with test ([), non-equity test:
for s in $(seq 1 5); do
if [ "$s" -ne 3 ]; then
echo "$s"
fi
done
Reverse test, equity check:
for s in $(seq 1 5); do
if [ "$s" -eq 3 ]; then
continue
fi
echo "$s"
done
continue will make the loop control to go at the top rather than evaluating the following commands.
There is also a bash keyword [[ which behaves similarly in most cases but more robust.
You can use BASH arithmetic construct ((...)) like this:
s1=3 # skip this
s2=6 # upper count
for ((i=1; i<s2; i+=(i==s1-1?2:1) )); do echo $i; done
1
2
4
5
About: i+=(i==s1-1?2:1)
In the for loop instead of always incrementing i by 1 here we are incrementing i by 2 when i is 1 less then the number to be skipped.
Alternatively solution using BASH array:
arr=({1..5}) # populate 1 to 5 in an array
unset arr[s1-1] # delete element s1-1
# print the array
printf "%s\n" "${arr[#]}"
1
2
4
5
I am trying to write a bash script that accepts a number from the keyboard, and then prints a set of integers from 0 to the number entered. I can't figure out how to do it at all.
This is my code:
while [ 1 ]
do
echo -n "Enter a color: "
read user_answer
for (( $user_answer = $user_answer; $user_answer>0; $user_answer--))
echo $user_answer
fi
done
exit
The error I'm recieving is:
number_loop: line 10: syntax error near unexpected token echo'
number_loop: line 10: echo $user_answer'
Assign a separate variable in order to use increment/decrement operators. $user_answer=$user_answer will always be true and it will throw an error when trying to use decrement. Try the following :
#!/bin/bash
while [ 1 ]
do
echo -n "Enter a color: "
read user_answer
for (( i=$user_answer; i>0; i-- ))
do
echo $i
done
done
exit
You missed the do statement between your for and the echo.
bash has many options to write numbers. What you seem to be trying to do is easiest done with seq:
seq $user_answer -1 0
If you want to use your loop, you have to insert a ; do and replace the fi with done, and replace several $user_answer:
for (( i = $user_answer; i>0; i--)); do
echo $i
done
(btw: I assumed that you wanted to write the numbers in reverse order, as you are going backwards in your loop. Forwards is even easier with seq:
seq 0 $user_input
)
This is where a c-style loop works particularly well:
#!/bin/bash
for ((i = 1; i <= $1; i++)); do
printf "%s\n" "$i"
done
exit 0
Example
$ bash simplefor.sh 10
1
2
3
4
5
6
7
8
9
10
Note: <= is used as the for loop test so it 10 it iterates 1-10 instead of 0-9.
In your particular case, iterating from $user_answer you would want:
for (( i = $user_answer; i > 0; i--)); do
echo $i
done
The for loop is a bash internal command, so it doesn't fork a new process.
The seq command has a nice, one-line syntax.
To get the best of the twos, you can use the { .. } syntax:
eval echo {1..$answer}
Can someone please explain the difference between these two while loops:
while read test; do
echo $test
done <<< "$(seq 5)"
-
while read test; do
echo $test
done < <(seq 5)
while read test; do
echo $test
done <<< "$(seq 5)"
Execute seq 5, collecting the result into a temporary variable. Then execute the while loop, feeding it the collecting result.
while read test; do
echo $test
done < <(seq 5)
Set up a subshell to execute seq 5 and connect its stdout to stdin. Then start the while loop. When it finishes, restore stdin.
What's the difference? For seq 5, practically nothing; however, it can still be made visible by changing seq 5 to seq 5; echo done generating sequence >&2. Then you can see that in the first case, the entire seq execution finishes before the while loop starts, while in the second case they execute in parallel.
$ while read n; do echo $n > /dev/stderr; done \
> <<<"$(seq 5; echo done generating sequence >&2)"
done generating sequence
1
2
3
4
5
$ while read n; do echo $n > /dev/stderr; done \
> < <(seq 5; echo done generating sequence >&2)
1
2
done generating sequence
3
4
5
If it were seq 10000000, the difference would be much clearer. The <<<"$(...) form would use a lot more memory in order to store the temporary string.
Based on what I perceive, the only difference is that process substitution would represent a named pipe e.g. /dev/fd/63 as a file for input whereas <<< "" would send the input internally as if reading a buffer. Of course is the command reading the input is on another process like a subshell or another binary, then it would be sent to it like a pipe. Sometimes in environments where process substitution is not possible like in Cygwin, here documents or here strings along with command substitutions are more helpful.
If you do echo <(:) you see the difference in concept for process substitution over other string inputs.
Process substitution is more of representing a file, while here strings are more of sending in a buffer for input.