Is it possible to go trough the results of grep in using a shell script like this:
while read line ; do
...
done < grep ...
Can anyone explain why this doesn't work? What are the alternatives?
thanks!
Looks like you were trying to use process substitution:
lines=5
while read line ; do
let ++lines
echo "$lines $line" # Number each line
# Other operations on $line and $lines
done < <(grep ...)
echo "Total: $lines lines"
Provided grep actually returns some output lines, the result should look like this:
6: foo
7: bar
Total: 7 lines
This is slightly different from grep ... | while ...: In the former, grep is run in a subshell, while in the latter the while loop is in a subshell. This is usually only relevant if you want to keep some state from within the loop - In that case you should use the first form.
On the other hand, if you write
lines=5
grep ... | while read line ; do
let ++lines
echo "$lines $line" # Number each line
# Other operations on $line and $lines
done
echo "Total: $lines lines"
the result would be:
6: foo
7: bar
Total: 5 lines
Ouch! The counter is passed to the subshell (the second part of the pipe), but it's not returned to the parent shell.
grep is a command but done < grep is telling the shell to use the file named grep as the input. You need something like:
grep ... | while read line ; do
...
done
Related
I am looping our the a grep result. The result contains 10 lines (every line has different content). So the loop stuff in the loop gets executed 10 times.
I need to get the index, 0-9, in the run so i can do actions based on the index.
ABC=(cat test.log | grep "stuff")
counter=0
for x in $ABC
do
echo $x
((counter++))
echo "COUNTER $counter"
done
Currently the counter won't really change.
Output:
51209
120049
148480
1211441
373948
0
0
0
728304
0
COUNTER: 1
If your requirement is to only print counter(which is as per shown samples only), in that case you could use awk(if you are ok with it), this could be done in a single awk like, without creating variable and then using grep like you are doing currently, awk could perform both search and counter printing in a single shot.
awk -v counter=0 '/stuff/{print "counter=" counter++}' Input_file
Replace stuff string above with the actual string you are looking for and place your actual file name for Input_file in above.
This should print like:
counter=1
counter=2
........and so on
Your shell script contains what should be an obvious syntax error.
ABC=(cat test.log | grep "stuff")
This fails with
-bash: syntax error near unexpected token `|'
There is no need to save the output in a variable if you only want to process one at a time (and obviously no need for the useless cat).
grep "stuff" test.log | nl
gets you numbered lines, though the index will be 1-based, not zero-based.
If you absolutely need zero-based, refactoring to Awk should solve it easily:
awk '/stuff/ { print n++, $0 }' test.log
If you want to loop over this and do something more with this information,
awk '/stuff/ { print n++, $0 }' test.log |
while read -r index output; do
echo index is "$index"
echo output is "$output"
done
Because the while loop executes in a subshell the value of index will not be visible outside of the loop. (I guess that's what your real code did with the counter as well. I don't think that part of the code you posted will repro either.)
Do not store the result of grep in a scalar variable $ABC.
If the line of the log file contains whitespaces, the variable $x
is split on them due to the word splitting of bash.
(BTW the statement ABC=(cat test.log | grep "stuff") causes a syntax error.)
Please try something like:
readarray -t abc < <(grep "stuff" test.log)
for x in "${abc[#]}"
do
echo "$x"
echo "COUNTER $((++counter))"
done
or
readarray -t abc < <(grep "stuff" test.log)
for i in "${!abc[#]}"
do
echo "${abc[i]}"
echo "COUNTER $((i + 1))"
done
you can use below increment statement-
counter=$(( $counter + 1));
I'm relatively new to bash scripting and I would like someone to explain this properly, thank you. Here is my code:
#! /bin/bash
echo "first arg: $1"
echo "first arg: $2"
var="$( grep -rnw $1 -e $2 | cut -d ":" -f1 )"
var2=$( grep -rnw $1 -e $2 | cut -d ":" -f1 | awk '{print substr($0,length,1)}')
echo "$var"
echo "$var2"
The problem I have is with the output, the script I'm trying to write is a c++ function searcher, so upon launching my script I have 2 arguments, one for the directory and the second one as the function name. This is how my output looks like:
first arg: Projekt
first arg: iseven
Projekt/AX/include/ax.h
Projekt/AX/src/ax.cpp
h
p
Now my question is: how do can I save the line by line output as a variable, so that later on I can use var as a path, or to use var2 as a character to compare. My plan was to use IF() statements to determine the type, idea: IF(last_char == p){echo:"something"}What I've tried was this question: Capturing multiple line output into a Bash variable and then giving it an array. So my code looked like: "${var[0]}". Please explain how can I use my line output later on, as variables.
I'd use readarray to populate an array variable just in case there's spaces in your command's output that shouldn't be used as field separators that would end up messing up foo=( ... ). And you can use shell parameter expansion substring syntax to get the last character of a variable; no need for that awk bit in your var2:
#!/usr/bin/env bash
readarray -t lines < <(printf "%s\n" "Projekt/AX/include/ax.h" "Projekt/AX/src/ax.cpp")
for line in "${lines[#]}"; do
printf "%s\n%s\n" "$line" "${line: -1}" # Note the space before the -1
done
will display
Projekt/AX/include/ax.h
h
Projekt/AX/src/ax.cpp
p
I'm processing a file by line, so my shell script is this:
check_vm_connectivity()
{
$res=`cat temp.txt` # this is line 10
i=0
for line in "$res"
do
i=$i+1
if [[ $i -gt 3 ]] ; then
continue
fi
echo "${line}"
done
}
The temp.txt is this:
+--------------------------------------+
| ID |
+--------------------------------------+
| cb91a52f-f0dd-443a-adfe-84c5c685d9b3 |
| 184564aa-9a7d-48ef-b8f0-ff9d51987e71 |
| f01f9739-c7a7-404c-8789-4e3e2edf314e |
| 825925cc-a816-4434-8b4b-a75301ddaefd |
when I run script, report this:
vm_connectivity.sh: line 10: =+--------------------------------------+: command not found
Why? How to fix this bug? Thank you~
May i ask why you are using
$res=`cat temp.txt`?
shouldn't it be
res=`cat temp.txt`
variables in a shell are set with var= rather than $var=
res is empty when you're entering the function, so that line
expands to the empty string, followed by an equals sign,
followed by the contents of temp.txt. The shell then interprets
that, and since a command is terminated by a newline, and
=+--------------------------------------+ has the syntax
of a command, rather than anything else, the shell tries to
run it as such.
You want: res=$(cat temp.txt)
However, it looks like you're trying to output
the first three lines, in which case just
do
head -n3 temp.txt
though from the looks of it, you probably want all except the first three
lines:
tail -n +4 temp.txt
and if you're looking for just the uuids:
tail -n +4 temp.txt | awk '{print $2}'
I'm trying to write an script that saves each line from "test" file in a variable line1, line2, and so on..
x=$(cat test | wc -l) #here i have how many lines
i="1"
while [ "$i" -lt "$x" ]
do
line$i=$(sed '$iq;d' test) #i try to get the number $i line one by one
i=$[$i+1]
done
Could you please help me?
Thanks!
To read each line of the file test into an array called lines, use:
mapfile -t lines <test
Example
Consider this file:
$ cat test
dogs and cats
lions and tigers
bears
Execute this statement:
$ mapfile -t lines <test
We can now see the value of lines using declare -p:
$ declare -p lines
declare -a lines2='([0]="dogs and cats" [1]="lions and tigers" [2]="bears")'
Each line of file test is now an element of array lines and we can access them by number:
$ echo "${lines[0]}"
dogs and cats
lines$i should be lines[$i] (or google eval) but chances are you're going about this completely wrong and you should be writing it in awk, e.g.:
awk '{lines[NR] = $0} END{print NR; for (i=1;i<=NR;i++) print i, lines[i]}' test
Let's take advantage of available tooling to see what's wrong:
$ shellcheck linereader
In linereader line 1:
x=$(cat test | wc -l) #here i have how many lines
^-- SC2148: Shebang (#!) missing. Assuming Bash.
^-- SC2002: Useless cat. Consider 'cmd < file | ..' or 'cmd file | ..' instead.
In linereader line 6:
line$i=$(sed '$iq;d' test) #i try to get the number $i line one by one
^-- SC1067: For indirection, use (associative) arrays or 'read "var$n" <<< "value"'
^-- SC2034: line appears unused. Verify it or export it.
^-- SC2016: Expressions don't expand in single quotes, use double quotes for that.
In linereader line 7:
i=$[$i+1]
^-- SC2007: Use $((..)) instead of deprecated $[..]
That's a good start:
#!/bin/bash
# 1. Shebang added
# 2. Redirection for reading files as suggested (stylistic)
x=$(wc -l < test) #here i have how many lines
i="1"
while [ "$i" -lt "$x" ]
do
# 3. Using arrays as suggested
# 4. Using double quotes for sed as suggested
# Additionally, use ${i} so we reference $i and not $iq
line[$i]=$(sed "${i}q;d" test) #i try to get the number $i line one by one
# 5. Use $((..)) as suggested (stylistic)
i=$((i+1))
done
# Print a line to show that it works:
echo "Line #2 is ${line[2]}"
And now let's try:
$ cat test
foo
bar
baz
$ ./linereader
Line #2 is bar
We could also have done this more easily (and in O(n) time) with mapfile:
#!/bin/bash
mapfile line < test
echo "Line #2 (zero-based index 1) is: ${line[1]}"
I have a file template.txt and its content is below:
param1=${1}
param2=${2}
param3=${3}
I want to replace ${1},{2},${3}...${n} string values by elements of scriptParams variable.
The below code, only replaces first line.
scrpitParams="test1,test2,test3"
cat template.txt | for param in ${scriptParams} ; do i=$((++i)) ; sed -e "s/\${$i}/$param/" ; done
RESULT:
param1=test1
param2=${2}
param3=${3}
EXPECTED:
param1=test1
param2=test2
param3=test3
Note: I don't want to save replaced file, want to use its replaced value.
If you intend to use an array, use a real array. sed is not needed either:
$ cat template
param1=${1}
param2=${2}
param3=${3}
$ scriptParams=("test one" "test two" "test three")
$ while read -r l; do for((i=1;i<=${#scriptParams[#]};i++)); do l=${l//\$\{$i\}/${scriptParams[i-1]}}; done; echo "$l"; done < template
param1=test one
param2=test two
param3=test three
Learn to debug:
cat template.txt | for param in ${scriptParams} ; do i=$((++i)) ; echo $i - $param; done
1 - test1,test2,test3
Oops..
scriptParams="test1 test2 test3"
cat template.txt | for param in ${scriptParams} ; do i=$((++i)) ; echo $i - $param; done
1 - test1
2 - test2
3 - test3
Ok, looks better...
cat template.txt | for param in ${scriptParams} ; do i=$((++i)) ; sed -e "s/\${$i}/$param/" ; done
param1=test1
param2=${2}
param3=${3}
Ooops... so what's the problem? Well, the first sed command "eats" all the input. You haven't built a pipeline, where one sed command feeding the next... You have three seds trying to read the same input. Obviously the first one processed the whole input.
Ok, let's take a different approach, let's create the arguments for a single sed command (note: the "" is there to force echo not to interpret -e as an command line switch).
sedargs=$(for param in ${scriptParams} ; do i=$((++i)); echo "" -e "s/\${$i}/$param/"; done)
cat template.txt | sed $sedargs
param1=test1
param2=test2
param3=test3
That's it. Note that this isn't perfect, you can have all sort of problems if the replace texts are complex (e.g.: contain space).
Let me think how to do this in a better way... (well, the obvious solution which comes to mind is not to use a shell script for this task...)
Update:
If you want to build a proper pipeline, here are some solutions: How to make a pipe loop in bash
You can do that with just bash alone:
#!/bin/bash
scriptParams=("test1" "test2" "test3") ## Better store it as arrays.
while read -r line; do
for i in in "${!scriptParams[#]}"; do ## Indices of array scriptParams would be populated to i starting at 0.
line=${line/"\${$((i + 1))}"/"${scriptParams[i]}"} ## ${var/p/r} replaces patterns (p) with r in the contents of var. Here we also add 1 to the index to fit with the targets.
done
echo "<br>$line</br>"
done < template.txt
Save it in a script and run bash script.sh to get an output like this:
<br>param1=test1</br>
<br>param2=test2</br>
<br>param3=test3</br>