I have a script which iterates through a file and finds matches in another file. How to I get the process to stop once I've found a match.
For example:
I take the first line in name.txt, and then try to find a match for it in file.txt.
name.txt:
7,7,FRESH,98,135,
65,10,OLD,56,45,
file.txt:
7,7,Dave,S
8,10,Frank,S
31,7,Gregg
45,5,Jake,S
Script:
while read line
do
name_id=`echo $line | cut -f1,2 -d ','`
identiferOne=`echo $name_id | cut -f1 -d ','`
identiferTwo=`echo $name_id | cut -f2 -d ','`
while IFS= read line
do
CHECK=`echo $line | cut -f4 -d','`
if [ $CHECK = "S" ]
then
symbolName=`echo $line | cut -f3 -d ','`
numberOne=`echo $line | awk -F',' '{print $1}'`
numberTwo=`echo $line | cut -f2 -d ','`
if [ "$numberOne" == $identiferOne ] && [ "$numberTwo" == $identifierTwo ]
then
echo "WE HAVE A MATCH with $symbolName"
break
fi
fi
done < /tmp/file.txt
done < /tmp/name.txt
My question is - how do I stop the script from iterating through file.txt once it has found an initial match, and then set that matched record into a variable, stop the if statement, then do some other stuff within the loop using that variable. I tried using break; but that exits the loop, which is not what I want.
You can tell grep different things:
Stop searching after the first match (option -m 1).
Read the searchkeys from a file (option -f file).
Pretend that the output of a command is a file (not really grep, bash helps here) with <(cmmnd).
Combining these will give you
grep -m1 -f <(cut -d"," -f1-2 name.txt) file.txt
Close, but not what you want. The substrings given by cut -d"," -f1-2 name.txt will match everywhere in the line, and you want to match the first two fields. Matching at the start of the line is done with ^, so we use sed to make strings like ^field1,field2 :
grep -m1 -f <(sed 's/\([^,]*,[^,]*,\).*/^\1/' name.txt) file.txt
Related
I have a version.properties file as below:
#some comments
major=2
minor=16
patch=4
I want to increase the minor value by 1. so new value should be 16 +1 = 17.
So the updated file will be:
#some comments
major=2
minor=17
patch=4
Is it possible to achieve this through bash script?
Thanks in advance
You could use awk:
awk -F"=" 'BEGIN{OFS=FS} $1=="minor"{$2=$2+1}1' versions.properties
This awk script says:
Split each record by a an equal sign (-F"=")
Ensure that the output field separator OFS is equal to the incoming field separator FS (BEGIN{OFS=FS})
If the first field is the word "minor" ($1=="minor")
Then add 1 to the second field ({$2=$2+1})
Finally print the record back out 1.
You can redirect this to a temp file and then mv the temp back over the top of the original:
awk -F"=" 'BEGIN{OFS=FS} $1=="minor"{$2=$2+1}1' versions.properties > versions.properties.tmp && mv versions.properties.tmp versions.properties
You can also use this:
#! /bin/bash
LINE=$(cat version.properties | grep minor)
VAL=$(echo $LINE | cut -f 2 -d "=")
VAL=$(($VAL + 1))
NEW_LINE=$(echo minor=$VAL)
sed -e "s,$LINE,$NEW_LINE,g" -i version.properties
It fixes the issue without the need to use additional redirections and mv commands. Hope this helps :-)
Steps:
1- Get line number of your parameter (to use in replacement)
grep -n "minor" version.properties | cut -d: -f1
This shows "3" according to your version.properties file
We save it in a variable
line=$(grep -n "minor" version.properties | cut -d: -f1)
2- Get key=value
grep -n "minor" version.properties | cut -d: -f2
This shows minor=16 according to your version.properties file
We save it in a variable called string:
string=$(grep -n "minor" version.properties | cut -d: -f2)
3- Get version of this string : minor=16
version=$(echo $string | cut -d= -f2)
4- Increase version
newVersion=$(( version + 1))
5- Replace in original properties file using line number:
sed -i "$line c $key=$newVersion" version.properties
Finally this is a complete script , with key and file as parameters, so you could use it to increase any key like in any properties file, in any location :
key=$1
file=$2
line=$(grep -n "$key" $file | cut -d: -f1)
string=$(grep -n "$key" $file | cut -d: -f2)
# get version from string
version=$(echo $string | cut -d= -f2)
#increase version
newVersion=$(( version + 1))
#update file
sed -i "$line c $key=$newVersion" $file
Usage:
bash update_version.sh "patch" /server/files/version.properties
Note : keys in file must be unique.
I'm trying to print domain and topLeveldomain variables (example.com)
$line = example.com
domain =$line | cut -d. -f 1
topLeveldomain = $line | cut -d. -f 2
However when I try and echo $domain, it doesn't display desired value
test.sh: line 4: domain: command not found
test.sh: line 5: topLeveldomain: command not found
I suggest:
line="example.com"
domain=$(echo "$line" | cut -d. -f 1)
topLeveldomain=$(echo "$line" | cut -d. -f 2)
The right code for this should be:
line="example.com"
domain=$(echo "$line" | cut -d. -f 1)
topLeveldomain=$(echo "$line" | cut -d. -f 2)
Consider the right syntax of bash:
variable=value
(there are no blanks allowed)
if you want to use the content of the variable you have to add a leading $
e.g.
echo $variable
You don't need external tools for this, just do this in bash
$ string="example.com"
# print everything upto first de-limiter '.'
$ printf "${string%%.*}\n"
example
# print everything after first de-limiter '.'
$ printf "${string#*.}\n"
com
Remove spaces around =:
line=example.com # YES
line = example.com # NO
When you create a variable, do not prepend $ to the variable name:
line=example.com # YES
$line=example.com # NO
When using pipes, you need to pass standard output to the next command. Than means, you usually need to echo variables or cat files:
echo $line | cut -d. -f1 # YES
$line | cut -d. -f1 # NO
Use the $() syntax to get the output of a command into a variable:
new_variable=$(echo $line | cut -d. -f1) # YES
new_variable=echo $line | cut -d. -f1 # NO
I would rather use AWK:
domain="abc.def.hij.example.com"
awk -F. '{printf "TLD:%s\n2:%s\n3:%s\n", $NF, $(NF-1), $(NF-2)}' <<< "$domain"
Output
TLD:com
2:example
3:hij
In the command above, -F option specifies the field separator; NF is a built-in variable that keeps the number of input fields.
Issues with Your Code
The issues with your code are due to invalid syntax.
To set a variable in the shell, use
VARNAME="value"
Putting spaces around the equal sign will cause errors. It is a good
habit to quote content strings when assigning values to variables:
this will reduce the chance that you make errors.
Refer to the Bash Guide for Beginners.
this also works:
line="example.com"
domain=$(echo $line | cut -d. -f1)
toplevel=$(cut -d. -f2 <<<$line)
echo "domain name=" $domain
echo "Top Level=" $toplevel
You need to remove $ from line in the beginning, correct the spaces and echo $line in order to pipe the value to cut . Alternatively feed the cut with $line.
We are having a wierd issue.
We have this lines :
while read line2; do
echo $line2
done < $1 | `echo grep '.*|.*|.*|.*|.*|.*|.*|.*'` | sort -nbsk1 | cut -d "|" -f1 | uniq -d
Which prints what they should print. but, when changing the echo to ->
while read line2; do
echo "Hello World"
done < $1 | `echo grep '.*|.*|.*|.*|.*|.*|.*|.*'` | sort -nbsk1 | cut -d "|" -f1 | uniq -d
It wont print anything, same result for anything different then $line2.
Whats even more wierd is :
echo " $line2 Hello"
Will print the line2 variable
echo "Hello $line2"
Print nothing
I have tried the same with printf, same results.
Any suggestions ?
What you've written is equivalent to the following shell code:
cat $1 |
while read line2; do
echo $line2
done |
`echo grep '.*|.*|.*|.*|.*|.*|.*|.*'` |
sort -nbsk1 |
cut -d "|" -f1 |
uniq -d
The while read loop takes the contents of file $1 and echoes them, which does nothing other than remove leading and trailing spaces and replace internal spaces with a single space. If you replace the echo $line2 line with echo "Hello World", that string is clearly not going to match the grep command that the output of the loop is being passed through, so producing no output is unsurprising.
When you change the echo line to echo " $line2 Hello", you tack "Hello" onto the end of the input line, which then matches the grep command and gets sliced off the end of the string with the cut command, so it makes sense that it would have essentially no ultimate effect.
If you change the echo line to echo "Hello $line2", any number at the beginning of the line becomes invisible to the sort -ns, which makes your sort call essentially a no-op. This is probably why you're not seeing anything in this situation, although you probably would see something if two identical lines appeared in the input one after the other. (In my testing on my machine, I see one such line because I happen to have two identical lines in succession in my test case.)
It's not exactly clear what you're trying to do since the while loop is almost a no-op. It's possible what you want to do is something more like this:
grep '.*|.*|.*|.*|.*|.*|.*|.*' < $1 |
sort -nbsk1 |
cut -d "|" -f1 |
uniq -d |
while read line2; do
echo $line2
done
... but I'm only speculating at this point.
I have the following:
FILENAME=$1
cat $FILENAME | while read LINE
do
response="$LINE" | cut -c1-14
request="$LINE" | cut -c15-31
difference=($response - $request)/1000
echo "$difference"
done
When I run this script it returns blank lines. What am I doing wrong?
Might be simpler in awk:
awk '{print ($1 - $2)/1000}' "$1"
I'm assuming that the first 14 chars and the next 17 chars are the first two blank-separated fields.
You need to change it to:
response=`echo $LINE | cut -c1-14`
request=`echo $LINE | cut -c15-31`
difference=`expr $response - $request`
val=`expr $difference/1000`
You are basically doing everything wrong ;)
This should be better:
FILENAME="$1"
cat "$FILENAME" | while read LINE
do
response=$(echo "$LINE" | cut -c1-14) # or cut -c1-14 <<< "$line"
request=$(echo "$LINE" | cut -c15-31)
difference=$((($response - $request)/1000)
echo "$difference"
done
I am trying to grep out the lines in a file where the third field matches certain criteria.
I tried using grep but had no luck in filtering out by a field in the file.
I have a file full of records like this:
12794357382;0;219;215
12795287063;0;220;215
12795432063;0;215;220
I need to grep only the lines where the third field is equal to 215 (in this case, only the third line)
Thanks a lot in advance for your help!
Put down the hammer.
$ awk -F ";" '$3 == 215 { print $0 }' <<< $'12794357382;0;219;215\n12795287063;0;220;215\n12795432063;0;215;220'
12795432063;0;215;220
grep:
grep -E "[^;]*;[^;]*;215;.*" yourFile
in this case, awk would be easier:
awk -F';' '$3==215' yourFile
A solution in pure bash for the pre-processing, still needing a grep:
while read line; do
OLF_IFS=$IFS; IFS=";"
line_array=( $line )
IFS=$OLD_IFS
test "${line_array[2]}" = 215 && echo "$line"
done < file | grep _your_pattern_
Simple egrep (=grep -E)
egrep ';215;[0-d][0-d][0-d]$' /path/to/file
or
egrep ';215;[[:digit:]]{3}$' /path/to/file
How about something like this:
cat your_file | while read line; do
if [ `echo "$line" | cut -d ";" -f 3` == "215" ]; then
# This is the line you want
fi
done
Here is the sed version to grep for lines where 3rd field is 215:
sed -n '/^[^;]*;[^;]*;215;/p' file.txt
Simplify your problem by putting the 3rd field at the beginning of the line:
cut -d ";" -f 3 file | paste -d ";" - file
then grep for the lines matching the 3rd field and remove the 3rd field at the beginning:
grep "^215;" | cut -d ";" -f 2-
and then you can grep for whatever you want. So the complete solution is:
cut -d ";" -f 3 file | paste -d ";" - file | grep "^215;" | cut -d ";" -f 2- | grep _your_pattern_
Advantage: Easy to understand; drawback: many processes.