How do you use grep to find a pattern in a file, EDIT IT with awk (or some other thing), and then save it? - bash

Question

I need to edit specific lines in a text file. I have a pattern here, pattern.txt:
1
3
6
17
etc...
and a file with text, file.txt:
1 text
2 text
3 text
4 text
5 text
etc...
I want to add the words _PUT FLAG HERE to the end of each line of file.txt on lines that have match indicated by the pattern.txt.
I have
grep -F -f pattern.txt file.txt | awk '{print $0 "_PUT FLAG HERE" }'
But I can't seem to figure out a way to shove those changes back into the original file so it looks like this:
1 text_PUT FLAG HERE
2 text
3 text_PUT FLAG HERE
4 text
5 text
6 teeskjtkljeltsj _PUT FLAG HERE
etc...
It's a lot like trying to use tr, but much more convoluted. There should be a logical way to string AWK and grep, I just can't seem conceive of a way to put the pieces together into one pipe that would do this, and I can't find the answer anywhere. (If you explain a sed way to do this, please explain the regex.)

Answer 1

Assume your awk has been taken hostage.
A GNU sed/grep solution! To generate a sed script that does what you want, we get the lines to change from the input file:
$ grep -wFf pattern.txt file.txt
1 text
3 text
6 text
17 text
This matches complete words (-w) so 1 text is matched, but 11 text is not; -F is for fixed strings (no regex, should be faster) and -f pattern.txt reads the patterns to look for from a file.
Now we pipe this to sed to generate a script:
$ grep -wFf pattern.txt file.txt | sed 's#.*#/^&$/s/$/_PUT FLAG HERE/#'
/^1 text$/s/$/_PUT FLAG HERE/
/^3 text$/s/$/_PUT FLAG HERE/
/^6 text$/s/$/_PUT FLAG HERE/
/^17 text$/s/$/_PUT FLAG HERE/
The sed command in the pipe matches the complete line (.*) and assembles an address plus substitution command (& stands for the whole previously matched line).
Now we take all that and use it as input for sed by means of process substitution (requires Bash):
$ sed -f <(grep -wFf pattern.txt file.txt | sed 's#.*#/^&$/s/$/_PUT FLAG HERE/#') file.txt
1 text_PUT FLAG HERE
2 text
3 text_PUT FLAG HERE
4 text
5 text
6 text_PUT FLAG HERE
7 text
8 text
9 text
10 text
11 text
12 text
13 text
14 text
15 text
16 text
17 text_PUT FLAG HERE
Done!
Yes, yes, awk is shorter1, faster and more beautiful.
1 Actually not, but still.
Another remark: the grep step is not actually required, see answers by potong and Walter A.

Answer 2

Try this:
pattern.txt:
1
3
6
17
file.txt:
1 text
2 text
3 text
4 text
5 text
Use awk:
$ awk 'NR == FNR{seen[$1];next} $1 in seen{printf("%s_PUT FLAG HERE\n",$0);next}1' pattern.txt file.txt
Output:
1 text_PUT FLAG HERE
2 text
3 text_PUT FLAG HERE
4 text
5 text

Answer 3

awk to the rescue!
you don't need other tools with the full power of awk at your disposal
$ awk -v tag='_PUT FLAG HERE' 'NR==FNR{a[$1];next}
{print $0 ($1 in a?tag:"")}' pattern file
1 text_PUT FLAG HERE
2 text
3 text_PUT FLAG HERE
4 text
5 text
just as an exercise, do the same with join/sort
$ sort <(join pattern file --nocheck-order |
sed 's/$/_PUT_FLAG_HERE/') <(join -v2 pattern file --nocheck-order)
1 text_PUT_FLAG_HERE
2 text
3 text_PUT_FLAG_HERE
4 text
5 text
perhaps defining function for DRY
$ f() { join $1 pattern file --nocheck-order; }; sort <(f "" |
sed 's/$/_PUT_FLAG_HERE/') <(f -v2)

Answer 4

The solution of #Benjamin can be simplified to
sed -f <(sed 's#.*#/^& /s/$/_PUT FLAG HERE/#' pattern.txt) file.txt
Explanation
# Read awk commands from a file
sed -f awkcommands.txt pattern.txt file.txt
# Read awk commands from other command
sed -f <(other_command) file.txt
# Append string to every line by replacing end-of-line character $
sed 's/$/_PUT FLAG HERE/'
# Only append string on lines matching something
sed '/something/s/$/_PUT FLAG HERE/#'
# Only append string on lines matching something at the beginning of the line followed by a space
sed '/^something /s/$/_PUT FLAG HERE/#'
# Get the word something in above command selecting the whole line with .* and putting it in the new sed command with &.
# The slashes are used for the inner sed command, so use # here
sed 's#.*#/^& /s/$/_PUT FLAG HERE/#' pattern.txt
# Now all together:
sed -f <(sed 's#.*#/^& /s/$/_PUT FLAG HERE/#' pattern.txt) file.txt

Answer 5

This might work for you (GNU sed):
sed 's#.*#/&/s/$/_PUT FLAG HERE/#' pattern.txt | sed -f - file
This turns the pattern file into a sed script which is then invoked against the text file.

Answer 6

This solution uses only Bash (4.0+) features:
# Set up associative array 'patterns' whose keys are patterns
declare -A patterns
for pat in $(< pattern.txt) ; do patterns[$pat]=1 ; done
# Slurp all the lines of 'file.txt' into the 'lines' array
readarray -t lines < file.txt
# Write each old line in the file, possibly with a suffix, back to the file
for line in "${lines[#]}" ; do
read -r label text <<< "$line"
printf '%s%s\n' "$line" "${patterns[$label]+_PUT FLAG HERE}"
done > file.txt
NOTES:
The changes are written back to 'file.txt', as the question seems to specify.
Bash 4.0 or later is required for associative arrays and readarray.
Bash is very slow, so this solution may not be practical if either of the files is large (more than 10 thousand lines).