I have two files and would like to insert the contents of one file into the other, replacing a specified line.
File 1:
abc
def
ghi
jkl
File 2:
123
The following code is what I have.
file1=numbers.txt
file2=letters.txt
linenumber=3s
echo $file1
echo $file2
sed "$linenumber/.*/r $file1/" $file2
Which results in the output:
abc
def
r numbers.txt
jkl
The output I am hoping for is:
abc
def
123
jkl
I thought it could be an issue with bash variables but I still get the same output when I manually enter the information.
How am I misunderstanding sed and/or the read command?
Your script replaces the line with the string "r $file1". The part in sed in s command is not re-interpreted as a command, but taken literally.
You can:
linenumber=3
sed "$linenumber"' {
r '"$file1"'
d
}' "$file2"
Read line number 3, print file1 and then delete the line.
See here for a good explanation and reference.
Surely we can make that a oneliner:
sed -e "$linenumber"' { r '"$file2"$'\n''d; }' "$file1"
Life example at tutorialpoints.
I would use the c command as follows:
linenumber=3
sed "${linenumber}c $(< $file1)" "$file2"
This replaces the current line with the text that comes after c.
Your command didn't work because it expands to this:
sed "3s/.*/r numbers.txt/" letters.txt
and you can't use r like that. r has to be the command that is being run.
Related
I have two files. One file contains a pattern that I want to match in a second file. I want to use that pattern to print between that pattern (included) up to a specified character (not included) and then concatenate into a single output file.
For instance,
File_1:
a
c
d
and File_2:
>a
MEEL
>b
MLPK
>c
MEHL
>d
MLWL
>e
MTNH
I have been using variations of this loop:
while read $id;
do
sed -n "/>$id/,/>/{//!p;}" File_2;
done < File_1
hoping to obtain something like the following output:
>a
MEEL
>c
MEHL
>d
MLWL
But have had no such luck. I have played around with grep/fgrep awk and sed and between the three cannot seem to get the right (or any output). Would someone kindly point me in the right direction?
Try:
$ awk -F'>' 'FNR==NR{a[$1]; next} NF==2{f=$2 in a} f' file1 file2
>a
MEEL
>c
MEHL
>d
MLWL
How it works
-F'>'
This sets the field separator to >.
FNR==NR{a[$1]; next}
While reading in the first file, this creates a key in array a for every line in file file.
NF==2{f=$2 in a}
For every line in file 2 that has two fields, this sets variable f to true if the second field is a key in a or false if it is not.
f
If f is true, print the line.
A plain (GNU) sed solution. Files are read only once. It is assumed that characters in File_1 needn't to be quoted in sed expression.
pat=$(sed ':a; $!{N;ba;}; y/\n/|/' File_1)
sed -E -n ":a; /^>($pat)/{:b; p; n; /^>/ba; bb}" File_2
Explanation:
The first call to sed generates a regular expression to be used in the second call to sed and stores it in the variable pat. The aim is to avoid reading repeatedly the entire File_2 for each line of File_1. It just "slurps" the File_1 and replaces new-line characters with | characters. So the sample File_1 becomes a string with the value a|c|d. The regular expression a|c|d matches if at least one of the alternatives (a, b, c for this example) matches (this is a GNU sed extension).
The second sed expression, ":a; /^>($pat)/{:b; p; n; /^>/ba; bb}", could be converted to pseudo code like this:
begin:
read next line (from File_2) or quit on end-of-file
label_a:
if line begins with `>` followed by one of the alternatives in `pat` then
label_b:
print the line
read next line (from File_2) or quit on end-of-file
if line begins with `>` goto label_a else goto label_b
else goto begin
Let me try to explain why your approach does not work well:
You need to say while read id instead of while read $id.
The sed command />$id/,/>/{//!p;} will exclude the lines which start
with >.
Then you might want to say something like:
while read id; do
sed -n "/^>$id/{N;p}" File_2
done < File_1
Output:
>a
MEEL
>c
MEHL
>d
MLWL
But the code above is inefficient because it reads File_2 as many times as the count of the id's in File_1.
Please try the elegant solution by John1024 instead.
If ed is available, and since the shell is involve.
#!/usr/bin/env bash
mapfile -t to_match < file1.txt
ed -s file2.txt <<-EOF
g/\(^>[${to_match[*]}]\)/;/^>/-1p
q
EOF
It will only run ed once and not every line that has the pattern, that matches from file1. Like say if you have a to z from file1,ed will not run 26 times.
Requires bash4+ because of mapfile.
How it works
mapfile -t to_match < file1.txt
Saves the entry/value from file1 in an array named to_match
ed -s file2.txt point ed to file2 with the -s flag which means don't print info about the file, same info you get with wc file
<<-EOF A here document, shell syntax.
g/\(^>[${to_match[*]}]\)/;/^>/-1p
g means search the whole file aka global.
( ) capture group, it needs escaping because ed only supports BRE, basic regular expression.
^> If line starts with a > the ^ is an anchor which means the start.
[ ] is a bracket expression match whatever is inside of it, in this case the value of the array "${to_match[*]}"
; Include the next address/pattern
/^>/ Match a leading >
-1 go back one line after the pattern match.
p print whatever was matched by the pattern.
q quit ed
I am leaning how to use awk and sed this week. I know this question might have been asked before but I am not sure what is wrong with my script. I have three files and I am using grep to search for the pattern gge0001x gge0001y gge0001z. x is in file1, y is in file 2, and z is in file3. If anyone wants to see L2E[1-3].iva they are here: https://gist.github.com/anonymous/1112988408874c730cd4f3d313226ba4
#!/bin/bash
echo "Performance Data"
sed -n '1,19p' L2E1.iva|cat > file1 #take lines 1-19 in L2E1 and take the
# output into file1. The next two commands do the same thing
sed -n '1,19p' L2E2.iva|cat > file2
sed -n '1,19p' L2E3.iva|cat > file3
curveName=`grep "F" file1|sed "s/F/ /"`
# This will search for F in file 1, and then substitute F with a space
curveName2=`grep "F" file2|sed "s/F/ /"`
curveName3=`grep "F" file3|sed "s/F/ /"`
echo "Curve Name" "$curveName $curveName2 $curveName3"
I want my output to be Curve Name gge0001x gge0001y gge0001z. But the output is this instead:
Performance Data gge0006ze gge0006x
If I echo them out by themselves then it is fine, but once I echo all three on the same line the output gets skewed. Why does x show up last when it is first when I echo it and where did my y go to?
A few tips at first:
sed -n '1,19p' L2E1.iva|cat > file1
You can omit the cat an redirect the output of sed directly to the file:
sed -n '1,19p' L2E1.iva > file1
curveName=`grep "F" file1|sed "s/F/ /"`
Use $() instead of backticks for process substitution:
curveName=$(grep "F" file1 | sed "s/F/ /")
But the output is this instead: Performance Data gge0006ze gge0006x
The reason for Performance Data in your output is, that you echo it at the beginning of your script.
Moreover you've got a typo in the last echo: $curvename2 -> $curveName2, this is why your y is missing.
Did you double check your files for the right contents? That's the only reason i can imagine, why your x comes last and the z first.
You can perhaps compress your script into one line
$ echo Curve name $(grep -Pohm1 '(?<=F ).*' L2E{1..3})
Curve name gge0006x gge0006y gge0006z
exercise is to search the options used in grep
There are many examples here and elsewhere on the interwebs for using sed's 'r' to replace a pattern, but it does not seem to work on a range, but maybe I'm just not holding it right.
The following works as expected, deleting BEGIN PATTERN and replacing it with the contents of /tmp/somefile.
sed -n "/BEGIN PATTERN/{ r /tmp/somefile d }" TARGET_FILE
This, however, only replaces END_PATTERN with the contents of /tmp/somefile.
sed -n "/BEGIN PATTERN/,/END PATTERN/ { r /tmp/somefile d }" TARGET_FILE
I suppose I could try perl or awk to do this as well, but it seems like sed should be able to do this.
I believe that this does what you want:
sed $'/BEGIN PATTERN/r somefile\n /BEGIN PATTERN/,/END PATTERN/d' file
Or:
sed -e '/BEGIN PATTERN/r somefile' -e '/BEGIN PATTERN/,/END PATTERN/d' file
How it works
/BEGIN PATTERN/r somefile
Whenever BEGIN PATTERN is found, this inserts the contents of somefile.
/BEGIN PATTERN/,/END PATTERN/d
Whenever we are in the range from a line with /BEGIN PATTERN/ to a line with /END PATTERN/, we delete (d) the contains of the pattern buffer.
Example
Let's consider these two test files:
$ cat file
prelude
BEGIN PATTERN
middle
END PATTERN
afterthought
and:
$ cat somefile
This is
New.
Our command produces:
$ sed $'/BEGIN PATTERN/r somefile\n /BEGIN PATTERN/,/END PATTERN/d' file
prelude
This is
New.
afterthought
This might work for you (GNU sed):
sed -e '/BEGIN PATTERN/,/END PATTERN/{/END PATTERN/!d;r somefile' -e 'd}' file
John1024's answer works if BEGIN PATTERN and END PATTERN are different. If this is not the case, the following works:
sed $'/PATTERN/,/PATTERN/d; 1,/PATTERN/ { /PATTERN/r somefile\n }' file
By preserving the pattern:
sed $'/PATTERN/,/PATTERN/ { /PATTERN/!d; }; 1,/PATTERN/ { /PATTERN/r somefile\n }' file
This solution can yield false positives if the pattern is not paired as potong pointed out.
So I grep for something in some file:
grep "import" test.txt | tail -1
In test.txt there is
import-one
import-two
import-three
some other stuff in the file
This will return the last search result:
import-three
Now how do I add some text -after-- import-three but before "some other stuff in the file". Basically I want to append a line but not at the end of a file but after a search result.
I understand that you want some text after each search result, which would mean after every matching line. So try
grep "import" test.txt | sed '/$/ a\Line to be added'
You can try something like this with sed
sed '/import-three/ a\
> Line to be added' t
Test:
$ sed '/import-three/ a\
> Line to be added' t
import-one
import-two
import-three
Line to be added
some other stuff in the file
One way assuming that you cannot distingish between different "import" sentences. It reverses the file with tac, then find the first match (import-three) with sed, insert a line just before it (i\) and reverse again the file.
The :a ; n ; ba is a loop to avoid processing again the /import/ match.
The command is written throught several lines because the sed insert command is very special with the syntax:
$ tac infile | sed '/import/ { i\
"some text"
:a
n
ba }
' | tac -
It yields:
import-one
import-two
import-three
"some text"
some other stuff in the file
Using ed:
ed test.txt <<END
$
?^import
a
inserted text
.
w
q
END
Meaning: go to the end of the file, search backwards for the first line beginning with import, add the new lines below (insertion ends with a "." line), save and quit
I would like to insert lines into a file in bash starting from a specific line.
Each line is a string which is an element of an array
line[0]="foo"
line[1]="bar"
...
and the specific line is 'fields'
file="$(cat $myfile)"
for p in $file; do
if [ "$p" = 'fields' ]
then insertlines() #<- here
fi
done
This can be done with sed: sed 's/fields/fields\nNew Inserted Line/'
$ cat file.txt
line 1
line 2
fields
line 3
another line
fields
dkhs
$ sed 's/fields/fields\nNew Inserted Line/' file.txt
line 1
line 2
fields
New Inserted Line
line 3
another line
fields
New Inserted Line
dkhs
Use -i to save in-place instead of printing to stdout
sed -i 's/fields/fields\nNew Inserted Line/'
As a bash script:
#!/bin/bash
match='fields'
insert='New Inserted Line'
file='file.txt'
sed -i "s/$match/$match\n$insert/" $file
Or anoter one example with the sed:
Prepare a test.txt file:
echo -e "line 1\nline 2\nline 3\nline 4" > /tmp/test.txt
cat /tmp/test.txt
line 1
line 2
line 3
line 4
Add a new line into the test.txt file:
sed -i '2 a line 2.5' /tmp/test.txt
# sed for in-place editing (-i) of the file: 'LINE_NUMBER a-ppend TEXT_TO_ADD'
cat /tmp/test.txt
line 1
line 2
line 2.5
line 3
line 4
This is definitely a case where you want to use something like sed (or awk or perl) rather than readling one line at a time in a shell loop. This is not the sort of thing the shell does well or efficiently.
You might find it handy to write a reusable function. Here's a simple one, though it won't work on fully-arbitrary text (slashes or regular expression metacharacters will confuse things):
function insertAfter # file line newText
{
local file="$1" line="$2" newText="$3"
sed -i -e "/^$line$/a"$'\\\n'"$newText"$'\n' "$file"
}
Example:
$ cat foo.txt
Now is the time for all good men to come to the aid of their party.
The quick brown fox jumps over a lazy dog.
$ insertAfter foo.txt \
"Now is the time for all good men to come to the aid of their party." \
"The previous line is missing 'bjkquvxz.'"
$ cat foo.txt
Now is the time for all good men to come to the aid of their party.
The previous line is missing 'bjkquvxz.'
The quick brown fox jumps over a lazy dog.
$
sed is your friend:
:~$ cat text.txt
foo
bar
baz
~$
~$ sed '/^bar/\na this is the new line/' text.txt > new_text.txt
~$ cat new_text.txt
foo
bar
this is the new line
baz
~$