Basically what I'm trying to do is move lines 1 through 4 from A.txt
and replace the lines 5 through 8 in B.txt with them.
I figured out how to get the first four lines with sed,
but I cannot figure out how to "send" them to replace the lines in the second txt file.
cat A.txt
1 a
2 b
3 c
4 d
5 e
cat B.txt
one
two
three
four
five
six
seven
eigh
nine
Result
one
two
three
four
1 a
2 b
3 c
4 d
nine
This might work for you (GNU sed):
sed -i -e '5,8R a.txt' -e '5,8d' b.txt
for your example, this awk one-liner works too:
awk 'NR>4&&NR<9{getline $0<"a.txt"}7' b.txt
this prints the expected output, you need play with redirection if you want to save it back to b.txt.
This awk should do:
awk 'FNR==NR {a[NR]=$0;next} FNR>=5 && FNR<=8 {$0=a[FNR-4]}1' A.txt B.txt > tmp && mv tmp B.txt
It stores the lines of A.txt in an array named a
Then if line number of B.txt is between 5 and 8 replace value using info from array a
Result is stored in a temp file tmp and then moved back to B.txt
#!/usr/local/bin/bash -x
sed -n '1,4p' B.txt > B.txt.tmp
sed -n '1,4p' A.txt >> B.txt.tmp
sed -n '9p' B.txt >> B.txt.tmp
mv B.txt B.txt.bak
mv B.txt.tmp B.txt
This is static. Still, as long as you know your line addresses, this will work.
If you want support for variable span lengths, you will need to do something like this in your files:-
#----------numbers-begin----------
one
two
three
four
#----------numbers-end----------
From there, you can get to them inside the file with:-
sed -n '/--numbers-begin--/,/--numbers-end--/p' <filename> > newfile
Not only does that give you anchors to play with, but sed printing is my own preferred method of importing strings for variables in scripts, because it doesn't cause the shell to try and literally interpret the text as a command, as cat does for some reason.
The other thing that you can do in future files, is something like this:-
numbers:one
numbers:two
numbers:three
numbers:four
words:dog
words:cat
words:rat
Then:-
#!/usr/local/bin/bash
for i in $(sed -n '/^/,/$/p' file)
do
if [ $(echo ${i} | sed -n '/numbers/p' ]
then
echo ${i} | cut -d':' -f2 >> numbers-only-file
fi
done
Data structuring. It's all about the data structuring. Structure your data properly, and you will have practically no work at all.
Related
I need some assistance on the below.
File1.txt
aaa:/path/to/aaa:777
bob:/path/to/bbb:700
ccc:/path/to/ccc:600
File2.txt
aaa:/path/to/aaa:700
bbb:/path/to/bbb:700
ccc:/path/to/ccc:644
I should iterate file2.txt and if aaa exists in File1.txt, then i should compare the file permission. If the file permission is same for aaa in both the files then ignore.
If they are different then write them in the output.txt
So in above case
Output.txt
aaa:/path/to/aaa:700
ccc:/path/to/ccc:644
How can i achieve this in unix shell script? Please suggest
I agree with the comment of #Marc that you should try something before asking here.
However, the following answer is difficult to find when you never have seen the constructions, so I give you something to study.
When you want to parse line by line, you can start with
while IFS=: read -r file path mode; do
comparewith=$(grep "^${file}:${path}:" File2.txt | cut -d: -f3)
# compare and output
done < File1.txt
For large files that will become very slow.
You can first filter the lines you want to compare from File2.txt.
You want to grep strings like aaa:/path/to/aaa:, including the last :. With cut -d: -f1-2 you might be fine with your inputfile, but maybe it is better to remove the last three characters:
sed 's/...$//' File1.txt.
You can let grep use the output as a file with expressions using <():
grep -f <(sed 's/...$//' File1.txt) File2.txt
Your example files don't show the situation when both files have identical lines (that you want to skip), you will need another process substitution to get that working:
grep -v -f File1.txt <(grep -f <(sed 's/...$//' File1.txt ) File2.txt )
Another solution, worth trying yourself, is using awk (see What is "NR==FNR" in awk? for accessing 2 files).
comm - compare two sorted files line by line
According to manual, comm -13 <file1> <file2> must print only lines unique to <file2>:
$ ls
File1.txt File2.txt
$ cat File1.txt
aaa:/path/to/aaa:777
bbb:/path/to/bbb:700
ccc:/path/to/ccc:600
$ cat File2.txt
aaa:/path/to/aaa:700
bbb:/path/to/bbb:700
ccc:/path/to/ccc:644
$ comm -13 File1.txt File2.txt
aaa:/path/to/aaa:700
ccc:/path/to/ccc:644
$ # Nice!
But it doesn't check for lines in <file1> that are "similar" to corresponding lines of <file2>. I. e. it won't work as you want if File1.txt has line BOB:/path/to/BOB:700 and File2.txt has BBB:/path/to/BBB:700 since it will print the latter (while you want it not to be printed).
It also won't do what you want if strings bbb:/path/to/bbb:700 and bbb:/another/path/to/bbb:700 are supposed to be "identical".
I have two files, file1 and file2. I want to reach each line from file1, and then search if any of the lines in file2 is present in file1. I am using the following bash script, but it does not seem to be working. What should I change? (I am new to bash scripting).
#!/bin/bash
while read line1
do
echo $line1
while read line2
do
if grep -Fxq "line2" "$1"
then
echo "found"
fi
done < "$2"
done < "$1"
Note: Both files are text files.
Use grep -f
grep -f file_with_search_words file_with_content
Note however that if file_with_search_words contains blank lines everything will be matched. But that can be easily avoided with:
grep -f <(sed '/^$/d' file_with_search_words) file_with_content
From the man page:
-f FILE, --file=FILE
Obtain patterns from FILE, one per line. If this option is used
multiple times or is combined with the -e (--regexp) option, search
for all patterns given. The empty file contains zero patterns, and
therefore matches nothing.
You may use the command "comm", it compare two sorted files line-by-line
This command show the common lines in file1 and file2
comm -12 file1 file2
The only problem with this command is that you have to sort the files before, like this:
sort file1 > file1sorted
http://www.computerhope.com/unix/ucomm.htm
File 1
Line 1
Line 3
Line 6
Line 9
File 2
Line 3
Line 6
awk 'NR==FNR{con[$0];next} $0 in con{print $0}' file1 file2
will give you
Line 3
Line 6
that is the content in file 2 which is present in file1.
If you wish to ignore the spaces you can achieve with the below one.
awk 'NR==FNR{con[$0];next} !/^$/{$0 in con;print $0}' file1 file2
I have two files, file1.txt and file2.txt. Each has an identical number of lines, but some of the lines in file1.txt are empty. This is easiest to see when the content of the two files is displayed in parallel:
file1.txt file2.txt
cat bear
fish eagle
spider leopard
snail
catfish rainbow trout
snake
koala
rabbit fish
I need to assemble these files together, such that the empty lines in file1.txt are filled with the data found in the lines (of the same line number) from file2.txt. The result in file3.txt would look like this:
cat
fish
spider
snail
catfish
snake
koala
rabbit
The best I can do so far, is create a while read -r line loop, create a counter that counts how many times the while loop has looped, then use an if-conditional to check if $line is empty, then use cut to obtain the line number from file2.txt according to the number on the counter. This method seems really inefficient.
Sometimes file2.txt might contain some empty lines. If file1.txt has an empty line and file2.txt also has an empty line in the same place, the result is an empty line in file3.txt.
How can I fill the empty lines in one file with corresponding lines from another file?
paste file1.txt file2.txt | awk -F '\t' '$1 { print $1 ; next } { print $2 }'
Here is the way to handle these files with awk:
awk 'FNR==NR {a[NR]=$0;next} {print (NF?$0:a[FNR])}' file2 file1
cat
fish
spider
snail
catfish
snake
koala
rabbit
First it store every data of the file2 in array a using record number as index
Then it prints file1, bit it thest if file1 contains data for each record
If there is data for this record, then use it, if not get one from file2
One with getline (harmless in this case) :
awk '{getline p<f; print NF?$0:p; p=x}' f=file2 file1
Just for fun:
paste file1.txt file2.txt | sed -E 's/^ //g' | cut -f1
This deletes tabs that are at the beginning of a line (those missing from file1) and then takes the first column.
(For OSX, \t doesn't work in sed, so to get the TAB character, you type ctrl-V then Tab)
a solution without awk :
paste -d"#" file1 file2 | sed 's/^#\(.*\)/\1/' | cut -d"#" -f1
Here is a Bash only solution.
for i in 1 2; do
while read line; do
if [ $i -eq 1 ]; then
arr1+=("$line")
else
arr2+=("$line")
fi
done < file${i}.txt
done
for r in ${!arr1[#]}; do
if [[ -n ${arr1[$r]} ]]; then
echo ${arr1[$r]}
else
echo ${arr2[$r]}
fi
done > file3.txt
My task
I have a file A.txt with the following content.
aijdish uhuih
buh iiu hhuih
zhuh hiu
d uhiuhg ui
...
I want to select lines with these words aijdish, d, buh ...
I only know that I can:
cat A.txt | grep "aijdish" > temp.txt
cat A.txt | grep "d" >> temp.txt
cat A.txt | grep "buh" >> temp.txt
...
But I have several thousands of words need to select this time, how can I do this under bash?
Since you have many words you want to look for I suggest putting the pattern into a file and use greps -f option:
$ cat grep-pattern.txt
aijdish
buh
d
$ grep -f grep-pattern.txt inputfile
aijdish uhuih
buh iiu hhuih
d uhiuhg ui
But if you have words like d you might want to add the -w option to match only whole words and not parts of words.
grep -wf grep-pattern.txt inputfile
$ grep -E "aijdish|d|buh" inputfile
aijdish uhuih
buh iiu hhuih
d uhiuhg ui
Store the words to be searched in a file (say a.txt) and then write a script for searching every line in a.txt and matching it in the required file
I have a text file with a marker somewhere in the middle:
one
two
three
blah-blah *MARKER* blah-blah
four
five
six
...
I just need to split this file in two files, first containing everything before MARKER, and second one containing everything after MARKER. It seems it can be done in one line with awk or sed, I just can't figure out how.
I tried the easy way — using csplit, but csplit doesn't play well with Unicode text.
you can do it easily with awk
awk -vRS="MARKER" '{print $0>NR".txt"}' file
Try this:
awk '/MARKER/{n++}{print >"out" n ".txt" }' final.txt
It will read input from final.txt and produces out1.txt, out2.txt, etc...
sed -n '/MARKER/q;p' inputfile > outputfile1
sed -n '/MARKER/{:a;n;p;ba}' inputfile > outputfile2
Or all in one:
sed -n -e '/MARKER/! w outputfile1' -e'/MARKER/{:a;n;w outputfile2' -e 'ba}' inputfile
The split command will almost do what you want:
$ split -p '\*MARKER\*' splitee
$ cat xaa
one
two
three
$ cat xab
blah-blah *MARKER* blah-blah
four
five
six
$ tail -n+2 xab
four
five
six
Perhaps it's close enough for your needs.
I have no idea if it does any better with Unicode than csplit, though.