I have files 1.txt, 2.txt, 3.txt and 1-bis.txt, 2-bis.txt, 3-bis.txt
cat 1.txt
#ok
#5
6
5
cat 2.txt
#not ok
#56
13
56
cat 3.txt
#nothing
#
cat 1-bis.txt
5
4
cat 2-bis.txt
32
24
cat 3-bis.txt
I would like to add lines starting with # (from non bis files) at the beginning of files "bis" in order to get:
cat 1-bis.txt
#ok
#5
5
4
cat 2-bis.txt
#not ok
#56
32
24
cat 3-bis.txt
#nothing
#
I was thinking to use grep -P "#" to select lines with # (or maybe sed -n) but I don't know how to loop files to solve this problem
Thank you very much for your help
You can use this solution:
for f in *-bis.txt; do
{ grep '^#' "${f//-bis}"; cat "$f"; } > "$f.tmp" && mv "$f.tmp" "$f"
done
If you only want # lines at the beginning of the files only then use:
Change
grep '^#' "${f//-bis}"
with:
awk '!/^#/{exit}1' "${f//-bis}"
You can loop over the ?.txt files and use parameter expansion to derive the corresponding bis- filename:
for file in ?.txt ; do
bis=${file%.txt}-bis.txt
grep '^#' "$file" > tmp
cat "$bis" >> tmp
mv tmp "$bis"
done
You don't need grep -P, simple grep is enough. Just add ^ to only match the octothorpes at the beginning of a line.
Related
I have a line like this:
INPUT file1
How can I get bash to read that line and directly copy in the contents of "file1.txt" in place of that line? Or if it sees: INPUT file2 on a line, put in `file2.txt" etc.
The best I can do is a lot of tr commands, to paste the file together, but that seems an overly complicated solution.
'sed' also replaces lines with strings, but I don't know how to input the entire content of a file, which can be hundreds of lines into the replacement.
Seems pretty straightforward with awk. You may want to handle errors differently/more gracefully, but:
$ cat file1
Line 1 of file 1
$ cat file2
Line 1 of file 2
$ cat input
This is some content
INPUT file1
This is more content
INPUT file2
This file does not exist
INPUT file3
$ awk '$1=="INPUT" {system("cat " $2); next}1' input
This is some content
Line 1 of file 1
This is more content
Line 1 of file 2
This file does not exist
cat: file3: No such file or directory
A perl one-liner, using the CPAN module Path::Tiny
perl -MPath::Tiny -pe 's/INPUT (\w+)/path("$1.txt")->slurp/e' input_file
use perl -i -M... to edit the file in-place.
Not the most efficient possible way, but as an exercise I made a file to edit named x and a couple of input sources named t1 & t2.
$: cat x
a
INPUT t2
b
INPUT t1
c
$: while read k f;do sed -ni "/$k $f/!p; /$k $f/r $f" x;done< <( grep INPUT x )
$: cat x
a
here's
==> t2
b
this
is
file ==> t1
c
Yes, the blank lines were in the INPUT files.
This will sed your base file repeatedly, though.
The awk solution given is better, as it only reads through it once.
If you want to do this in pure Bash, here's an example:
#!/usr/bin/env bash
if (( $# < 1 )); then
echo "Usage: ${0##*/} FILE..."
exit 2
fi
for file; do
readarray -t lines < "${file}"
for line in "${lines[#]}"; do
if [[ "${line}" == "INPUT "* ]]; then
cat "${line#"INPUT "}"
continue
fi
echo "${line}"
done > "${file}"
done
Save to file and run like this: ./script.sh input.txt (where input.txt is a file containing text mixed with INPUT <file> statements).
Sed solution similar to awk given erlier:
$ cat f
test1
INPUT f1
test2
INPUT f2
test3
$ cat f1
new string 1
$ cat f2
new string 2
$ sed 's/INPUT \(.*\)/cat \1/e' f
test1
new string 1
test2
new string 2
test3
Bash variant
while read -r line; do
[[ $line =~ INPUT.* ]] && { tmp=($BASH_REMATCH); cat ${tmp[1]}; } || echo $line
done < f
I have multiple files like this:
BOB_1.brother_bob12.txt
BOB_2.brother_bob12.txt
..
BOB_35.brother_bob12.txt
How to join these files in order from {1..36} and append filename at the end of each row? I have tried:
for i in *.txt; do sed 's/$/ '"$i"'/' $i; done > outfile #joins but not in order
cat $(for((i=1;i<38;i++)); do echo -n "BOB_${i}.brother_bob12.txt "; done) # joins in order but no filename at the end
file sample:
1 345 378 1 3 4 5 C T
1 456 789 -1 2 3 4 A T
Do not do cat $(....). You may just:
for ((i=1;i<38;i++)); do
f="BOB_${i}.brother_bob12.txt"
sed "s/$/ $f/" "$f"
done
You may also do:
printf "%s\n" bob.txt BOB_{1..38}.brother_bob12.txt |
xargs -d'\n' -i sed 's/$/ {}/' '{}'
You may use:
for i in {1..36}; do
fn="BOB_${i}.brother_bob12.txt"
[[ -f $fn ]] && awk -v OFS='\t' '{print $0, FILENAME}' "$fn"
done > output
Note that it will insert FILENAME as the last field in every record. If this is not what you want then show your expected output in question.
This might work for you (GNU sed);
sed -n 'p;F' BOB_{1..36}.brother_bob12.txt | sed 'N;s/\n/ /' >newFile
Used 2 invocations of sed, the first to append the file name after each line of each file. The second to replace the newline between each 2 lines by a space.
I have 3 txt files:
file1.txt:
11
file2.txt:
22
file3.txt:
33
I want to combine the 3 text files into a single file and put a comma between them.
endfile.txt should be as follows:
11,22,33
I'd try:
cat file1.txt; cat file2.txt; cat file3.txt > endfile.txt
Wrote line by line but I want to print side by side and put a comma
Could you help?
cat file1.txt | cat - file2.txt | cat - file3.txt | tr '\n' ',' | head --bytes -1
A very easy approach simply uses printf:
(printf "%s" $(cat file1.txt); printf ",%s" $(cat file2.txt file3.txt)) > endfile.txt
Which would results in 11,22,33 in endfile.txt. The two grouping of printf were used to prevent a comma from being written before 11 and the entire line is executed as a subshell so output from all commands is redirected to endfile.txt. You may also want to write a final '\n' after file3.txt to ensure the resulting endfile.txt contains a POSIX line-ending.
My answer is following.
$ cat *.txt | sed -z 's/\n\(.\)/,\1/g'
If you define exactly order, it is following.
$ cat file{1,2,3}.txt | sed -z 's/\n\(.\)/,\1/g'
CAUTION
My sed is version 4.8.
$ sed --version | head -n 1
sed (GNU sed) 4.8
Use paste:
paste -sd, file1.txt file2.txt file3.txt > endfile.txt
I have a text file:
10 20 30 40
50 60 60 80
By using
$ wc -l file.txt
2 file.txt
I get the count, but I want to add that result in my text file.
I want the result to be like this:
2
10 20 30 40
50 60 70 80
What should I do in order to prepend the result in the text file?
I have many of these files in one folder, and instead of providing a single text file at a time, I want to provide all the files at the same time.
For one file, you can do this:
wc -l < file.txt | cat - file.txt > tmp && mv tmp file.txt
This uses cat to concatenate the result of wc -l < file.txt with the contents of file.txt. The result is written to a temporary file, then the original file is overwritten.
For many files (e.g. all files ending in .txt), you can use a loop:
for file in *.txt; do
wc -l < "$file" | cat - "$file" > tmp && mv tmp "$file"
done
Try:
echo `wc -l myfile.txt` | cat - myfile.txt > tmp && mv tmp myfile.txt
Awk may be your friend :
awk 'BEGIN{RS="^$"}NR=FNR{printf "%d\n%s",gsub(/\n/,"\n"),$0}' file \
> temp && mv temp file
I have n files, like:
file1:
1aaa
2eee
Test XXX
Hanna
Lars
file2:
1fff
2ddd
3zzz
Test XXX
Mike
Charly
I want to remove all rows before "Test XXX" from all n files.
The number of rows to delete varies between files.
My idea:
for file in 1 :n
do
pos=grep -n "Test XXX" file$file
sed -i "1:$pos-1 d" file$file >new$file
done
This should work for you:
sed -i '1,/Test XXX/d' file1
sed -i '1,/Test XXX/d' file2
or simply
sed -i '1,/Test XXX/d' file*
This will work for your examples and even if the matched pattern is on the very first line:
sed -n -E -e '/Text XXX/,$ p' input.txt | sed '1 d'
For example if you input is simply
Test XXX
Mike
Charly
This will give you
Mike
Charly
If you want to keep the first match Test XXX then just use:
sed -n -E -e '/Text XXX/,$ p' input.txt
You can do it with bash ( eg for 1 file)
t=0
while read -r line
do
[[ $line =~ Test.*XXX ]] && t="1"
case "$t" in
1) echo "$line";;
esac
done < file > tempo && mv tempo file
Use a for loop as necessary to go through all the files
cat <<-EOF > file1.txt
1aaa
2eee
Test XXX
Hanna
Lars
EOF
cat file1.txt | sed -e '/Test *XXX/p' -e '0,/Test *XXX/d'
Output:
Test XXX
Hanna
Lars
Explanation:
-e '/Test *XXX/p' duplicates the line matching /Test *XXX/
-e '0,/Test *XXX/d' deletes from line 0 to the first line matching /Test *XXX/
By duplicating the line, then removing the first one, we effectively retain the matched line, successfully deleting all lines BEFORE Test XXX
Note: this will not work as expected if there are multiple Test XXX lines.