I want to grep multiple files in a directory and collect the output of each grep in a separate file ..So if I grep 20 files, I should get 20 output-files which contain the searched item. Can anybody help me with this? Thanks.
Use a for statement:
for a in *.txt; do grep target $a >$a.out; done
just one gawk command
gawk '/target/ {print $0 > FILENAME".out"}' *.txt
you can use just the shell, no need external commands
for file in *.txt
do
while read -r line
do
case "$line" in
*pattern*) echo $line >> "${file%.txt}.out";;
esac
done < "$file"
done
Related
I have a file which consists of number of users which I need to compare it with multiple files and print if any particular user is not present in the files with filename.
#!/bin/bash
awk '{print $1}' $1 | while read -r line; do
if ! grep -q "$line" *.txt;
then
echo "$line User doesn't exist"
fi
done
In the above script, passing user_list file as $1 and can able to find the users for single target file, but it fails for multiple files.
File contents:
user_list:
Johnny
Stella
Larry
Jack
One of the multiple files contents:
root:x:0:0:root:/root:/bin/bash
Stella:x:1:1:Admin:/bin:/bin/bash
Jack:x:2:2:admin:/sbin:/bin/bash
Usage:
./myscript user_list.txt
Desired output:
File1:
Stella doesn't exist
Jack doesn't exist
File2:
Larry doesn't exist
Johnny doesn't exist
Any suggestion here to achieve it for multiple files with printing filename headers?
Use a for loop to iterate over each file and execute your code for each file separately.
#!/bin/bash
for f in *.txt; do
echo $f:
awk '{print $1}' $1 | while read -r line; do
if ! grep -q "$line" $f
then
echo "$line doesn't exist"
fi
done
echo
done
You could do simply
for file in *.txt; do
echo "$file"
grep -Fvf "$file" "$1"
echo
done
This might do what you want.
#!/usr/bin/env bash
for f in *.txt; do ##: Loop through the *.txt files
for j; do ##: Loop through the argument files, file1 file2 file3
printf '\n%s:\n' "$j" ##: Print one of the multiple files.
while read -r lines; do ##: Read line-by-line & one-by-one all the *.txt files
if ! grep -q "$lines" "$j"; then ##: If grep did not found a match.
printf "%s not doesn't exist.\n" "$lines" ##: Print the desired output.
fi
done < "$f"
done
done
The *.txt files should be in the current directory, otherwise add the absolute path, e.g. /path/to/files/*.txt
Howto use.
./myscript file1 file2 file3 ...
The downside is you're running grep line-by-line on each files as opposed to what #Quasimodo did.
I have been looking around trying to combine multiple text files into including the name of the file.
My current file content is:
1111,2222,3333,4444
What I'm after is:
File1,1111,2222,3333,4444
File1,1111,2222,3333,4445
File1,1111,2222,3333,4446
File1,1111,2222,3333,4447
File2,1111,2222,3333,114444
File2,1111,2222,3333,114445
File2,1111,2222,3333,114446
I found multiple example how to combine them all but nothing to combine them including the file name.
Could you please try following. Considering that your Input_file names extensions are .csv.
awk 'BEGIN{OFS=","} {print FILENAME,$0}' *.csv > output_file
After seeing OP's comments if file extensions are .txt then try:
awk 'BEGIN{OFS=","} {print FILENAME,$0}' *.txt > output_file
Assuming all your files have a .txt extension and contain only one line as in the example, you can use the following code:
for f in *.txt; do echo "$f,$(cat "$f")"; done > output.log
where output.log is the output file.
Well, it works:
printf "%s\n" *.txt |
xargs -n1 -d $'\n' bash -c 'xargs -n1 -d $'\''\n'\'' printf "%s,%s\n" "$1" <"$1"' --
First output a newline separated list of files.
Then for each file xargs execute sh
Inside sh execute xargs for each line of file
and it executes printf "%s,%s\n" <filename> for each line of input
Tested in repl.
Solved using grep "" *.txt -I > $filename.
I am relatively new to bash and I am testing my code for the first case.
counter=1
for file in not_processed/*.txt; do
if [ $counter -le 1 ]; then
grep -v '2018-07' $file > bis.txt;
counter=$(($counter+1));
fi;
done
I want to subtract all the lines containing '2018-07' from my file. The new file needs to be named $file_bis.txt.
Thanks
With sed or awk it's much easier and faster to process complex files.
sed -n '/2018-07/p' not_processed/*.txt
then you get the output in your console. If you want you can pipe the output to a new file.
sed -n '/2018-07/p' not_processed/*.txt >> out.txt
This is to do it on all files in not_processed/*.txt
for file in not_processed/*.txt
do
grep -v '2018-07' $file > "$file"_bis.txt
done
And this is to do it only on the first 2 files in not_processed/*.txt
for file in $(ls not_processed/*.txt|head -2)
do
grep -v '2018-07' $file > "$file"_bis.txt
done
Don't forget to add "" on $file, because otherwise bash considers $file_bis as a new variable, which has no assigned value.
I don't understood why you are using a counter and if condition for this simple requirement. Use below script which will fulfill you requirement:-
#first store all the files in a variable
files=$(ls /your/path/*.txt)
# now use a for loop
for file in $files;
do
grep '2018-07' $file >> bis.txt
done
Better avoid for loop here as below single line is suffice
grep -h '2018-07' /your/path/*.txt > bis.txt
Here is what I have and not working:
for i in `cat cnames.csv`
do nslookup $i | grep -v "8.8.8.8\|=\|Non-authoritative" >> output.txt
done
Any better solutions?
This is Bash FAQ 001; you don't iterate over a file using a for loop.
while IFS= read -r i; do
nslookup "$i"
done < cnames.csv | grep -v "8.8.8.8\|=\|Non-authoritative" > output.txt
Note that you don't need to run grep separate for each call to nslookup; you can pipe the aggregate output to a single call.
You can use the exit status of nslookup.
for i in $(cat cnames.csv); do
if nslookup "$i"; then
echo "$i is valid"
else
echo "$i not found"
fi
done
Is cnames.csv a real .csv file? Wouldn't that require to extract only the column with addresses in them? Right now the commas and other fields (if existing) are read too.
You could probably get them all looked up faster in parallel and more succinctly with GNU Parallel
parallel -a cnames.csv nslookup {} | grep ...
I have three files in a directory that has the structure like this:
file.exe.trace, file.exe.trace.functions and file.exe.trace.netlog
I want to know how can I get file.exe as file name?
In other world I need to get file name that has the .trace extension? I should note that as you can see all the files has the .trace part.
If $FILENAME has the name, the root part can be gotten from ${FILENAME%%.trace*}
for FILENAME in *.trace; do
echo ${FILENAME%%.trace*}
done
You can also use basename:
for f in *.trace; do
basename "$f" ".trace"
done
Update: The previous won't process files with extra extensions besides .trace like .trace.functions, but the following sed will do:
sed -r 's_(.*)\.trace.*_\1_' <(ls -c1)
You can also use it in a for loop instead:
for f in *.trace*; do
sed -r 's_(.*)\.trace.*_\1_' <<< "$f"
done
Try:
for each in *exe*trace* ; do echo $each | awk -F. '{print $1"."$2}' ; done | sort | uniq