I have a file which consists of number of users which I need to compare it with multiple files and print if any particular user is not present in the files with filename.
#!/bin/bash
awk '{print $1}' $1 | while read -r line; do
if ! grep -q "$line" *.txt;
then
echo "$line User doesn't exist"
fi
done
In the above script, passing user_list file as $1 and can able to find the users for single target file, but it fails for multiple files.
File contents:
user_list:
Johnny
Stella
Larry
Jack
One of the multiple files contents:
root:x:0:0:root:/root:/bin/bash
Stella:x:1:1:Admin:/bin:/bin/bash
Jack:x:2:2:admin:/sbin:/bin/bash
Usage:
./myscript user_list.txt
Desired output:
File1:
Stella doesn't exist
Jack doesn't exist
File2:
Larry doesn't exist
Johnny doesn't exist
Any suggestion here to achieve it for multiple files with printing filename headers?
Use a for loop to iterate over each file and execute your code for each file separately.
#!/bin/bash
for f in *.txt; do
echo $f:
awk '{print $1}' $1 | while read -r line; do
if ! grep -q "$line" $f
then
echo "$line doesn't exist"
fi
done
echo
done
You could do simply
for file in *.txt; do
echo "$file"
grep -Fvf "$file" "$1"
echo
done
This might do what you want.
#!/usr/bin/env bash
for f in *.txt; do ##: Loop through the *.txt files
for j; do ##: Loop through the argument files, file1 file2 file3
printf '\n%s:\n' "$j" ##: Print one of the multiple files.
while read -r lines; do ##: Read line-by-line & one-by-one all the *.txt files
if ! grep -q "$lines" "$j"; then ##: If grep did not found a match.
printf "%s not doesn't exist.\n" "$lines" ##: Print the desired output.
fi
done < "$f"
done
done
The *.txt files should be in the current directory, otherwise add the absolute path, e.g. /path/to/files/*.txt
Howto use.
./myscript file1 file2 file3 ...
The downside is you're running grep line-by-line on each files as opposed to what #Quasimodo did.
Related
I'm trying to create a bash script that first looks for a name and then checks whether a certain filename, for example vacation021.jpg exists in the file system, and if it exists I want to append the filename to a .txt file.
I'm having a lot of issues with this, I'm still very new to bash.
This is as far as I've gotten.
> oldFiles.txt
files=$(grep "jane " list.txt)
for i in $files; do
if test -e vacation021.jpg;
then echo $i >> oldFiles.txt; fi
done
This however appends all the separate words in the list.txt to the oldFiles.txt.
Any help would be much appreciated.
for i in $files will iterate over each word in $files, not the lines. If you want to iterate over the lines, pipe the output of grep to the loop:
grep 'jane ' list.txt | while read -r i; do
if test -e vacation021.jpg
then printf "%s" "%i"
fi
done > oldFiles.txt
But as mentioned in the comments, unless the vacation021.jpg file is going to be created or deleted during the loop, you can simply use a single if:
if test -e vacation021.jpg
then
grep 'jane ' list.txt > oldFiles.txt
fi
I want to grep lines from a variable number of log files and connect their outputs with paste. If I had a fixed number of outputs, I could do it thus:
paste <(grep $PATTERN $FILE1) <(grep $PATTERN $FILE2)
But is there a way to do this with a variable number of input files? I want to write a shell script whose arguments are the input files. The shell script should paste the grepped lines from ALL of them.
Use explicit named pipes, instead of process substitution.
pipes=()
for f in "$FILE1" "$FILE2" "$FILE3"; do
n="$(mktemp)" # Or some other command to create a temporary name
mkfifo "$n"
pipes+=( "$n" )
grep "$PATTERN" "$f" > "$n" &
done
paste "${pipes[#]}"
rm "${pipes[#]}" # When done with them
You can do this by combining find command to list the files and piping its output to grep usings xargs to ensure grep is applied on each file listed in find command
$ find /dir/containing/files -name "file.*" | xargs grep $PATTERN
I need to output all lines in file1.txt, except those that are found in file2.txt, by matching the entirely lines.
E.g., file1.txt:
Cats eat fish.
Mice eat cheese.
Eagles can fly.
Mountains are tall.
E.g., file2.txt:
Cats eat fish.
Birds can fly.
Trees are tall.
E.g., output:
Mice eat cheese.
Eagles fly.
Mountains are tall.
I have used the following command:
grep -v -x -f file1.txt file2.txt
This appears to work, however, when the files are of certain lengths, it often reports grep: memory exhausted, so I need an alternative that will not create this memory problem.
The order of the lines is important, so they should not be sorted.
Any tool typically found within a default Linux install will be acceptable.
How can I output the lines of file1.txt, except those found in file2.txt, without encountering a memory problem?
Try:
grep -Fxvf file2.txt file1.txt
References: find difference between two text files with one item per line
try:
rm -f out.txt && while read -r line; do echo "checking if line $line exists in file2.txt"; if `grep -Fxq "$line" file2.txt`; then echo "$line exists in other for"; else echo "$line" >> out.txt; fi; done < file.txt
Explaination:
this deletes the output file (in case of continuous use...), then it checks line by line if one exists in the other.
As a bash file it's clearer:
rm out.txt
while read -r line
do
echo "checking if line $line exists in file2.txt"
if `grep -Fxq "$line" file2.txt`
then
echo "$line exists in other file"
else
echo "$line" >> out.txt
fi
done < file.txt
And the obvious generalization:
while read -r line
do
echo "checking if line $line exists in $2"
if `grep -Fxq "$line" $2`
then
echo "$line exists in $2"
else
echo "$line" >> $3
fi
done < $1
Where the first and second arguments would be the files and the outut file would be the third argument
I would like to print all file names in a folder.How can I do this with awk or bash?
ls -l /usr/bin | awk '{ print $NF }'
:)
find . -type f -maxdepth 1
exclude the -maxdepth part if you want to also do it recursively for subdirectories
Following is a pure-AWK option:
gawk '
BEGINFILE {
print "Processing " FILENAME
};
' *
It may be helpful if you want to use it as part of a bigger AWK script processing multiple files and you want to log which file is currently being processed.
This command will print all the file names:
for f in *; do echo "$f"; done
or (even shorter)
printf '%s\n' *
Alternatively, if you like to print specific file types (e.g., .txt), you can try this:
for f in *.txt; do echo "$f"; done
or (even shorter)
printf '%s\n' *.txt
/bin/ls does this job for you and you may call it from bash.
$> /bin/ls
[.. List of files..]
Interpreting your question you might be interested in iterating over every single file in this directory. This can be done using bash as well:
for f in `ls`; do
echo $f;
done
for f in *; do var=`find "$f" | wc -l`; echo "$f: $var"; done
This will print name of the directory and number of files in it. wc -l here returns count of files +1 (Including directory)
Sample output:
aa: 4
bb: 4
cc: 1
test2.sh: 1
I want to grep multiple files in a directory and collect the output of each grep in a separate file ..So if I grep 20 files, I should get 20 output-files which contain the searched item. Can anybody help me with this? Thanks.
Use a for statement:
for a in *.txt; do grep target $a >$a.out; done
just one gawk command
gawk '/target/ {print $0 > FILENAME".out"}' *.txt
you can use just the shell, no need external commands
for file in *.txt
do
while read -r line
do
case "$line" in
*pattern*) echo $line >> "${file%.txt}.out";;
esac
done < "$file"
done