Compare files with Bash - bash

Question

I'm trying to compare Bro/Zeek logs against a second file to determine if IP addresses or domain names from second file exist in the zeek logs. I want to be able to pass conn/dns.log as a parameter (compressed/uncompressed) to the script and have it parsed with duplicates removed and compared to the second file as a second parameter. The final result should only show the file name and the matching IP/Domains between the two files.
I've made an attempt below to accomplish this however,I can only cut successfully I see the sort isn't working as I'm still getting duplicates and I'm not sure how to do the comparison against the second parameter.
If there is a better or more efficient way I'm all for it. Thanks.
compare.sh <conn.log/dns.log> indicators.txt
#!/bin/bash
# Compare files to see if they have matching strings.
clog=conn.log
dlog=dns.log
if [ $1 == $clog ]
then
cut -f3 $1;cut -f5 $1 | sort -u | grep -Fwf $2
echo "We have a match in $1"
elif
[ $1 == $dlog ]
then
cut -f10 $1|sort -u|grep -Fwf $2
echo "We have a match in $1"
else
echo "No matches"
fi
echo "Comparison complete"
Below is some example data and expected output:
Example: conn.log
1.2.3.4 1.2.3.5
172.3.2.4 10.2.20.50
...
Example: indicators
1.2.3.4
10.20.20.50
172.3.2.4
...
Expected Output:
1.2.3.4
172.3.2.4
We have a match in conn.log

Answer 1

For conn.log you're only piping the second cut to sort and grep. You need to group the two cut commands to get both of them piped.
{ cut -f3 "$1";cut -f5 "$1"; } | sort -u | grep -Fwf "$2"
Another option would be to use grep instead of running cut twice:
awk -F'\t' '{print $3; print $5}' "$1" | sort -u | grep -Fwf "$2"