Right now i a have a working script to pass 2 arguments to a shell script. The script basically takes a ticket# and svn URL as arguments on command line and gives an output of all the revisions that have been changed associated with that ticket# (in svn comments).
#!/bin/sh
jira_ticket=$1
src_url=$2
revs=(`svn log $2 --stop-on-copy | grep -B 2 $1 | grep "^r" | cut -d"r" -f2 | cut -d" " -f1| sort`)
for revisions in ${!revs[*]}
do
printf "%s %s\n" ${revs[$revisions]}
done
Output:
4738
4739
4743
4744
4745
I need some help to pass an array of arguments - meaning more than one ticket# and give the output of revisions associated with those ticket numbers that get passed as args to the script.
I don't think POSIX shell has arrays, so be plain and use #!/bin/bash
I would put the url as the first arg, and all the reset are tickets
#!/bin/bash
revs=()
src_url=$1
svn_log=$(svn log "$src_url" --stop-on-copy)
shift
for jira_ticket in "$#"; do
revs+=( $(grep -B 2 "$jira_ticket" <<< "$svn_log" | grep "^r" | cut -d"r" -f2 | cut -d" " -f1) )
done
for revisions in $( printf "%s\n" "${!revs[#]}" | sort )
do
printf "%s %s\n" ${revs[$revisions]}
done
Related
I'm trying to compare Bro/Zeek logs against a second file to determine if IP addresses or domain names from second file exist in the zeek logs. I want to be able to pass conn/dns.log as a parameter (compressed/uncompressed) to the script and have it parsed with duplicates removed and compared to the second file as a second parameter. The final result should only show the file name and the matching IP/Domains between the two files.
I've made an attempt below to accomplish this however,I can only cut successfully I see the sort isn't working as I'm still getting duplicates and I'm not sure how to do the comparison against the second parameter.
If there is a better or more efficient way I'm all for it. Thanks.
compare.sh <conn.log/dns.log> indicators.txt
#!/bin/bash
# Compare files to see if they have matching strings.
clog=conn.log
dlog=dns.log
if [ $1 == $clog ]
then
cut -f3 $1;cut -f5 $1 | sort -u | grep -Fwf $2
echo "We have a match in $1"
elif
[ $1 == $dlog ]
then
cut -f10 $1|sort -u|grep -Fwf $2
echo "We have a match in $1"
else
echo "No matches"
fi
echo "Comparison complete"
Below is some example data and expected output:
Example: conn.log
1.2.3.4 1.2.3.5
172.3.2.4 10.2.20.50
...
Example: indicators
1.2.3.4
10.20.20.50
172.3.2.4
...
Expected Output:
1.2.3.4
172.3.2.4
We have a match in conn.log
For conn.log you're only piping the second cut to sort and grep. You need to group the two cut commands to get both of them piped.
{ cut -f3 "$1";cut -f5 "$1"; } | sort -u | grep -Fwf "$2"
Another option would be to use grep instead of running cut twice:
awk -F'\t' '{print $3; print $5}' "$1" | sort -u | grep -Fwf "$2"
I'm working on a shell script.
OUT=$1
here, the OUT variable is my filename.
I'm using grep search as follows:
l=`grep "$pattern " -A 15 $OUT | grep -w $i | awk '{print $8}'|tail -1 | tr '\n' ','`
The issue is that the filename parameter I must pass is test.log.However, I have the folder structure :
test.log
test.log.001
test.log.002
I would ideally like to pass the filename as test.log and would like it to search it in all log files.I know the usual way to do is by using test.log.* in command line, but I'm facing difficulty replicating the same in shell script.
My efforts:
var-$'.*'
l=`grep "$pattern " -A 15 $OUT$var | grep -w $i | awk '{print $8}'|tail -1 | tr '\n' ','`
However, I did not get the desired result.
Hopefully this will get you closer:
#!/bin/bash
for f in "${1}*"; do
grep "$pattern" -A15 "$f"
done | grep -w $i | awk 'END{print $8}'
I'm trying to print domain and topLeveldomain variables (example.com)
$line = example.com
domain =$line | cut -d. -f 1
topLeveldomain = $line | cut -d. -f 2
However when I try and echo $domain, it doesn't display desired value
test.sh: line 4: domain: command not found
test.sh: line 5: topLeveldomain: command not found
I suggest:
line="example.com"
domain=$(echo "$line" | cut -d. -f 1)
topLeveldomain=$(echo "$line" | cut -d. -f 2)
The right code for this should be:
line="example.com"
domain=$(echo "$line" | cut -d. -f 1)
topLeveldomain=$(echo "$line" | cut -d. -f 2)
Consider the right syntax of bash:
variable=value
(there are no blanks allowed)
if you want to use the content of the variable you have to add a leading $
e.g.
echo $variable
You don't need external tools for this, just do this in bash
$ string="example.com"
# print everything upto first de-limiter '.'
$ printf "${string%%.*}\n"
example
# print everything after first de-limiter '.'
$ printf "${string#*.}\n"
com
Remove spaces around =:
line=example.com # YES
line = example.com # NO
When you create a variable, do not prepend $ to the variable name:
line=example.com # YES
$line=example.com # NO
When using pipes, you need to pass standard output to the next command. Than means, you usually need to echo variables or cat files:
echo $line | cut -d. -f1 # YES
$line | cut -d. -f1 # NO
Use the $() syntax to get the output of a command into a variable:
new_variable=$(echo $line | cut -d. -f1) # YES
new_variable=echo $line | cut -d. -f1 # NO
I would rather use AWK:
domain="abc.def.hij.example.com"
awk -F. '{printf "TLD:%s\n2:%s\n3:%s\n", $NF, $(NF-1), $(NF-2)}' <<< "$domain"
Output
TLD:com
2:example
3:hij
In the command above, -F option specifies the field separator; NF is a built-in variable that keeps the number of input fields.
Issues with Your Code
The issues with your code are due to invalid syntax.
To set a variable in the shell, use
VARNAME="value"
Putting spaces around the equal sign will cause errors. It is a good
habit to quote content strings when assigning values to variables:
this will reduce the chance that you make errors.
Refer to the Bash Guide for Beginners.
this also works:
line="example.com"
domain=$(echo $line | cut -d. -f1)
toplevel=$(cut -d. -f2 <<<$line)
echo "domain name=" $domain
echo "Top Level=" $toplevel
You need to remove $ from line in the beginning, correct the spaces and echo $line in order to pipe the value to cut . Alternatively feed the cut with $line.
I have below for loop in shell script
#!/bin/bash
#Get the year
curr_year=$(date +"%Y")
FILE_NAME=/test/codebase/wt.properties
key=wt.cache.master.slaveHosts=
prop_value=""
getproperty(){
prop_key=$1
prop_value=`cat ${FILE_NAME} | grep ${prop_key} | cut -d'=' -f2`
}
#echo ${prop_value}
getproperty ${key}
#echo "Key = ${key}; Value="${prop_value}
arr=( $prop_value )
for i in "${arr[#]}"; do
echo $i | head -n1 | cut -d "." -f1
done
The output I am getting is as below.
test1
test2
test3
I want to process the test2 from above results to below script in place of 'ABCD'
grep test12345 /home/ptc/storage/**'ABCD'**/apache/$curr_year/logs/access.log* | grep GET > /tmp/test.access.txt
I tried all the options but could not able to succeed as I am new to shell scripting.
Ignoring the many bugs elsewhere and focusing on the one piece of code you say you want to change:
for i in "${arr[#]}"; do
val=$(echo "$i" | head -n1 | cut -d "." -f1)
grep test12345 /dev/null "/home/ptc/storage/$val/apache/$curr_year/logs/access.log"* \
| grep GET
done > /tmp/test.access.txt
Notes:
Always quote your expansions. "$i", "/path/with/$val/"*, etc. (The * should not be quoted on the assumption that you want it to be expanded).
for i in $prop_value would have the exact same (buggy) behavior; using arr buys you nothing. If you want using arr to increase correctness, populate it correctly: read -r -a arr <<<"$prop_value"
The redirection is moved outside the loop -- that way the second iteration through the loop doesn't overwrite the file written by the first one.
The extra /dev/null passed to grep ensures that its behavior is consistent regardless of the number of matches; otherwise, it would display filenames only if more than one matching log file existed, and not otherwise.
I'm attempting to grep IPs from a number of files, look them up in DNS and compare them to the hostnames already in the same files to ensure both are correct. Then print out anything that is wrong.
I've gathered I need to put the information into arrays and diff them somehow.
Here is my horrible bash code which does not work. I'm pretty sure at least my for loop is wrong:
declare -a ipaddr=(`grep -h address *test.com.cfg | awk '{print $2}'`)
declare -a host_names=(`grep -h address *test.com.cfg | awk '{print $2}'`)
for i in "${ipaddr[#]}"
do
lookedup_host_names=( $(/usr/sbin/host ${ipaddr[#]} | awk '{print $5}' | cut -d. -f1-4 | tr '[:upper:]' '[:lower:]'))
done
if [[ -z diff <(printf "%s\n" "${lookedup_host_names[#]}"| sort ) <(printf "%s\n" "${host_names[#]}"| sort) ]]
then
printf "%s\n" "${lookedup_host_names[#]}"
fi
I don't see a difference between your arrays ipaddr and host_names. Supposed your files contain lines like
address 1.2.3.4 somehost.tld
a script like this may do what you want.
cat *test.com.cfg | grep address | while read line; do
IP=$(awk {'print $2'});
CO=$(awk {'print $3'});
CN=$(host $CO | cut -d ' ' -f 4)
[ "$CN" = "$IP" ] || echo "Error with IP $IP";
done
The two principal problems are that your for loop overwrites the array each time rather than appending, and your diff check is invalid.
To quickly fix the for loop, you could use += instead of =, .e.g lookedup_host_names+=( ... ).
To do the diff, you don't really need a condition. You could just run
diff <(printf "%s\n" "${host_names[#]}"| sort ) <(printf "%s\n" "${lookedup_host_names[#]}"| sort)
and it would show any differences between the two in diff format which most Unix users are familiar with (note that I switched the arguments around, since the first argument is supposed to be original).
If, like in your example, you actually do want to compare them and show the entire final list if there is a difference, you could do
if diff <(printf "%s\n" "${host_names[#]}"| sort ) <(printf "%s\n" "${lookedup_host_names[#]}"| sort) > /dev/null
then
printf "%s\n" "${lookedup_host_names[#]}"
fi