I am trying to extract the highest number from $countip and copy it to $totalip, the problem is that $totalip is allways returning 0. Can anybody help me please, i am new to bash script.
for srcip in `cat /var/log/messages | grep "WACSLAW1 CRITICAL INCOMING" | awk '{ print $14 }'|grep -v 192.168.1. |grep -v IN=eth1 |grep -v MAC`;do
if (! grep "$srcip" /var/wacstemp/ids.tmp > /dev/null) ; then
countip=0
echo $srcip >> /var/wacstemp/ids.tmp
else
countip=`expr $countip + 1`
if [ $countip -gt $totalip ];
then
# echo $countip
countip=$totalip
# echo $totalip
fi
fi
done
You must switch
countip=$totalip
to
totalip=$countip
otherwise totalip will never be updated.
OT: You could simplify the initial filter to
awk '/WACSLAW1 CRITICAL INCOMING/ && $14 !~ /192.168.1./ && $14 !~ /IN=eth1/ && $14 !~ /MAC/ { print $14 }' /var/log/messages
Related
Would it be possible to print the filename used to redirect STDERR, given the sample command below:
command.sh 2>file.err
Code in command.sh:
#!/bin/sh
ls -l non_existing_file.txt
echo "STDERR file is: $stderrFilename" # variable should print file.err
It's a little risky, but you could try parsing AIX's procfiles output. It involves capturing the major and minor numbers of the stderr device, along with the inode number, then looking for the corresponding device, its mountpoint, and then using find to look for the file with the given inode number:
#!/bin/sh
dev=$(procfiles $$ | awk '$1 == "2:" { print substr($4, 5) }')
inode=$(procfiles $$ | awk '$1 == "2:" { print substr($5, 5) }')
major=${dev%%,*}
minor=${dev##*,}
if [ "$major}" -eq 0 ]
then
echo I give up, the major number is zero
exit 1
fi
for file in /dev/*
do
[ -b "$file" ] || continue
if istat "$file" | grep -q "^Major Device ${major}.*Minor Device ${minor}$"
then
break
fi
done
fs=$(mount | awk '$1 == "'"${file}"'" { print $2 }')
stderrFilename=$(find "$fs" -inum "$inode")
I made a solution using history. Not sure if there is an easier way to do this ( or a proper one).
#!/bin/sh
stderrfname=`history | tail -1 | awk '{ print $3 }' | sed "s/.*>//"`
echo "STDERR file is: $stderrfname"
Say i have:
> id|lastName|firstName|gender|birthday|creationDate|locationIP|browserUsed
>
> 13194139534963|Berty|Jean|male|1988-08-02|2012-04-02T08:33:15.012+0000|41.216.190.153|Google
> Chrome
>
> 13194139535544|Oliveira|Manuel|male|1984-10-31|2012-03-14T16:00:12.287+0000|109.71.166.230|Internet
> Explorer
>
> 13194139537327|Wei|Lei|male|1987-01-06|2012-03-13T03:07:51.899+0000|27.99.188.150|Internet
> Explorer
>
> 13194139539118|Alvarez|Monica|male|1989-10-17|2012-02-25T19:18:54.137+0000|190.169.213.242|Internet
> Explorer
>
> 13194139539746|Xu|Wei|female|1986-11-30|2012-03-19T23:16:12.495+0000|27.103.77.193|Firefox
I want to make a command with those parameters : ./tool.sh --born-since dateA --born-until dateB -f file
1)If born-since and born-until dates are given i want to print all those born(the whole line)between two specific dates ( Year-Month-Date )
Example
./tool.sh --born-since 1988-08-02 --born-until 2012-09-13 -f file
Output:
13194139534963|Berty|Jean|male|1988-08-02|2012-04-02T08:33:15.012+0000|41.216.190.153|Google
13194139539118|Alvarez|Monica|male|1989-10-17|2012-02-25T19:18:54.137+0000|190.169.213.242|Internet
Explorer
2)If only born-since date is given i want to list all the people(whole line) with born dates of that and after.
Example:
./tool.sh --born-since 1988-08-02 -f file
Output:
Same as 1)
3)If only born-until date is given i want to list all the people born until that date(again the whole line about them).
./tool.sh --born-until 1988-08-02 -f file
Output:
13194139535544|Oliveira|Manuel|male|1984-10-31|2012-03-14T16:00:12.287+0000|109.71.166.230|Internet Explorer
13194139537327|Wei|Lei|male|1987-01-06|2012-03-13T03:07:51.899+0000|27.99.188.150|Internet Explorer
13194139539746|Xu|Wei|female|1986-11-30|2012-03-19T23:16:12.495+0000|27.103.77.193|Firefox
My code is :
while [ $# -gt 0 ];do #Get and store Dates (Since-Until)
if [ "$1" = --born-since ];then
if [[ "$2" =~ $re ]];then #re='[0-9]-*' # Check if $2 is number
BSDate=$2
BSYear=$(echo "$BSDate" | awk -F '-' '{print $1}') # Get BSYear
BSMonth=$(echo "$BSDate" | awk -F '-' '{print $2}') # Get BSMonth
BSDay=$(echo "$BSDate" | awk -F '-' '{print $3}') # Get BSDay
fi
elif [ "$1" = --born-until ];then
if [[ "$2" =~ $re ]];then
BUDate=$2
BUYear=$(echo "$BUDate" | awk -F '-' '{print $1}') # Get BUYear
BUMonth=$(echo "$BUDate" | awk -F '-' '{print $2}') # Get BUMonth
BUDay=$(echo "$BUDate" | awk -F '-' '{print $3}') # Get BUDay
fi
fi
shift
done
if [ "$BSDate" ] && [ "$BUDate" ];then #If both date arguments exist
elif [ "$BSDate" ];then
elif [ "$BUDate" ];then
fi
If i enter --born-since 1998-10-30 the arguments get passed correctly for evaluation in awk , 1998 = BSYear , 10 = BSMonth , 30 = BSDay. Can someone help me implement the awk part ?
For the awk part :
cat ./tool.sh
awk -F'|' -vs="$1" -ve="$2" '
BEGIN{if(!s)s="0000-00-00";if(!e)e="9999-99-99"}
NR>1 && $5>=s && $5<=e' infile
And you call it like that
./tool.sh '1987-01-06' '1988-08-02'
or
./tool.sh '' '1988-08-02'
or
./tool.sh '1987-01-06' ''
I fixed it with : awk -F'|' '{if ($5 >= "'$BSDate'" && $5 <= "'$BUDate'")
I have this until now:
if [[ $1 = "-s" ]] && [[ $2 =~ ^[0-9]+$ ]]
echo "ok"
for $2 = 1 awk {print $1} something.txt
fi
What I want is, for example, when I enter -s 2, to list the second word of every line first!
I already know the for part is wrong.
Replace
for $2 = 1 awk {print $1} something.txt
by
awk -v var="$2" '{print $var}' something.txt
I'm trying to separate a large pipe-delimited file based on a week number field. The file contains data for a full year thus having 53 weeks. I am hoping to create a loop that does the following:
1) check if week number is less than 10 - if it is paste a '0' in front
2) use grep to send the rows to a file (ie `grep '|01|' bigFile.txt > smallFile.txt` )
3) gzip the smaller file (ie `gzip smallFile.txt`)
4) repeat
Is there a resource that would show how to do this?
EDIT :
Data looks like this:
1|#gmail|1|0|0|0|1|01|com
1|#yahoo|0|1|0|0|0|27|com
The column I care about is the 2nd from the right.
EDIT 2:
Here's the script I'm using but it's not functioning:
for (( i = 1; i <= 12; i++ )); do
#statements
echo 'i :'$i
q=$i
# echo $q
# $q==10
if [[ q -lt 10 ]]; then
#statements
k='0'$q
echo $k
grep '|$k|' 20150226_train.txt > 'weeks_files/week'$k
gzip weeks_files/week $k
fi
if [[ q -gt 9 ]]; then
#statements
echo $q
grep \'|$q|\' 20150226_train.txt > 'weeks_files/week'$q
gzip 'weeks_files/week'$q
fi
done
Very simple in awk ...
awk -F'|' '{ print > ("smallfile-" $(NF-1) ".txt";) }' bigfile.txt
Edit: brackets added for "original-awk".
You're almost there.
#!/bin/bash
for (( i = 1; i <= 12; i++ )); do
#statements
echo 'i :'$i
q=$i
# echo $q
# $q==10
#OLD if [[ q -lt 10 ]]; then
if [[ $q -lt 10 ]]; then
#statements
k='0'$q
echo $k
#OLD grep '|$k|' 20150226_train.txt > 'weeks_files/week'$k
grep "|$k|" 20150226_train.txt > 'weeks_files/week'$k
#OLD gzip weeks_files/week $k
gzip weeks_files/week$k
#OLD fi
#OLD if [[ q -gt 9 ]]; then
elif [[ $q -gt 9 ]] ; then
#statements
echo $q
#OLD grep \'|$q|\' 20150226_train.txt > 'weeks_files/week'$q
grep "|$q|" 20150226_train.txt > 'weeks_files/week'$q
gzip 'weeks_files/week'$q
fi
done
You didn't alway use $ in front of your variable values. You can only get away with using k or q without a $ inside the shell arthimetic substitution feature, ie z=$(( x+k)) or just to operate on a variable like (( k++ )). There are others.
You need to learn the difference between single quoting and dbl-quoting. You need to use dbl-quoting when you want a value substituted for a variable, as in your lines
grep "|$q|" 20150226_train.txt > 'weeks_files/week'$q
and others.
I'm guessing that your use of grep \'|$q|\' 20150226_train.txt was an attempt to get the value of $q.
The way to get comfortable with debugging this sort of situation is to set the shell debugging option with set -x (turn it off with set +x). You'll see each line that is executed with the values substituted for the variables. Advanced debugging requires echo "varof Interset now = $var" (print statements). Also, you can use set -vx (and set +vx) to see each line or block of code before it is executed, and then the -x output will show which lines where acctually executed. For your script, you'd see the whole if ... elfi ...fi block printed, and then just the lines of -x output with values for variables. It can be confusing, even after years of looking at it. ;-)
So you can go thru and remove all lines with the prefix #OLD, and I'm hoping your code will work for you.
IHTH
mkdir -p weeks_files &&
awk -F'|' '
{ file=sprintf("weeks_files/week%2d",$(NF-1)); print > file }
!seen[file]++ { print file }
' 20150226_train.txt |
xargs gzip
If your data is ordered so that all of the rows for a given week number are contiguous you can make it simpler and more efficient:
mkdir -p weeks_files &&
awk -F'|' '
$(NF-1) != prev { file=sprintf("weeks_files/week%2d",$(NF-1)); print file }
{ print > file; prev=$(NF-1) }
' 20150226_train.txt |
xargs gzip
There are certainly a number of approaches - the 'awk' line below will reformat your data. If you take a sequential approach, then:
1) awk to reformat
awk -F '|' '{printf "%s|%s|%s|%s|%s|%s|%s|%02d|%s\n", $1, $2, $3, $4, $5, $6, $7, $8, $9}' SOURCE_FILE > bigFile.txt
2) loop through the weeks, create small file an zip it
for N in {01..53}
do
grep "|${N}|" bigFile.txt > smallFile.${N}.txt
gzip smallFile.${N}.txt
done
3) test script showing reformat step
#!/bin/bash
function show_data {
# Data set w/9 'fields'
# 1| 2 |3|4|5|6|7| 8|9
cat << EOM
1|#gmail|1|0|0|0|1|01|com
1|#gmail|1|0|0|0|1|2|com
1|#gmail|1|0|0|0|1|5|com
1|#yahoo|0|1|0|0|0|27|com
EOM
}
###
function stars {
echo "## $# ##"
}
###
stars "Raw data"
show_data
stars "Modified data"
# 1| 2| 3| 4| 5| 6| 7| 8|9 ##
show_data | awk -F '|' '{printf "%s|%s|%s|%s|%s|%s|%s|%02d|%s\n", $1, $2, $3, $4, $5, $6, $7, $8, $9}'
Sample run:
$ bash test.sh
## Raw data ##
1|#gmail|1|0|0|0|1|01|com
1|#gmail|1|0|0|0|1|2|com
1|#gmail|1|0|0|0|1|5|com
1|#yahoo|0|1|0|0|0|27|com
## Modified data ##
1|#gmail|1|0|0|0|1|01|com
1|#gmail|1|0|0|0|1|02|com
1|#gmail|1|0|0|0|1|05|com
1|#yahoo|0|1|0|0|0|27|com
i am new to bash script. so,i need help. Below command print out USB serial number. But i don't how to store it in a string. Because i want to compare with other string to see if they are match.The other string can be any string. so, if someone can help me write the code,i would very much appreciate. Thanks in Advanced.
lsusb -v | awk '/iSerial/ {if ($2 == "3" || $2 == "2") {print $3}}'
anubhava, would this be right if i want to compare two strings.
#!/bin/sh
string="asd11ds"
output=$(lsusb -v | awk '/iSerial/ && ($2 == "3" || $2 == "2")) {print $3}')
echo $output
if [[$output==$string]];then
echo"the two string are the same/are different"
fi
You use $(...) notation (command substitution) to store a command's output.
You awk can also be shortened:
output=$(lsusb -v | awk '/iSerial/ && ($2 == "3" || $2 == "2")) {print $3}')
since your question tag is bash, I provide bash script
#!/usr/bin/env bash
string="asd11ds"
output=$(lsusb -v | awk '/iSerial/ && ($2 == "3" || $2 == "2")) {print $3}')
echo $output
if [[ "$output" == "$string" ]]; then
echo "the two strings are same"
else
echo "the two strings are different"
fi