I know this has been asked before, and I have looked at the previous posts, but can't seem to get what I am trying to do working.
I know it's really basic, so I'm hoping this will be easy for most people on SO.
I want to create a simple bash script that pings an address, and then inserts the ping output into a csv file (or rather, into a file, separated with commas).
The ping command would be
ping -D google.com
and the output looks like
PING google.com (74.125.239.103) 56(84) bytes of data.
[1393992465.052723] 64 bytes from nuq05s01-in-f7.1e100.net (74.125.239.103): icmp_req=1 ttl=55 time=2.66 ms
I want to insert epoch time, ttl, and time into a file, separated by commas.
If it's easy enough, then it wouldn't hurt to convert to the epoch time to a date also, but this can be my next step in this simple project.
I figured out the command to convert epoch time to human format is
date -d #[epoch_time]
Side trivia - why do I need the #?
ping -D google.com | sed -n -e 2p -e 2q |
while read epoch b64 bytes from host ip icmp ttl time ms
do
date=$(date -d #$(sed 's/[][]//g' <<< "$epoch"))
ttl=$(sed 's/ttl=//' <<< "$ttl")
time=$(sed 's/time=//' <<< "$time")
echo "$date,$ttl,$time"
done >> file.csv
Clearly, you can extend the number of lines you analyze by adjusting the first sed command. You can probably do the editing within the loop with ${var/x/y} notation, but I don't think of doing that until I'm writing up at the end because I've spent longer working with shells where that wasn't an option than I have with bash. Are you sure you don't want the host or IP address information?
In the cold light of morning, there's an easy way to improve the code:
ping -D google.com | sed -n -e 2p -e 2q |
while read epoch b64 bytes from host ip icmp ttl time ms
do
epoch=${epoch#[}
epoch=${epoch%]}
date=$(date -d #${epoch})
echo "$date,${ttl#ttl=},${time#time=}"
done >> file.csv
It's a matter of choice whether you create the date variable or simply embed the $(date -d #${epoch}) directly in the echo argument.
Related
I have a long running process running and I want to monitor its RAM usage. I can do this by watching top. However I would like to be able to log out and have a record written, every minute say, to a shared disk space instead.
My solution which works is:
nohup top -b -d 60 -p 10036|grep 10036 >> ramlog.txt &
But I would like to know when each line is outputted too. How can I modify the one-liner to add this information on each line?
I know about screen and tmux but I would like to get this simple one-liner working.
You could add a loop that reads each line from grep and prepends a date. Make sure to use grep --line-buffered to ensure each line is printed without delay.
nohup top -b -d 60 -p 10036 |
grep --line-buffered 10036 |
while read line; do echo "$(date): $line"; done >> ramlog.txt &
I would like to see whenever the ping to a specific server goes down.
So at the moment I have something like this: ping 8.8.8.8 | grep time=11 && date
This pings google and prints every line that has a ping of 11.x that is just for testing so that I have some output that is not there all the time but often enough that I can test this. And then I have && date that I thought would print the date when grep finds the string.. But it does not. So my question now Is how can I get it so that in every line that grep spits out I can append the current timestamp. Or what would be even better is to have it as a prefix in front of every line that grep spits out. Is this possible with one command? Or do I need a simple shell script?
Thanks in advance. I hope I get an answer soon. :)
Also merry x-mas to everyone!
See if it works for you:
ping 8.8.8.8 | while read line; do echo "$(date): $line"; done | grep time=13
You can also turn the setup around to check when a remote host becomes available by using while ! ping ... Example, in case your path to 8.8.8.8 goes down:
declare -i cnt=0; while ! ping -c 1 8.8.8.8; do ((cnt++)); echo "test '$cnt' - network down on $(date)"; sleep 60; done
Which will check once per-minute if the host 8.8.8.8 is available and report, for example:
test '10' - network down on Sat Dec 26 16:38:48 CST 2015
The script will terminate on its own once the network path is established.
I need to perform a whois lookup on a file containing IP addresses and output both the country code and the IP address into a new file. In my command so far I find the IP addresses and get a unique copy that doesn't match allowed ranges. Then I run a whois lookup to find out who the foreign addresses are. Finally it pulls the country code out. This works great, but I can't get it show me the IP alongside the country code since that isn't included in the whois output.
What would be the best way to include the IP address in the output?
awk '{match($0,/[0-9]+\.[0-9]+\.[0-9]+\.[0-9]+/); ip = substr($0,RSTART,RLENGTH); print ip}' myInputFile \
| sort \
| uniq \
| grep -v '66.33\|66.128\|75.102\|216.106\|66.6' \
| awk -F: '{ print "whois " $1 }' \
| bash \
| grep 'country:' \
>> myOutputFile
I had thought about using tee, but am having troubles lining up the data in a way that makes sense. The output file should be have both the IP Address and the country code. It doesn't matter if they are a single or double column.
Here is some sample input:
Dec 27 04:03:30 smtpfive sendmail[14851]: tBRA3HAx014842: to=, delay=00:00:12, xdelay=00:00:01, mailer=esmtp, pri=1681345, relay=redcondor.itctel.c
om. [75.102.160.236], dsn=4.3.0, stat=Deferred: 451 Recipient limit exceeded for this se
nder
Dec 27 04:03:30 smtpfive sendmail[14851]: tBRA3HAx014842: to=, delay=00:00:12, xdelay=00:00:01, mailer=esmtp, pri=1681345, relay=redcondor.itctel.c
om. [75.102.160.236], dsn=4.3.0, stat=Deferred: 451 Recipient limit exceeded for this se
nder
Thanks.
In general: Iterate over your inputs as shell variables; this then lets you print them alongside each output from the shell.
The below will work with bash 4.0 or newer (requires associative arrays):
#!/bin/bash
# ^^^^- must not be /bin/sh, since this uses bash-only features
# read things that look vaguely like IP addresses into associative array keys
declare -A addrs=( )
while IFS= read -r ip; do
case $ip in 66.33.*|66.128.*|75.102.*|216.106.*|66.6.*) continue;; esac
addrs[$ip]=1
done < <(grep -E -o '[0-9]+[.][0-9]+[.][0-9]+[.][0-9]+')
# getting country code from whois for each, printing after the ip itself
for ip in "${!addrs[#]}"; do
country_line=$(whois "$ip" | grep -i 'country:')
printf '%s\n' "$ip $country_line"
done
An alternate version which will work with older (3.x) releases of bash, using sort -u to generate unique values rather than doing that internal to the shell:
while read -r ip; do
case $ip in 66.33.*|66.128.*|75.102.*|216.106.*|66.6.*) continue;; esac
printf '%s\n' "$ip $(whois "$ip" | grep -i 'country:')"
done < <(grep -E -o '[0-9]+[.][0-9]+[.][0-9]+[.][0-9]+' | sort -u)
It's more efficient to perform input and output redirection for the script as a whole than to put a >> redirection after the printf itself (which would open the file before each print operation and close it again after, incurring a substantial performance penalty), which is why suggested invocation for this script looks something like:
countries_for_addresses </path/to/logfile >/path/to/output
I am trying to get some numbers from tcpdump inside a shell script and print that number.
Here is my script
while true
do
{
b=`tcpdump -n -i eth1 | awk -F'[, ]' '{print $10}'`
echo $b
}
done
When I execute this script, I get this
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on eth1, link-type EN10MB (Ethernet), capture size 65535 bytes
Is there anything special I need to do to capture tcpdump o/p inside shell script ?
By default, tcpdump runs forever (or until it's interrupted by Control-C or something similar). The
b=`tcpdump ...`
construct runs until tcpdump exits... which is never ... and then puts its output into $b. If you want to capture the output from a single packet, you can use tcpdump -c1 ... (or -c5 to capture groups of 5, or similar). Alternately, you could let it run forever but capture its output one line at a time with a while read loop (although you need to use tcpdump -l to prevent excessive buffering):
tcpdump -l -n -i eth1 | awk -F'[, ]' '{print $10}' | while read b; do
echo $b
done
I'm not entirely sure what your script is supposed to do, but I see some other issues. First, unless your version of tcpdump is much more consistent than mine, printing the 10th comma-delimited field of each packet will not get you anything meaningful. Here's some sample output from my computer:
00:05:02:ac:54:1e
1282820004:1282820094
90
73487384:73487474
1187212630:1187212720
90
90
host
2120673524
Second, what's the point of capturing the output into a variable, then printing it? Why not just run the command and let it output directly? Finally, echo $b may garble the output due to word splitting and wildcard expansion (for example, if $b happened to be "*", it would print a list of files in the current directory). For this reason, you should double-quote variables when you use them (in this case, echo "$b").
It's been so long since this question was asked but the simplest way to accomplish the goal of what the script was intending to catch would be to simply record a pcap matching only the packets you're interestedin seeing; as an example, to write a pcap file consisting only of packets where the ack flag is set and the acknowledgment number is a value between 19000 and 20000:
tcpdump -c2500 -iany 'tcp[8:4]>=19000&&tcp[8:4]<=20000&&tcp[13]&16!=0' -Uw./TCP_ACKs.pcap
What I want is to ping a server but only want the avg time to be returned, I think grep or something like that would help but a google search doesn't return much useful.
This is to go into a geektool script that will display a servers ping for me to see if its up.
Needs to be a script / command that will work in terminal as I need to pull it over to python as well.
Thanks
Somehow on my system ping command doesn't output round-trip but rtt.
So this one is going to work better:
ping -q -c 5 google.com | tail -n 1 | cut -f 5 -d '/'
-q makes it less verbose because we don't need much of the output anyway. tail simply returns the last line of the output
How about something like:
ping -c 5 google.com | grep "round-trip" | cut -f 5 -d "/"