Portable way to resolve host name to IP address - shell

I need to resolve a host name to an IP address in a shell script. The code must work at least in Cygwin, Ubuntu and OpenWrt(busybox).
It can be assumed that each host will have only one IP address.
Example:
input
google.com
output
216.58.209.46
EDIT:
nslookup may seem like a good solution, but its output is quite unpredictable and difficult to filter. Here is the result command on my computer (Cygwin):
>nslookup google.com
Unauthorized answer:
Serwer: UnKnown
Address: fdc9:d7b9:6c62::1
Name: google.com
Addresses: 2a00:1450:401b:800::200e
216.58.209.78

I've no experience with OpenWRT or Busybox but the following one-liner will should work with a base installation of Cygwin or Ubuntu:
ipaddress=$(LC_ALL=C nslookup $host 2>/dev/null | sed -nr '/Name/,+1s|Address(es)?: *||p')
The above works with both the Ubuntu and Windows version of nslookup. However, it only works when the DNS server replies with one IP (v4 or v6) address; if more than one address is returned the first one will be used.
Explanation
LC_ALL=C nslookup sets the LC_ALL environment variable when running the nslookup command so that the command ignores the current system locale and print its output in the command’s default language (English).
The 2>/dev/null avoids having warnings from the Windows version of nslookup about non-authoritative servers being printed.
The sed command looks for the line containing Name and then prints the following line after stripping the phrase Addresses: when there's more than one IP (v4 or 6) address -- or Address: when only one address is returned by the name server.
The -n option means lines aren't printed unless there's a p commandwhile the-r` option means extended regular expressions are used (GNU sed is the default for Cygwin and Ubuntu).

If you want something available out-of-the-box on almost any modern UNIX, use Python:
pylookup() {
python -c 'import socket, sys; print socket.gethostbyname(sys.argv[1])' "$#" 2>/dev/null
}
address=$(pylookup google.com)
With respect to special-purpose tools, dig is far easier to work with than nslookup, and its short mode emits only literal answers -- in this case, IP addresses. To take only the first address, if more than one is found:
# this is a bash-specific idiom
read -r address < <(dig +short google.com | grep -E '^[0-9.]+$')
If you need to work with POSIX sh, or broken versions of bash (such as Git Bash, built with mingw, where process substitution doesn't work), then you might instead use:
address=$(dig +short google.com | grep -E '^[0-9.]+$' | head -n 1)
dig is available for cygwin in the bind-utils package; as bind is most widely used DNS server on UNIX, bind-utils (built from the same codebase) is available for almost all Unix-family operating systems as well.

Here's my variation that steals from earlier answers:
nslookup blueboard 2> /dev/null | awk '/Address/{a=$3}END{print a}'
This depends on nslookup returning matching lines that look like:
Address 1: 192.168.1.100 blueboard
...and only returns the last address.
Caveats: this doesn't handle non-matching hostnames at all.

TL;DR; Option 2 is my preferred choice for IPv4 address. Adjust the regex to get IPv6 and/or awk to get both. There is a slight edit to option 2 suggested use given in EDIT
Well a terribly late answer here, but I think I'll share my solution here, esp. because the accepted answer didn't work for me on openWRT(no python with minimal setup) and the other answer errors out "no address found after comma".
Option 1 (gives the last address from last entry sent by nameserver):
nslookup example.com 2>/dev/null | tail -2 | tail -1 | awk '{print $3}'
Pretty simple and straight forward and doesn't really need an explanation of piped commands.
Although, in my tests this always gave IPv4 address (because IPv4 was always last line, at least in my tests.) However, I read about the unexpected behavior of nslookup. So, I had to find a way to make sure I get IPv4 even if the order was reversed - thanks regex
Option 2 (makes sure you get IPv4):
nslookup example.com 2>/dev/null | sed 's/[^0-9. ]//g' | tail -n 1 | awk -F " " '{print $2}'
Explanation:
nslookup example.com 2>/dev/null - look up given host and ignore STDERR (2>/dev/null)
sed 's/[^0-9. ]//g' - regex to get IPv4 (numbers and dots, read about 's' command here)
tail -n 1 - get last 1 line (alt, tail -1)
awk -F " " '{print $2} - Captures and prints the second part of line using " " as a field separator
EDIT: A slight modification based on a comment to make it actually more generalized:
nslookup example.com 2>/dev/null | printf "%s" "$(sed 's/[^0-9. ]//g')" | tail -n 1 | printf "%s" "$(awk -F " " '{print $1}')"
In the above edit, I'm using printf command substitution to take care of any unwanted trailing newlines.

Related

Can't capture the ouput of the paste bash command

( ping -n apple.com | ack -o --flush '((?<=icmp_seq=)[0-9]+|(?<=time=)[0-9]+[.][0-9]+)' | paste - - ) > ~/Desktop/ping_ouput.txt
Doesn't seem to be working for some reason, whereas if I take away the
| paste - -
section, it works just fine. I need to join every other line together with a tab instead of newline. Any ideas are appreciated.
Update:
By default, ping produces output indefinitely, until terminated, so your pipe will keep producing output and growing the output file indefinitely, and your command will never finish by itself.
Thus, you need to limit the number of pings performed; e.g.:
using the -c count option to stop after the specified number of pings.
alternatively, on BSD-like systems, you can use -o to stop after the first successful ping.
alternatively, you can stop after a specified timeout in seconds, regardless of how many packets were received:
Linux: -w timeout
BSD-like systems: -t timeout
Incidentally, in order to redirect a pipeline to a file, there's no need to enclose it in (...) as a whole, which needlessly creates a(nother) subshell.
The following was written before (a) the specific symptom was known and (b) before the OP revealed in a comment that their platform is OSX. The commands below show alternatives to the OP's extraction command; -c 2 has been added to limit ping to 2 attempts.
tivn's answer has found at least potential problem in your command: the assumption that time values always have a decimal point, which does not hold on Linux platforms; on BSD/OSX platforms, however, there are always 3 decimal places.
You can bypass this issue as well as the need to merge consecutive lines as follows (I'm using 127.0.0.1 instead of apple.com for easier testing):
Using either GNU sed (Linux) or BSD sed (BSD-like systems, including OSX):
ping -c 2 -n 127.0.0.1 |
sed -E '1d; s/^.* icmp_seq=([^ ]+).* time=([^ ]+).*$/\1'$'\t''\2/'
Alternative solutions, using awk:
ping -c 2 -n 127.0.0.1 | awk -F'[ =]' -v OFS='\t' 'NR>1 { print $6, $10 }'
If you'd rather not rely on field indices, here's an alternative (still relies on field icmp_seq to come before time, and for both to be preceded by a space):
ping -c 2 -n 127.0.0.1 | awk -F' (icmp_seq|time)=' -v OFS='\t' '
NR>1 { sub(" .+$", "", $2); sub(" .+$", "", $3)
print $2, $3 }'
You don't provide sample input you got from the ping, so it is not clear what your problem is. But I guess your issue is because the time part from ping result may not always contains dot ., for example: 64 bytes from x.x.x.x: icmp_seq=1 ttl=63 time=137 ms . You may want to try with the following command :
( ping -n yahoo.com \
| ack -o --flush '((?<=icmp_seq=)\d+|(?<=time=)[\d.]+)' \
| stdbuf -o0 paste - - \
) > ~/Desktop/ping_ouput.txt
Edit: after comment from OP, the issue may actually come from buffered paste command. Please try with adding stdbuf -o0 to the paste command.

bash script to perform dig -x

Good day. I was reading another post regarding resolving hostnames to IPs and only using the first IP in the list.
I want to do the opposite and used the following script:
#!/bin/bash
IPLIST="/Users/mymac/Desktop/list2.txt"
for IP in 'cat $IPLIST'; do
domain=$(dig -x $IP +short | head -1)
echo -e "$domain" >> results.csv
done < domainlist.txt
I would like to give the script a list of 1000+ IP addresses collected from a firewall log, and resolve the list of destination IP's to domains. I only want one entry in the response file since I will be adding this to the CSV I exported from the firewall as another "column" in Excel. I could even use multiple responses as semi-colon separated on one line (or /,|,\,* etc). The list2.txt is a standard ascii file. I have tried EOF in Mac, Linux, Windows.
216.58.219.78
206.190.36.45
173.252.120.6
What I am getting now:
The domainlist.txt is getting an exact duplicate of list2.txt while the results has nothing. No error come up on the screen when I run the script either.
I am running Mac OS X with Macports.
Your script has a number of syntax and stylistic errors. The minimal fix is to change the quotes around the cat:
for IP in `cat $IPLIST`; do
Single quotes produce a literal string; backticks (or the much preferred syntax $(cat $IPLIST)) performs a command substitution, i.e. runs the command and inserts its output. But you should fix your quoting, and preferably read the file line by line instead. We can also get rid of the useless echo.
#!/bin/bash
IPLIST="/Users/mymac/Desktop/list2.txt"
while read IP; do
dig -x "$IP" +short | head -1
done < "$IPLIST" >results.csv
Seems that in your /etc/resolv.conf you configured a nameserver which does not support reverse lookups and that's why the responses are empty.
You can pass the DNS server which you want to use to the dig command. Lets say 8.8.8.8 (Google) for example:
dig #8.8.8.8 -x "$IP" +short | head -1
The commands returns the domain with a . appended. If you want to replace that you can additionally pipe to sed:
... | sed 's/.$//'

Best way to handle objects property:value in the stdin in bash

Bonjour,
I launch nslookup someServer. I consider I get a serie of object (as in powers hell) separated by empty lines and not simply a stdout.
$ nslookup someServer
Server: 10.0.0.1
Address: 10.0.0.1#53
Name: someServer
Address: 10.0.0.5
$
How to get the object who have both properties Name and Address?
nslookup someServer | haveboth Name Address | wc -l
Does it exists in GNU utilities?
are you simply looking for a way to check that you have both of these values in your output?
Then you could use perl in oneline mode (probably not the most pretty solution imaginable, but does what you want and could be expanded to check more things easily)
nslookup someServer | perl -ne '$v{"NAME"}++ if /Name/; $v{"ADDRESS"}++ if /Address/; END{ print "Has both values\n" if $v{"NAME"} && $v{"ADDRESS"} }'
this goes through your output and counts the occurrences of Name and Address and then prints a message if it has more than zero of both.
If using perl in this way is a viable option, then I can recommend this page for further reading on perl oneliners
EDIT:
in case you want to have access to the values that are stored in your property: you can use
nslookup someServer | perl -F: -lane '$v{"NAME"} = $F[1] if /Name/; $v{"ADDRESS"} = $F[1] if /Address/; END{ print $v{"ADDRESS"}." ".$v{"NAME"} if exists $v{"ADDRESS"} && $v{"NAME"} }'
This will split a given line on : as field separator and store the value in the variable instead of simple counting occurrences. Note that done in this way it would only store the last occurrence.

How to get Primary MX IP from domain in Bash

I am trying to write a bash script that will from a domain name find it MX records, from them figure out which is the primary (they are not always in order) and then find its IP.
(when there are more then one primary MX the first one found would be ok)
For example:
./findmxip.sh gmail.com
Would give me 173.194.71.26. For me to do this I need to host gmail.com
then find the primary MX in the results and host it, getting its IP.
To get exactly 0 or 1 answers:
dig +short gmail.com mx | sort -n | nawk '{print $2; exit}' | dig +short -f -
You'll need a non-ancient dig that supports +short.
As noted there may be more than one "primary" MX as the preferences need not be unique. If you want all the IP addresses of all of the lowest preference records then:
dig +short oracle.com mx | sort -n |
nawk -v pref=65536 '($1<=pref) {pref=$1; print $2}' |
dig +short -f - | uniq
This does not handle the case where there is no MX record and the A record accepts email, an uncommon but perfectly valid configuration.
Sadly all the dig versions I've tested return 0 whether the domain exists or not (NXDOMAIN), and whether any MX records exist or not. You can catch a DNS time-out (rc=9) though. The related host command does return a non-zero rc with NXDOMAIN, but its behaviour is a little inconsistent, it's messy to script and the output harder to parse.
A poor man's error-checking version (inspired by tripleee's comment) that might be slightly more robust depending on your host command is:
DOMAIN=gmail.com
if ! host -t any $DOMAIN >/dev/null 2>&1 ; then
echo "no such domain"
elif ! host -t mx $DOMAIN >/dev/null 2>&1; then
echo "no MX records"
else
dig +short $DOMAIN mx | sort -n | nawk '{print $2; exit}' | dig +short -f -
fi
(Perversely, you may require an older version of host (bind-8.x) for the -t mx test to work, newer versions just return 0 instead.)
This is just about the point people start backing away nervously asking why you're not using perl/python/$MFTL.
If you really need to write a robust version in bash, check out the djbdns CLI tools and debugging tools which are rather easier to parse (though sadly don't set user exit codes either).

tcpdump - ignore unkown host error

I've got a tcpdump command running from a bash script. looks something like this.
tcpdump -nttttAr /path/to/file -F /my/filter/file
The filter file has a combination of ip addresses and host names. i.e.
host 111.111.111.111 or host 112.112.112.112 and not (host abc.com or host def.com or host zyx.com).
And it works great - as long as the host names are all valid. My problem is sometimes these hostnames will not be valid and upon encountering one - tcpdump spits out
tcpdump: Unknown Host
I thought with the -n option it would skip dns lookup - but in anycase I need it to ignore the unknown host and continue along the filter file.
Any ideas?
Thank you in advance.
The -n option prevents conversion of IP addresses into names, but not the other way around. If you supply a hostname as an argument, it has to be looked up to get the IP address since packets only contain the numeric address and not the hostname. However, there ought to be a way to ignore invalid hostnames, but I can't find one. Perhaps you could pre-process your filter file using dig.
dig +short non-existent-domain.com # returns null
dig +short google.com # returns multiple IP addresses
This could probably be better, but it should show you hostnames in your filter file that aren't valid:
grep -Po '(?<=host )[^ )]*' filterfile | grep -v '[0-9]$' | xargs -I % sh -c 'echo -n "% "; echo $(dig +short %)' | grep -v ' [0-9]'
Any hostnames it prints didn't have IP addresses returned by dig.

Resources