I've got a text file with a bunch of IPv4 addresses, and I'd like to know the hostname of each one in order to know if they are tor addresses. Is there a simple script that can help me to do that ?
You can loop using dig:
#!/bin/bash
while read line
do
dig -x "$line" +short
done
Then given IPs 1 per line, you can run something like ./reverse.sh < addrs.txt.
Caveats: DNS is not a 1-to-1 mapping, and reverse DNS is somewhat less reliable than forward DNS.
Related
I'm going to try to describe my problem and my end goal as best as I can, here it goes:
I have a script that fetches AWS ELB information (elb name + ports that's associated with a specific certifican arn).
So, in the end I have a text file (I call it elb_ports) and it looks something like this:
ccds-lb 636
cf-router 443 4443
dev-cf-router 443 4443
eng-jenkins-monit 443
gitlab-lb 443
gitlab-mattermost-elb 443
jenkins-np-elb 443
saml 443
uaa 443
I have another script that comes after that which I want it to go through that elb_ports file and replace the certificates with a new one, but according to Amazon's documentation: It says in order to replace the certificates, I need two things from that elb_ports file. The load balancer name and the load balancer port.
So basically their command looks like this
aws elb set-load-balancer-listener-ssl-certificate \
--load-balancer-name my-load-balancer \
--load-balancer-port 443 \
--ssl-certificate-id arn:aws:iam::123456789012:server-certificate/my-new-certificate
I want to be able to loop through the file and execute the command above to each elb and port, but my problem is with the elbs that has multiple ports associated with the cert like: cf-router 443 4443 for example.
So my idea was to split that into two lines, so like this:
cf-router 443
cf-router 4443
But I'm not sure how to add cf-router (for example) to the ports that come after the first one (there could be more than two ports using the same cert).
I hope I was able to explain my problem and end goal clearly, if this isn't a good method, I'm open to suggestions also.
EDIT: Perhaps something like this is beneficial, but not sure how to tailor it to my needs.. Like put each line in an array and the space as a delimiter and then loop through each line putting arr(1) (load balancer name) and then the load balancer port, but not sure how to count and go through >arr(2) in bash.
To split out your extra columns into separate lines:
while read -r lb_name lb_ports_str; do ## split line into lb name and port list string
read -r -a lb_ports <<<"$lb_ports_str" ## split out port list string into an array
for port in "${lb_ports[#]}"; do ## iterate through that array
printf '%s %s\n' "$lb_name" "$port" ## handle each port separately
done
done <elb_ports ## reading lines from elb_ports
Of course, that printf could be any other line referring to $lb_name and $port -- meaning you could potentially run your code that's installing new certificates here.
The goal was to frequently change default outgoing source ip on a machine with multiple interfaces and live ips.
I used ip route replace default as per its documentation and let a script run in loop for some interval. It changes source ip fine for a while but then all internet access to the machine is lost. It has to be remotely rebooted from a web interface to have any thing working
Is there any thing that could possibly prevent this from working stably. I have tried this on more than one servers?
Following is a minimum example
# extract all currently active source ips except loopback
IPs="$(ifconfig | grep 'inet addr:'| grep -v '127.0.0.1' | cut -d: -f2 |
awk '{ print $1}')"
read -a ip_arr <<<$IPs
# extract all currently active mac / ethernet addresses
Int="$(ifconfig | grep 'eth'| grep -v 'lo' | awk '{print $1}')"
read -a eth_arr <<<$Int
ip_len=${#ip_arr[#]}
eth_len=${#eth_arr[#]}
i=0;
e=0;
while(true); do
#ip route replace 0.0.0.0 dev eth0:1 src 192.168.1.18
route_cmd="ip route replace 0.0.0.0 dev ${eth_arr[e]} src ${ip_arr[i]}"
echo $route_cmd
eval $route_cmd
sleep 300
(i++)
(e++)
if [ $i -eq $ip_len ]; then
i=0;
e=0;
echo "all ips exhausted - starting from first again"
# break;
fi
done
I wanted to comment, but as I'm not having enough points, it won't let me.
Consider:
Does varying the delay time before its run again change the number of iterations before it fails?
Exporting the ifconfig & routes every time you change it, to see if something is meaningfully different over time. Maybe some basic tests to it (ping, nslookup, etc) Basically, find what is exactly going wrong with it. Also exporting the commands you send to a logfile (text file per change?) to see changes in them to see if some is different after x iterations.
What connectivity is lost? Incoming? Outgoing? Specific applications?
You say you use/do this on other servers without problems?
Are the IP's: Static (/etc/network/interfaces), bootp/DHCP, semi-static (bootp/DHCP server serving, based on MAC address), and if served by bootp/DHCP, what is the lease duration?
On the last remark:
bootp/dhcp will give IP's for x duration. say its 60 minutes. After half that time it will "check" with the bootp/dhcp server if it can keep the IP, and extend the lease to 60 minutes again, this can mean a small reconfig on the ifconfig (maybe even at the same time of your script?).
hth
I am kicking machines with cobbler to install redhat 6 and also put in hostname.
I am managing my machines with /etc/hosts file right now (may switch to dns in the future) and would like to have puppet (or anything that will work) parse through /etc/hosts and find my ip address and hostname (based on the hostname supplied by cobbler at kickstart). The trick is that some machines have multiple IP's and hostnames in the /etc/hosts file, like such:
# Maintenance Network
192.168.80.192 testsrv01-maint
192.168.80.193 testsrv02-maint
192.168.80.194 testsrv03-maint
# Lights Out Network
192.168.120.192 testsrv01-ilo
192.168.120.193 testsrv02-ilo
192.168.120.194 testsrv03-ilo
# Primary Data Network
192.168.150.192 testsrv01-pri
192.168.150.193 testsrv02-pri
192.168.150.194 testsrv03-pri
# Secondary Data Network
192.168.200.192 testsrv01-sec
192.168.200.193 testsrv02-sec
192.168.200.194 testsrv03-sec
I need to capture each ip and hostname pair (in a line) and transpose into /etc/sysconfig/network-scripts/ifcfg-eth* (eth1, eth2, eth3, ...). Puppet will need to create as many ifcfg-eth* files as there are matches in /etc/hosts for the hostname.
I just need puppet to append the $IP and $hostname to the ifcfg-eth file, the rest of the content is common.
So how would I get 4x ifcfg-eth files for 'testsrv01', with puppet?
Puppet is a very ill fit for this task. This calls for a script, which Puppet does not support. Puppet allows you to declare a piece of machine state that can portably enforced on different platforms.
The task is simple enough for a Shell or Perl Script.
With Puppet, it would entail the following scripts
writing custom facts to retrieve each address / hostname pair
devising a defined type to render such a pair into an ethX file
You'd possibly even need to generate the respective manifest, so that the appropriate interface index is chosen for each address.
All things considered, you would not make use of Puppet's strengths and suffer some of its weaknesses.
Have a text file w/ around 3 million URL's of sites I want to block.
Trying to ping them one by one (yes, I know it is going to take some time).
Have a script (yes, I am a bit slow in BASH) which reads the lines one at a time from text file.
Obviously cannot print text file here. Text file was created >> w/ Python some time ago.
Problem is that ping returns "unknown host" w/ every entry. If I make a smaller file by hand using the same entries the script works. I thought it may be a white space or end of line issue so tried addressing that in script. What could the issue possibly be?
#!/bin/bash
while read line
do
li=$(echo $line|tr -d '\n')
li2=$(echo $li|tr -d ' ')
if [ ${#line} -lt 2 ]
then
continue
fi
ping -c 2 -- $li2>>/dev/null
if [ $? -gt 0 ]
then
echo 'bad'
else
echo 'good'
fi
done<'temp_file.txt'
Does the file contains URLs or hostnames ?
If it contains URLs you must extract the hostname from URLs before pinging:
hostname=$(echo "$li2"|cut -d/ -f3);
ping -c 2 -- "$hostname"
Ping is used to ping hosts. If you have URLs of websites, then it will not work. Check that you have hosts in your file , example www.google.com or an IP address and not actual full website urls. If you want to check actual URLs, use a tool like wget and another tool like grep/awk to grab for errors like 404 or others. Last but not least, people who are security conscious will sometimes block pinging from the outside, so take note.
C heck if the file contains windows-style \r\n line endings: head file | od -c
If so, to fix it: dos2unix filename filename
I wouldn't use ping for this. It can easily be blocked, and it's not the best way to check for either ip addresses or if a server presents web pages.
If you just want to find the corresponding IP, use host:
$ host www.google.com
www.google.com is an alias for www.l.google.com.
www.l.google.com has address 209.85.149.106
www.l.google.com has address 209.85.149.147
www.l.google.com has address 209.85.149.99
www.l.google.com has address 209.85.149.103
www.l.google.com has address 209.85.149.104
www.l.google.com has address 209.85.149.105
As you see, you get all the IPs registered to a host. (Note that this requires you to parse the hostname from your urls!)
If you want to see if a URL points at a web server, use wget:
wget --spider $url
The --spider flag makes wget not save the page, just check that it exists. You could look at the return code, or add the -S flag (which prints the HTTP headers returned)
I'm in the process of maintaining a Ruby on Rails app and am looking for an easy way to find the hostname or IP address of the box I'm on (since it's a VM and new instances may have different hostnames or IP addresses). Is there a quick and easy way to do this in Ruby on Rails?
Edit: The answer below is correct but the clarification Craig provided is useful (see also provided link in answer):
The [below] code does NOT make a
connection or send any packets (to
64.233.187.99 which is google). Since UDP is a stateless protocol connect()
merely makes a system call which
figures out how to route the packets
based on the address and what
interface (and therefore IP address)
it should bind to. addr() returns an
array containing the family (AF_INET),
local port, and local address (which
is what we want) of the socket.
Hostname
A simple way to just get the hostname in Ruby is:
require 'socket'
hostname = Socket.gethostname
The catch is that this relies on the host knowing its own name because it uses either the gethostname or uname system call, so it will not work for the original problem.
Functionally this is identical to the hostname answer, without invoking an external program. The hostname may or may not be fully qualified, depending on the machine's configuration.
IP Address
Since ruby 1.9, you can also use the Socket library to get a list of local addresses. ip_address_list returns an array of AddrInfo objects. How you choose from it will depend on what you want to do and how many interfaces you have, but here's an example which simply selects the first non-loopback IPV4 IP address as a string:
require 'socket'
ip_address = Socket.ip_address_list.find { |ai| ai.ipv4? && !ai.ipv4_loopback? }.ip_address
From coderrr.wordpress.com:
require 'socket'
def local_ip
orig, Socket.do_not_reverse_lookup = Socket.do_not_reverse_lookup, true # turn off reverse DNS resolution temporarily
UDPSocket.open do |s|
s.connect '64.233.187.99', 1
s.addr.last
end
ensure
Socket.do_not_reverse_lookup = orig
end
# irb:0> local_ip
# => "192.168.0.127"
Try this:
host = `hostname`.strip # Get the hostname from the shell and removing trailing \n
puts host # Output the hostname
A server typically has more than one interface, at least one private and one public.
Since all the answers here deal with this simple scenario, a cleaner way is to ask Socket for the current ip_address_list() as in:
require 'socket'
def my_first_private_ipv4
Socket.ip_address_list.detect{|intf| intf.ipv4_private?}
end
def my_first_public_ipv4
Socket.ip_address_list.detect{|intf| intf.ipv4? and !intf.ipv4_loopback? and !intf.ipv4_multicast? and !intf.ipv4_private?}
end
Both return an Addrinfo object, so if you need a string you can use the ip_address() method, as in:
ip= my_first_public_ipv4.ip_address unless my_first_public_ipv4.nil?
You can easily work out the more suitable solution to your case changing the Addrinfo methods used to filter the required interface address.
Simplest is host_with_port in controller.rb
host_port= request.host_with_port
This IP address used here is Google's, but you can use any accessible IP.
require "socket"
local_ip = UDPSocket.open {|s| s.connect("64.233.187.99", 1); s.addr.last}
Similar to the answer using hostname, using the external uname command on UNIX/LINUX:
hostname = `uname -n`.chomp.sub(/\..*/,'') # stripping off "\n" and the network name if present
for the IP addresses in use (your machine could have multiple network interfaces),
you could use something like this:
# on a Mac:
ip_addresses = `ifconfig | grep 'inet ' | grep -v 127.0.0.1 | cut -d' ' -f 2`.split
=> ['10.2.21.122','10.8.122.12']
# on Linux:
ip_addresses = `ifconfig -a | grep 'inet ' | grep -v 127.0.0.1 | cut -d':' -f 2 | cut -d' ' -f 1`.split
=> ['10.2.21.122','10.8.122.12']
The accepted answer works but you have to create a socket for every request and it does not work if the server is on a local network and/or not connected to the internet. The below, I believe will always work since it is parsing the request header.
request.env["SERVER_ADDR"]
Put the highlighted part in backticks:
`dig #{request.host} +short`.strip # dig gives a newline at the end
Or just request.host if you don't care whether it's an IP or not.
You will likely find yourself having multiple IP addresses on each machine (127.0.0.1, 192.168.0.1, etc). If you are using *NIX as your OS, I'd suggest using hostname, and then running a DNS look up on that. You should be able to use /etc/hosts to define the local hostname to resolve to the IP address for that machine. There is similar functionality on Windows, but I haven't used it since Windows 95 was the bleeding edge.
The other option would be to hit a lookup service like WhatIsMyIp.com. These guys will kick back your real-world IP address to you. This is also something that you can easily setup with a Perl script on a local server if you prefer. I believe 3 lines or so of code to output the remote IP from %ENV should cover you.
io = IO.popen('hostname')
hostname = io.readlines
io = IO.popen('ifconfig')
ifconfig = io.readlines
ip = ifconfig[11].scan(/\ \d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3}\ /)
The couple of answers with require 'socket' look good. The ones with request.blah_blah_blah
assume that you are using Rails.
IO should be available all the time. The only problem with this script would be that if ifconfig is output in a different manor on your systems, then you would get different results for the IP. The hostname look up should be solid as Sears.
try: Request.remote_ip
remote_ip()
Determine originating IP address. REMOTE_ADDR is the standard but will
fail if the user is behind a proxy. HTTP_CLIENT_IP and/or
HTTP_X_FORWARDED_FOR are set by proxies so check for these if
REMOTE_ADDR is a proxy. HTTP_X_FORWARDED_FOR may be a comma- delimited
list in the case of multiple chained proxies; the last address which
is not trusted is the originating IP.
Update:
Oops, sorry I misread the documentation.