Can I use google -- specifically i am thinking of the google ajax api -- to enumerate a list of host names of websites that are hosted on a particular IP address.
Note
Yes, I know that other mechanisms, such as MSN search and obviously DNS services can be used, but I am specificially looking for whether a google solution exists.
AFAIK Google doesn't give out the IP addresses in its search results (unless the URL is only accessible from IP address rather than a host name).
I know you only want Google solutions, but have you tried My IP Neighbors? You put in a URL or IP and it gives you the sites also hosted on the IP.
Not sure you can do with google ajax api; however I think that the best solution would be more oriented towards a sysadmin job (thus ask serverfault..), mainly:
find which nameserver is authoritative for such website
find out which other domains using that nameserver as authoritative
simply because websites hosted on the same server are often served by the same DNS.
On a side note, since a DNS can be authoritative for other domains (not hosted on that IP), you might want to double-check that list and do a lookup on all domains, filtering out those that are hosted on a different server.
This leaves open the question of load balancing, tho: what if a domain is hosted on more than one server?
The answer is left as an exercise to the reader. :)
I searched through Google's forums for SEO Q&A and technical Q&A. The issue of whether Google captures IP addresses is not directly addressed. However, there is at least one answer which suggests Google doesn't care about IP addresses (see squibble's second response.)
Related
See screenshot below. Airbnb knew my location was in Los Angeles. I visited the page in incognito mode. I'm not logged in. I did not consent to geolocation via HTML5 (the browser didn't even prompt me). So how do sites like Airbnb know my location?
My guess is with my IP address. If that's the case, could someone show me how I could replicate that if I were to build a site? Is there an API by Google or someone that provides that service? As far as I can tell the majority of Google Maps' API requires some sort of user input.
Airbnb_screenshot
How do sites like Airbnb know my location without my consent?
Through IP Address Geolocation. If you visit https://www.ipinfo.io, you will see that that your geolocation estimate is fairly accurate. But you haven't provided that service any kind of location permission, hence the word "estimate".
For IP Geolocation:
They don't need a permission for IP Geolocation through HTML5
They only need your IP address
To populate the location information from the IP address, the IP geolocation service provider will aggregate data from various sources and perform analytics to come up a with an estimated location which is accurate on a city or postal code level.
But because this data is collected from IP addresses, changing your IP address through a VPN or hosting can result in inaccurate IP geolocation address. But IPinfo can detect VPN usage as well:
disclaimer: I am the developer success manager at IPinfo
Nowadays, it quite common to geolocate users based on their IP addresses. Despite what people may argue, IP geolocation, especially country estimation is really accurate (more than 99% for IP addresses assigned in developed countries).
Some users may hide their real IP address using a proxy. That's why services such as Ipregistry (disclaimer: I run the service) include threat data along with IP geolocation data. This way you can detect suspicious IP addresses.
Getting information is as simple as calling an HTTP endpoint:
https://api.ipregistry.co/?key=tryout
Okay, so we implement Recaptcha in production. We get errors because it can't reach the IP address it needs to use the service. We open a port for the IP address to reach Google. No problem. We do that and configure that IP address explicitly to work. It works great. Then, the next day, we start getting errors again because Recaptcha is using a different IP address. I can allow requests from that IP address, too, but now I'm unsettled. Where are these addresses coming from? How do I configure this to work reliably?
Recatpcha from Google can use any Google IP address and there are lots of them.
Ran this from Windows:
_netblocks.google.com text =
nslookup -type=TXT _netblocks.google.com
"v=spf1 ip4:216.239.32.0/19 ip4:64.233.160.0/19 ip4:66.249.80.0/20 ip4:72.14.192.0/18 ip4:209.85.128.0/17 ip4:66.102.0.0/20 ip4:74.125.0.0/16 ip4:64.18.0.0/20 ip4:207.126.144.0/20 ip4:173.194.0.0/16 ?all"
That's all the network Google uses currently. These can change so check them often.
Google suggest allowing port 80 to all IPs outbound, this highly insecure. They recommend going through a proxy server but again that is highly insecure if your web server is an DMZ. Proxy aware trojans do exist. All that need to be done is exploit a vulnerability to execute arbitrary code and you can create reverse connection on port 80 through a proxy server to download the payload. Then it is trivial to escalate privileges and own the box. I don't mean just Windows servers but Linux as well. I've done it in lab environment on security was on. It's really easy to do.
This is the Google website I got this from:
http://code.google.com/p/recaptcha/wiki/FirewallsAndRecaptcha
I wanted to append to this answer with more recent information. The documentation that Chris is pointing to does not include all of the TXT records necessary to dig (thanks Google):
_netblocks2.google.com (IPv6 subnets)
_netblocks3.google.com (Additional IPv4 subnets)
In my particular case, the _netblocks3 entry contained 2 large /19's that made my initial rule ineffective
(I found additional references here: https://support.google.com/a/answer/60764?hl=en)
Perhaps you should be using a hostname rather than IP
I often visit various sites and like to do so anonymously from behind a proxy. However, it seems as if some websites are still able to detect my real IP address. I know this because they use the IP address to attempt to geolocate me for services.
How did they actually get my real IP Address if I am using a proxy?
How can I truly hide myself?
Most proxies you'll find online are so-called 'transparant proxies'. This means they identify themselves as proxy and even give your real IP-address to the site.
These proxies are commonly used to load pages on the proxy-server instead of your PC.
IP proxies won't protect you at all, to be honest.
What you need are VPN's (Virtual Private Networks) a.k.a. Network Tunnel.
VPN's allow you to do everything online via an external PC inside that network. You'll get that PC's IP and all loading etc happens on that PC.
Not all VPN's are undetected. If you always want to browse the web anonymously, you'll need to find some HQ VPN's (barely for free) and you'll need to change VPN frequently.
Try searching on HideMyAss, SecurityKISS, ... or maybe even the TOR-network
I have been search for a host that has static/fixed ip address. Right now i use mediatemple's grid server and because it is a shared environment the ip address is dynamic and changes randomly.
My issue is that i have a script on my sever that fetches data from an ftp site. To access the data via ftp, i have to authenticate using an ip address, username and password. Because the ip address is ever changing, i cannot access the data.
One alternative was asking the data company if they would accept a range of ip address or a wildcard but they will not. they will only approve one ip address for download.
The other option was to purchase a self managed server. I do not want to manage my own server nor do i have the expertise to do so. So I need a solution like a shared environment (low cost) with a cpanel but one that has fixed ip address.
Any suggestions would be much appreciated.
You should look for "managed server" or "VPS" solutions.
Which host you choose is up to you. Many hosts offer such solutions, and a smaller host might fit your budget better than a "big name" hosting company.
This is really surprising ... usually in a shared hosting account where your site is hosted on a shared IP, the shared IP address is always fixed it rarely changes. I think you're right now on CDN hosting, so your data delivery IP address varies according to your VISITOR location.
I am hosting my various sites like thewebhostingdir.com and etc at http://www.accuwebhosting.com/ ... on shared hosting and their IP address is fixed since the hosting setup.
I suggest you to change your hosting provider. You don't need VPS at this moment.
I found a solution. Godaddy vps virtual server starting at only $29.
so I'm looking for a server host for a website that I'm building. Generally I know from continually visiting sites which ones I like and which I don't. I think this is a much better way than simply measuring ping times to determine speed.
So I want to know if there's a way to find out which hosting companies are hosting certain domains. Is this possible? whois.domaintools.com tells you certain information about the namesevers and the other about the DNS information about the host IP location. This is fine but I still don't get the website URL where I too can sign up for hosting. Often times the IP location name resolves to something really formal like XYZ Mallard Group Company LTD so this is basically useless to me. I need the hosting web URL instead.
Anyone got any ideas how I can find a domain's server hosting website url?
Thanks,
Marv
http://www.whois.net/ Will tell you the DNS (which a lot of time has the one you can sign up for, i.e. dns.dreamhost.com points you to dreamhost.com), and it also gives you more info about the registrar.
Finding hostnames of Ip's of server and DNS will help you to identify hosting provider