Finding Websites From Company Name - asp.net-web-api

I've got a list of 6,000 company names (along with their headquarters address) and I need to find the web address for each of them. I'm considering using the Google Web API (obviously this will take a few days as only 1,000 queries per day are allowed) to do this(search for "COMPANY_NAME CITY STATE") and then take the first result. However I'm not 100% sure this will work, and I feel like there's a better way. I can do this in any language I know really, C++, Java, PHP, Python. This only has to be run once.
How would I use WHOIS to do this? I know how I would do it if I already knew the URL, but not the other way around(name to URL). And what would I do if it were privately registered?
BTW, these are US businesses.

You can use WHOIS instead of Google API for it.

Use Amazons Mechanical Turk. It's perfect for these kinds of tasks which can be hard to automate and typically need a person to validate them. It will cost a little but it should be manageable, depending on how bad you want the results.

Related

Multi-Language Websites

Can anyone recommend a good option to translate websites into Spanish? We tried using the Google translate plugin but the translation was so rough (very inaccurate, bordering on embarrassing the company) we had to hire a company to refine the translation so that it was much more accurate which makes for an extremely inefficient process for updating the site moving forward.
We're in health insurance, so the language we're translating is very specialized in nature and needs to be accurate for our members. To make it even more complicated, the Google Translate plugin happens instantly, so the translation is live before we have a chance to refine it before users can see it. In other words, there's no way to refine the translation before you make the content visible to users in the production environment. This is a legal regulatory requirement for Covered California and the Affordable Care Act, so it has to be a top notch implementation.
Short of a proxy solution that intercepts the content before it hits the production site or a separate site coded in Spanish, I'm not sure what other solutions exist if any. Ideas? The separate site solution is also problematic because it requires a bilingual staff and it doubles the work because both environments have to mirror each other exactly at all times.
Recommendations? Ideas? Any suggestions based on experience are most welcome!
Hire developer - he will describe all you need. You will never do it by your own. If you already have - hire new one, he will know how to do it. Question is very spiciefied but any (let's take for example php) php-engine (framework) or even custom php-engine can be updated the way you want.
Preview before upload to public? Easy! Change by moderator|admin values of translations? Easy! Main thing that each sentence (or even paragraph) you will describe by your own... I don't want describe all mechanism of it - hire developer and he will do all you need. $)

Best/fastest way to find current user location on Windows Phone App

I am trying to figure out what is the fastest way the figure out the current user Country. I know there are a lot of location examples online, my actual is question: since I don't care much about precision, is it OK if I use RegionInfo.CurrentRegion.DisplayName; to figure out the user country?
What are the downsides of using this approach (besides precision)?
No, you shouldn't use RegionInfo.CurrentRegion for that.
For example, I live in Montenegro, but RegionInfo.CurrentRegion on my phone will return USA, which is completely irrelevant being almost 4000 miles away from here.
You have to use Geolocator class for that. Here's some sample code.

Quova API anyone? it's a complete disaster, anyone knows better ip-geolocation-services?

i need a geolocation service and i wanted to try some of them before buying anything for my client.
i tried api.ipinfodb.com and is pretty good...
then i recently tried Quova APi, that as far as i remeber Quova was considered good...
well...i tried it and the result is really sloppy... the zipcode with ipinfodb.com was perfect, whereas Quova was quite distant...
also the XML of the first was good formatted, whereas Quova gives you all lowercase name..why? shouldn be the city name cpaitalize ? i know i can do it with php but with name syou have to be careful...sound sjust sloppy to me...
I wonder if the paid Quova service is the same..
I'm actually the product manager for Quova, so I hope I can help. Sorry you're having problems with the API.
To answer your first question about the zip code, no vendor can be right 100% of the time, and there will always be individual cases where we are wrong and someone else is right, or we are right and someone else is wrong. We do provide confidence factors to help you decide how confident we are in the assignments we make, which helps customers make better decisions about the data. Our customers stay with us because they know that the overall quality of our data outperforms the other vendors they've tried. If you respond with the actual IP addresses and ZIP codes that you think are wrong, I can have them investigated.
With regard to our data being all lowercase, we made that decision a long time ago to make the data predictable and to make comparisions with our data easier. I know there are use cases where having the correct capitalization of place names would be valuable, and lowercasing strings is easy enough if you have to do that, so we're considering how to provide capitalized names without impacting current customers who might be relying on the data in its current format. One thing you can do in the meantime is use the Lat/Long to lookup the place name with a service like geonames.org.
To answer your last question, yes, the data is also lowercase in the commercial service.

Using flickr to get photos of a specific location and put together a model

I've read about systems which use the Flickr database of photos to fill in gaps in photos (http://blogs.zdnet.com/emergingtech/?p=629).
How feasible is a system like this? I was toying with the idea (not just a way of killing time but as a good addition to something I am coding) of using Flickr to get photos of a certain entity (in this case, race tracks) and reconstruct a model. My biggest concern is that there aren't enough photos of a particular track and even then, it would be difficult to tell if two photos are of the same part of the racetrack, in which case one of them may be irrelevant.
How feasible is something like this? Is it worth attempting by a sole developer?
Sounds like you're wanting to build a Photosynth style system - check out Blaise Aguera y Arcas' demo at TED back in 2007. There's a section about 4 minutes in where he builds a model of the Sagrada Família from photographs.
I say +1 for photosynth answer, its a great tool. Not sure how well you could incorporate it into your own app though.
Its definately feasable. Anything is possible. And yes, doable for a single developer, just depends how much free time you have. It would be great to see something like this integrated into Virtual Earth or Google Maps Street View. Someone who could nail some software like this could help 3D model the entire world based purely on photographs. That would be a great product and make any single developer rich and famous.
So get coding. :)
I have plenty of free time, as I am in between jobs.
One way to do it is to get an overhead view of the track layout, make a blueprint based on this model, and then get one photo of the track and mimic the track's road colour. That would be a start.
LINQ to Flickr on codeplex has a great API and would be helpful for your task.

How do you perform address validation? [closed]

Closed. This question does not meet Stack Overflow guidelines. It is not currently accepting answers.
We don’t allow questions seeking recommendations for books, tools, software libraries, and more. You can edit the question so it can be answered with facts and citations.
Closed 2 years ago.
The community reviewed whether to reopen this question 11 months ago and left it closed:
Original close reason(s) were not resolved
Improve this question
Is it even possible to perform address (physical, not e-mail) validation? It seems like the sheer number of address formats, even in the US alone, would make this a fairly difficult task. On the other hand it seems like a task that would be necessary for several business requirements.
Here's a free and sort of "outside the box" way to do it. Not 100% perfect, but it should reject blatantly non-existent addresses.
Submit the entire address to Google's geocoding web service. This service attempts to return the exact coordinates of the location you feed it, i.e. latitude and longitude.
In my experience if the address is invalid you will get a result of 602 from the service. There's definitely a possibility of false positives or false negatives, but used in conjunction with other consistency checks it could be useful.
(Yahoo's geocoding web service, on the other hand, will return the coordinates of the center of the town if the town exists but the rest of the address is bogus. Potentially useful as long as you pay close attention to the "precision" field in the result).
There are a number of good answers in here but most of them make the assumption that the user wants an "API" solution where they must write code to connect to a 3rd-party service and/or screen scrape the USPS. This is all well and good, but should be factored into the business requirements and costs associated with the implementation and then weighed against the desired benefits.
Depending upon the business requirements and the way that the data is received into the system, a real-time address processing solution may be the best bet. If a real-time solution is required, you will want to consider the license agreement and technical limitations of the Google Maps/Bing/Yahoo APIs. They typically limit the number of calls you can make each day. The USPS web tools API is the same in additional they restrict how/why you can use their system and how you are allowed to use the data thereafter.
At the same time, there are a handful of great service providers that can easily process a static list of addresses. Essentially, you give the service provider a CSV file or Excel file, they clean it up and get it back to you. It's a one-time deal with no long-term commitment or obligation—usually.
Full disclosure: I'm the founder of SmartyStreets. We do address verification for addresses within the United States. We are easily able to CASS certify a list and we also offer a address verification web service API. We have no hidden fees, contracts, or anything. You use our service until you no longer need it and you can walk away. (Unlike cell phone companies that require a contract.)
USPS has an address cleaner online, which someone has screen scraped into a poor man's webservice. However, if you're doing this often enough, it'd be a better idea to apply for a USPS account and call their own webservice.
I will refer you to my blog post - A lesson in address storage, I go into some of the techniques and algorithms used in the process of address validation. My key thought is "Don't be lazy with address storage, it will cause you nothing but headaches in the future!"
Also, there is another StackOverflow question that asks this question entitled How should international geographic addresses be stored in a relational database.
In the course of developing an in-house address verification service at a German company I used to work for I've come across a number of ways to tackle this issue. I'll do my best to sum up my findings below:
Free, Open Source Software
Clearly, the first approach anyone would take is an open-source one (like openstreetmap.org), which is never a bad idea. But whether or not you can really put this to good and reliable use depends very much on how much you need to rely on the results.
Addresses are an incredibly variable thing. Verifying U.S. addresses is not an easy task, but bearable, but once you're going for Europe, especially the U.K. with their extensive Postal Code system, the open-source approach will simply lack data.
Web Services / APIs
Enterprise-Class Software
Money gets it done, obviously. But not every business or developer can spend ~$0.15 per address lookup (that's $150 for 1,000 API requests) - a very expensive business model the vast majority of address validation APIs have implemented.
What I ended up integrating: streetlayer API
Since I was not willing to take on the programmatic approach of verifying address data manually I finally came to the conclusion that I was in need of an API with a price tag that would not make my boss want to fire me and still deliver solid and reliable international verification results.
Long story short, I ended up integrating an API built by apilayer, called "streetlayer API". I was easily convinced by a simple JSON integration, surprisingly accurate validation results and their developer-friendly pricing. Also, 100 requests/month are entirely free.
Hope this helps!
I have used the services of http://www.melissadata.com Their "address object" works very well. Its pricey, yes. But when you consider costs of writing your own solutions, the cost of dirty data in your application, returned mailers - lost sales, and the like - the costs can be justified.
For us-based address data my company has used GeoStan. It has bindings for C and Java (and we created a Perl binding). Note that it is a commercial product and isn't cheap. It is quite fast though (~300 addresses per second) and offers features like CASS certification (USPS bulk mail discount), DPV (Delivery point verification) flagging, and LON/LAT geocoding.
There is a Perl module Geo::PostalAddress, but it uses heuristics and doesn't have the other features mentioned for GeoStan.
Edit: some have mentioned 'doing it yourself', if you do decide to do this, a good source of information to start with is the US Census Tiger Data Set, which contains a lot of information about the US including address information.
As seen on reddit:
$address = urlencode('1600 Pennsylvania Avenue, Washington, DC');
$json = json_decode(file_get_contents("http://where.yahooapis.com/geocode?q=$address&flags=J"));
print_r($json);
Fixaddress.com service is available that provides following services,
1) Address Validation.
2) Address Correction.
3) Address spell correcting.
4) Correct addresses phonetic mistakes.
Fixaddress.com uses USPS and Tiger data as reference data.
For more detail visit below link,
http://www.fixaddress.com/
One area where address lookups have to be performed reliably is for VOIP E911 services. I know companies reliably using the following services for this:
Bandwidth.com 9-1-1 Access API MSAG Address Validation
MSAG = Master Street Address Guide
https://www.bandwidth.com/9-1-1/
SmartyStreet US Street Address API
https://smartystreets.com/docs/cloud/us-street-api
There are companies that provide this service. Service bureaus that deal with mass mailing will scrub an entire mailing list to that it's in the proper format, which results in a discount on postage. The USPS sells databases of address information that can be used to develop custom solutions. They also have lists of approved vendors who provide this kind of software and service.
There are some (but not many) packages that have APIs for hooking address validation into your software.
However, you're right that its a pretty nasty problem.
http://www.usps.com/ncsc/ziplookup/vendorslicensees.htm
As mentioned there are many services out there, if you are looking to truly validate the entire address then I highly recommend going with a Web Service type service to ensure that changes can quickly be recognized by your application.
In addition to the services listed above, webservice.net has this US Address Validation service. http://www.webservicex.net/WCF/ServiceDetails.aspx?SID=24
We have had success with Perfect Address.
Their database has all the US street names and street number ranges. Also acts as a pretty decent parser for free-form address fields, if you are lucky enough to have that kind of data.
Validating it is a valid address is one thing.
But if you're trying to validate a given person lives at a given address, your only almost-guarantee would be a test mail to the address, and even that is not certain if the person is organised or knows somebody at that address.
Otherwise people could just specify an arbitrary random address which they know exists and it would mean nothing to you.
The best you can do for immediate results is request the user send a photographed / scanned copy of the head of their bank statement or some other proof-of-recent-residence, because at least then they have to work harder to forget it, and forging said things show up easily with a basic level of image forensic analysis.
There is no global solution. For any given country it is at best rather tricky.
In the UK, the PostOffice controlls postal addresses, and can provide (at a cost) address information for validation purposes.
Government agencies also keep an extensive list of addresses, and these are centrally collated in the NLPG (National Land and Property Gazetteer).
Actually validating against these lists is very difficult. Most people don't even know exactly how their address as it is held by the PostOffice. Some businesses don't even know what number they are on a particular street.
Your best bet is to approach a company that specialises in this kind of thing.
Yahoo has also a Placemaker API. It is good only for locations but it has an universal id for all world locations.
It look that there is no standard in ISO list.
You could also try SAP's Data Quality solutions which are available in both a server platform is processing a large number of requests or as an embeddable SDK if you wanted to run it in process with your application. We use it in our application and it's very robust and scalable.
NAICS.com is coming out with an API that will add all kinds of key business data including street address. This would happen on the fly as your site's forms are processed. https://www.naics.com/business-intelligence-api/
You can try Pitney Bowes “IdentifyAddress” Api available at - https://identify.pitneybowes.com/
The service analyses and compares the input addresses against the known address databases around the world to output a standardized detail. It corrects addresses, adds missing postal information and formats it using the format preferred by the applicable postal authority. I also uses additional address databases so it can provide enhanced detail, including address quality, type of address, transliteration (such as from Chinese Kanji to Latin characters) and whether an address is validated to the premise/house number, street, or city level of reference information.
You will find a lot of samples and sdk available on the site and i found it extremely easy to integrate.
For US addresses you can require a valid state, and verify that the zip is valid. You could even check that the zip code is in the right state, but beyond that I don't think there are many tests you could run that wouldn't provide a lot of false negatives.
What are you trying to do -- prevent simple mistakes or enforcing some kind of identity check?

Resources