Is it illegal to encrypt data using a public key which has a use only for non commerical use comment on top - public-key-encryption

CONTEXT: We have a business customer to whom we need to send an encrypted file.
The business customer has send us a public key file that has a usage statement "Non commercial usage only". Our software is legit to encrypt stuff.
QUESTION: would it be illegal for us to encrypt stuff using client's public key

Best consult a lawyer. But a lot of those legal sounding notices are completely unenforceable, they're just there to scare people. In general, once someone sends something without a prior agreement, the receiver owns it entirely, as long as the sender has ownership of it. That is regardless of whether the sender changes their mind after the fact. Post conditions, which you can only read after receiving something even if they are sent at the same time, are not a prior agreement (you need the option to decline) before the transaction).
The important part is what rights the sender has. It's possible the sender doesn't have the rights to ask you to use the key as they asked you to. Again, a lawyer can clarify, I don't think you would be liable for that infraction, but the notice might be enough to mean you "should have known", and might have some liability.
Even better, ask your customer for clarification. I'm surprised you didn't do that in the first place. If they offer to indemnify you - that is, take responsibility for all liability you might incur by violating that notice - you should be in the clear, providing a lawyer agrees.
I'll say "lawyer" again, because the legal system takes about ten years full time study for a lawyer to just get started knowing it well enough to start practicing, and decades longer to get good at it. You can't expect anyone with a casual opinion about it to know enough.

Related

In a Web API, isn't providing a DELETE ALL too dangerous?

In studying Web API design (regardless of the specific technology), I often come across these two uses of the DELETE verb:
DELETE /SomeResource/123 /* deletes entity with ID 123 */
DELETE /SomeResource/ /* deletes all entities */
I always get the feeling that there's something wrong with providing the latter as an operation in most applications. In rare cases where the resource is trivial, sure, why not just blow the whole collection away without a second thought? And yes, I understand that typically it's the client app's job to present an "Are you sure?" confirmation. But I like to envision my API being driven in a safe way even by some low-level agent like Fiddler.
So is there some mechanism I'm missing, like a way for the server/API to initiate some kind of dialog with the client agent to get confirmation before blowing away 10,000 customer records?
EDIT
Yes, assume that I do want to provide the functionality to remove all entities in a given scenario, but feel compelled to avoid it because of the perceived danger (typos, not paying attention, etc; the things that a UI confirmation dialog is for)
You'll NEVER present the option to do a delete all function in a web api.
While there may be exceptions, its usually an awful idea to do this because it allows a user to perform an irreversible action. On the other hand, think of how awful it would be to have someone with ill intent find this endpoint!
Most libraries such as ASP.NET will require each method to be implemented explicitly before accepting a request (this is handled in ASP.NET during routing). So, if you don't provide DELETE /someresource/ the problem will never come up. This is particularly great, because you can't accidentally blow away data with a typo.
If you really, really want this functionality, you'll want to be 100% sure you have a reason to have it because this is a dangerous, dangerous endpoint to have.
Bottom line? I wouldn't provide it to protect your data.

How to get users to read error messages? [closed]

Closed. This question is opinion-based. It is not currently accepting answers.
Want to improve this question? Update the question so it can be answered with facts and citations by editing this post.
Closed 1 year ago.
Improve this question
If you program for a nontechnical audience, you find yourself at a high risk that users will not read your carefully worded and enlightening error messages, but just click on the first button available with a shrug of frustration.
So, I'm wondering what good practices you can recommend to help users actually read your error message, instead of simply waiving it aside. Ideas I can think of would fall along the lines of:
Formatting of course help; maybe a simple, short message, with a "learn more" button that leads to the longer, more detailed error message
Have all error messages link to some section of the user guide (somewhat difficult to achieve)
Just don't issue error messages, simply refuse to perform the task (a somewhat "Apple" way of handling user input)
Edit: the audience I have in mind is a rather broad user base that doesn't use the software too often and is not captive (i.e., no in-house software or narrow community). A more generic form of this question was asked on slashdot, so you may want to check there for some of the answers.
That is an excellent question worthy of a +1 from me. The question despite being simple, covers many aspects of the nature of end-users. It boils down to a number of factors here which would benefit you and the software itself, and of course for the end-users.
Do not place error messages in the status bar - they will never read them despite having it jazzed up with colours etc....they will always miss them! No matter how hard you'll try... At one stage during the Win 95 UI testing before it was launched, MS carried out an experiment to read the UI (ed - it should be noted that the message explicitly stated in the context of 'Look under the chair'), with a $100 dollar bill taped to the underside of the chair that the subjects were sitting on...no one spotted the message in the status bar!
Make the messages short, do not use intimidating words such as 'Alert: the system encountered a problem', the end-user is going to hit the panic button and will over-react...
No matter how hard you try, do not use colours to identify the message...psychologically, it's akin to waving a red-flag to the bull!
Use neutral sounding words to convey minimal reaction and how to proceed!
It may be better to show a dialog box listing the neutral error message and to include a checkbox indicating 'Do you wish to see more of these error messages in the future?', the last thing an end-user wants, is to be working in the middle of the software to be bombarded with popup messages, they will get frustrated and will be turned off by the application! If the checkbox was ticked, log it to a file instead...
Keep the end-users informed of what error messages there will be...which implies...training and documentation...now this is a tricky one to get across...you don't want them to think that there will be 'issues' or 'glitches' and what to do in the event of that...they must not know that there will be possible errors, tricky indeed.
Always, always, be not afraid to ask for feedback when the uneventful happens - such as 'When that error number 1304 showed up, how did you react? What was your interpretation' - the bonus with that, the end-user may be able to give you a more coherent explanation instead of 'Error 1304, database object lost!', instead they may be able to say 'I clicked on this so and so, then somebody pulled the network cable of the machine accidentally', this will clue you in on having to deal with it and may modify the error to say 'Ooops, Network connection disconnected'... you get the drift.
Last but not least, if you want to target international audiences, take into account of internationalization of the error messages - hence that's why to keep it neutral, because then it will be easier to translate, avoid synonyms, slang words, etc which would make the translation meaningless - for example, Fiat Ford, the motor car company was selling their brand Fiat Ford Pinto, but noticed no sales was happening in South America, it turned out, Pinto was a slang there for 'small penis' and hence no sales...
(ed)Document the list of error messages to be expected in a separate section of the documentation titled 'Error Messages' or 'Corrective Actions' or similar, listing the error numbers in the correct order with a statement or two on how to proceed...
(ed) Thanks to Victor Hurdugaci for his input, keep the messages polite, do not make the end-users feel stupid. This goes against the answer by Jack Marchetti if the user base is international...
Edit: A special word of thanks to gnibbler who mentioned another extremely vital point as well!
Allow the end-user to be able to select/copy the error message so that they can if they do so wish, to email to the help support team or development team.
Edit#2: My bad! Whoops, thanks to DanM who mentioned that about the car, I got the name mixed up, it was Ford Pinto...my bad...
Edit#3: Have highlighted by ed to indicate additionals or addendums and credited to other's for their inputs...
Edit#4: In response to Ken's comment - here's my take...
No it is not, use neutral standard Windows colours...do not go for flashy colours! Stick to the normal gray back-colour with black text, which is a normal standard GUI guideline in the Microsoft specifications..see UX Guidelines (ed).
If you insist on flashy colours, at least, take into account of potential colour-blind users i.e. accessibility which is another important factor for those that have a disability, screen magnification friendly error messages, colour-blindness, those that suffer with albino, they may be sensitive to flashy colours, and epileptics as well...who may suffer from a particular colours that could trigger a seizure...
Show them the message. Due dilligence and all, but log every error to a file. Users can't remember what they were doing or what the error message was seconds after the event, it's like eye-witness accounts of perpatrators.
Provide a good way to allow them to email or upload the log to you so that you can assist them in reconciling the issue. If it's a web application: even better, you can be receiving information about the situation ahead of anyone even reporting the problem.
Short answer: You can't.
Less short answer: Make them visible, relevant, and contextual (highlight what they messed up). But still, you're fighting a losing battle. People don't read on computer screens, they scan, and they've been trained to click the buttons until the dialog boxes go away.
We put a simple memorable graphic in the error box: not an icon, a fairly large bitmap, and nothing like the standard Windows message icons. Nobody can ever remember the wording of a messagebox (most won't even read it if the box has an "OK" button they can press), but most people DO remember the picture they saw. So our support people can ask the customer "did you see the coffee-drinking guy?" or "did you see the empty desk?". At least that way we know roughly what went wrong.
Depending on your user base, writing funny/rude/personal error messages can work great.
For instance, I wrote an application which allowed our HR people to better track the hire/fire dates of employees. [we were a small company, very laid back].
When they entered wrong dates I would write:
Hey dumb ass, learn how to enter a date!
EDIT: Of course a more helpful message is to say: "Please enter date as mm/dd/yyyy" or perhaps in code to try and figure out what they entered and if they entered "blahblah" to show an error. However, this was a very small application for an HR person I knew personally. Hence again people, read the first line of this post: Depending on your user base...
I recently worked on an Art Institute project, so the error messages were geared towards the audience, such as:
Most art before the Baroque period was
unsigned. However, we’re beyond the
Baroque period now, so all fields must
be completed.
Basically gear it to your audience if at all possible, and avoid boring as all unearthly general errors such as: "please enter email" or "please enter valid email".
Alerts/popups are annoying, that's why everyone hits the first button they see.
Make it less annoying. Example: if the user entered the date incorrectly, or entered a text where numbers are expected, then DON'T popup a message, just highlight the field and write a message somewhere around it.
Make a custom message box. Do not ever use the default message box of the system, for example Windows XP message boxes are annoying themselves. Make a new colored message box, with a different background color than system default.
Very Important: do not insist. Some message boxes use the Modal dialog and insist on making you read it, this is very annoying. If you can make the message box appear as a warning message it would be better, for example, Stack Overflow messages that appear right on the top of the page, informing but not annoying.
UPDATE
Make the message meaningful and helpful. For example, do not write something like, "No Keyboard found, press F1 to continue."
The best UI design will be where you virtually never show an error message. The software should adapt to the user. With that sort of a design, an error message will be novel and will grab the users attention. If you pepper the user with senseless dialogs like that you're explicitly training them to ignore your messages.
In my opinion and experience, it's the power users, who do not read error messages. The nontechnical audience I know reads every message on the screen most carefully and the problem at this point mostly is: They don't understand it.
This point may be the cause of your experience, because at some point they will stop reading them, because "they don't understand it anyway", so your task is easy:
Make the error message as easy to understand as possible and keep the technical part under the hood.
For example I transfer a message like this:
ORA-00237: snapshot operation disallowed: control file newly created
Cause: An attempt to invoke cfileMakeAndUseSnapshot with a currently mounted control file that was newly created with CREATE CONTROLFILE was made.
Action: Mount a current control file and retry the operation.
to something like:
This step could not be processed due to momentary problems with the database. Please contact (your admin|the helpdesk|anyone who can contact the developer or admin to solve the problem). Sorry for the inconvenience.
Show users that the error message has a meaning, and it's a way to provide assistance to them and they will read it. If it's just jargon-bable or generic nonsense message they will learn to dismiss them quicly.
I have learned that is very good practice to include an error dialog with default action to send (eg. via email) detailed diagnostic info, if you quickly respond to those emails with valuable information or workaround, they will worship you.
This is also a great learning tool. In future versions you can solve known-issues or at least provide in-place workaround info. Until then users will learn that this message is caused by X and the problem can by solved by Y - all because someone did explain it to them.
Of course this won't work on a large scale application, but works very well in enterprise applications with few hundred users, and in a lean agile, release early release often, environment.
EDIT:
Since you have a broad user base I recommend to provide software that does what users are/can expect it to do, eg. do not show them eroror message if phone number is not formatted well, reformat if for them.
I personally like software that does not make me think, and when occasionally there is nothing you (the developer) can do to interpret my intention, provide a very well written (and reviewed by actual users) messages.
It's common knowlege that people do not read documentation (did you read instructions back-to-back do when you did plugged in household appliance?), they try a way to get results quickly, when failed you have to grab their attention (eg. disable default button for a while) with meaningful and helpful info. They don't care about your sofware failure, they want to get results, now.
One good tip I've learned is that you should write a dialog box like a newspaper article. Not in the size-sense, but in the importance-sense. Let me explain.
You should write the most important things to read, first, and provide more detailed information second.
In other words, this is no good:
There was a problem loading the file, the file might have been deleted, or
it might be present on a network share that you don't have access to at
your present location.
Do you want to retry opening the file?
Instead, change the order:
Problem loading file, do you want to retry?
There was a problem loading the file, the file might have been deleted, or
it might be present on a network share that you don't have access to at
your present location.
This way, the user can read just as much as he wants, or bothers, and still have an idea about what's being asked.
To start, write error messages that users can actually understand. "Error: 1023" is not good example. I think better way is logging the error, than showing it to the user with some "fancy" code. Or if logging is not possible, give the users proper way to send the error details to the support department.
Also, be short and clear enough. Do not include some technical details. Do not show them information that they cannot use. If possible provide a workaround for the error. If not provide a default route, that should be taken.
If your application is a web app, designing custom error pages is a good idea. They stress users less, take SO for example. You can get some ideas how to design a good error page here: http://www.smashingmagazine.com/2007/07/25/wanted-your-404-error-pages/
Make them fun. (It seemed relevant, given the site we're on :) )
One thing I'd like to add.
Use verbs for your action buttons to close your error messages rather than exclamations, example don't use "Ok!" "Close" etc.
Unless you can provide the user some simple work-around, don't bother showing the user an error message at all. There is just no point, since 90% of users won't care what it says.
On the other hand If you CAN actually show the user a useful workaround, then one way to force them to read it is make the OK button become enabled after 10 seconds or so. Sort of how Firefox does it whenever you are trying to install a new plug-in.
If it is a total crash that you cannot gracefully recover from, then inform the user in very layman terms saying:
"I'm sorry we screwed up, we would like to send some information about this crash, will you allow us to do so? YES / NO"
In addition, try not to make your error messages longer than a sentence. When people (me included) see a whole paragraph talking about the error, my mind just shuts off.
With so much social media and information overload, people's mind freeze when they see a wall of text.
EDIT:
Someone once also recently suggested using comic strips along with whatever message you want to show. Such as something from Dilbert that may be close to the type of error you may have.
From my experience: you don't get users (especially non-technical ones) to read error messages. No matter how clear and understandable, bold, red and flashing the message is, that you display, most users will just click anything away that they're not used to, even if it's "Do you really want to delete everything?". I have seen users click the "window close"-icon instead of "OK" or "cancel" even though they didn't even know which option they chose by doing so ...
If you really need to force users to read what you're displaying, I'd suggest a JavaScript-Countdown until a button is clickable. That way the user will hopefully use the waiting time to really read what he's supposed to. Be careful though: most users will be even more annoyed by that :)
I furthermore like your idea of a "read more"-link, although I doubt that will get users more interested that just want to get rid of the message by all means ...
Just for the record: there are users that DO read error messages but are so afraid that they won't do anything with it. I once had a support call where the customer would read an error message to me, asking me, what he should do. "Well, what are your options?", I asked. "The window only has an 'OK'-button.", he replied. ... mmh, hard one :)
I often display the error in red (when the design allows it).
Red stands for "alert", etc. so it's more often read.
Well, to answer your question directly: Don't have your programmers write your error messages. If you follow this one piece of advice, you'd save, cumulatively, thousands of hours of user angst and productivity and millions of dollars in technical support costs.
The real goal, however, should be to design your application so users can't make mistakes. Don't let them take actions that lead to error massages and require them to back up. As a simple example, in a web form that requires all its fields to be filled in, instead of popping up an error message when users click on the Send button, don't enable the Send button until all the field contain valid content. It means more work on the back side, but it results in a better user experience.
Of course, that's a bit of an ideal world. Sometimes, program errors are unavoidable. When they do occur, you need to provide clear, complete, and useful information, and most importantly, don't expose the system to user and don't blame users for their actions.
A good error message should contain:
What the problem is and why it happened.
How to resolve the problem.
One of the worst things you can do is simply pass system error messages through to users. For example, when your Java program throws an exception, don't simply pass the programmer-ese up to the UI and expose it to the user. Catch it, and have a clear message created by your user assistance developer that you can present to your user.
I was lucky enough, on my last job, to work with a team of programmer who wouldn't think of writing their own error messages. Any time they found themselves in a situation where one was required and the program couldn't be designed to avoid it (often because of limited resources), they always came to me, explained what they needed, and let me create an error message that was clear and followed company style. If that was the default mindset of every programmer, the computing world would be a far, far better place.
Less errors
If an application throws vomit at you on a regular basis, you become immune to it, and errors become irritating background muzak. If an error is a rare event, it will garner more attention.
Quosh anything which isn't a major deal, throw out all those warnings, find ways of understanding user intent, take out the decisions wherever possible. I have a few apps which I continue to streamline in this way. Developers see every error as important, but this is not true from a user perspective. Look for the users' common response to a problem and capture that, deploy that as your response.
If you do need to raise an error: short, concise, low terror factor, no exclamation marks. Paragraphs are fail.
There's no silver bullet, but you need to socially engineer to make errors important.
We told users their manager had been contacted (which was a lie). It worked a little too well and had to be removed.
Adding an "Advanced" button that enables some more technical details will provide an incentive to read it for the part of the target audience that thinks itself as technical
I'd suggest that you give feedback (stating that the user made a mistake) immediately after the mistake is made. (For instance, when entering a value of a date field, check the value and, if it is wrong, make the input field visually different).
If there are errors on the page (I'm more into web development, hence I'm referring to it as a "page", but it can be also called "form"), show an "error summary", explaining that there were errors and a bulleted list of what exactly errors happened. However, if there are more than 5-6 words per message, those won't be read/understood.
How about making the button state "Click here to speak with a support technician who will assist you with this issue."
There are many websites that provide the option to speak with a real person.
I read a candidate for the most horrific solution on slashdot:
We have found that the only way to
make users take responsibility for
errors is to give them a penalty for
forcing the error to go away. For
starters, where possible, the error
wont actually close for them unless we
enter an admin password to make it go
away, and if they reboot to get rid of
it (Task Manager is disabled on all
client PC's) the machine will not open
the application that crashed for 15
minutes. Of course, this all depends
on the type of users you are dealing
with, as more technically adept users
wouldnt accept this kind of system,
but after trying for literally YEARS
to make users take responsibility for
crashes and making sure the IT
department is aware of them in order
to fix the issue before it gets too
hard to manage, these are the only
steps that worked. Now, all of our end
users are aware that if they ignore
errors, they are going to suffer for
it themselves.
"ATTENTION! ATTENTION! If you do not read error message you WILL DIE!"
Despite all the recommendations in the accepted answer, my users continued to click the first button they could find. So now I show this:
The user has to make a choice before the OK button appears
If he selects the 3rd option, he can continue, otherwise the application quits.

How to make sure the visitor is unique

Say you have a pay-site with some online courses. And you want to make sure that one person doesn't just buy access, and then give the username and password to all his friends, so they can do the courses for free.
How would you go about this?
What we've thought of so far:
IP tracking
SMS password for each entry
Max number of runs through each course
Any other suggestions?
It's impossible to get a perfect system to do what you want. You find yourself in a situation where the stronger you make your protection (to defend against cheating customers), the more you annoy all your customers (including the honest ones).
You're going to have to ask yourself at what point the extra protection actually reduces the value of your site to the point that you're losing more honest customers than you're winning customers by converting cheaters into honest (paying) customers.. It might well be that the optimal thing to do is to use cookies, and only take remedial action if you see two concurrent sessions from different IP addresses, since that's fairly likely to be caused by cheating (though not guaranteed; it could be a dual-homed customer).
There's no way you can absolutely, positively guarantee that users are unique - even if you had some way to uniquely identify users, like biometric data (which you don't), you'd still be unable to be certain the the client wasn't just spoofing that information.
The best you can hope to do is make it a hassle for someone to "cheat" the system. IP+SMS would probably do that, although it'd also probably annoy the heck out of your users (at least, the latter part).
Your best bet is probably just to log IPs used for each account - if the number goes above a certain threshold, flag it for inspection, and close the account if it looks like the info is being widely shared.
Associate an IP address with a cookie. Then associate that cookie with the user account and require use of the cookie to login. If the user logs in in with a different IP address then associate that new IP address with the cookie and ask for some sort of verification to authenticate the user.
There is no 100% guarantee at all. Someone can just sit next to the user who bought the access and read the site over his shoulder. Your methods are good (but I personally think that SMS-authorization is a little too much), but I'd suggest maximum personalization of the information you provide, so nobody except the payer can benefit from accessing it.
I'm sure some people would try to use cookies (assuming users don't change computers)

who (NOT WHY) should have the final say on the layout/design of a website that is developed in-house? [closed]

Closed. This question is opinion-based. It is not currently accepting answers.
Want to improve this question? Update the question so it can be answered with facts and citations by editing this post.
Closed 3 years ago.
Improve this question
forget the weird cases/companies, but in general, who, in your view, is the person who should be able to put his/her foot down and say "THIS is the layout. THIS is where the marketing message comes, THIS is where the price comes, THIS is where the testimonial comes"?? - the boss man, the template designer, users (split testing or something?)
I have never seen a never-ending series of stupid arguments and suggestions that lead up to a website that is a total disaster. That is what is happening at my work-place.
Technically, who should have the final say? I am very new to corporate world so I'd like to know how it is done.
Edit: when the client is paying, what they want goes in and what they don't want doesn't. Clear. What about when the website is built in-house? the top brass is acting as if the company is democratic and stuff when its actually not. So, going by their acclaimed democratic process, who should make the decision?
My company is product based and we develop our own website.
The question is WHO... not WHY. So please don't even begin to justify your answer or this question will be closed as argumentative or something. If 8 out of 10 replies say a particular person is responsible, there is no need for an argument there. If the replies are equally distributed, i'll take it that there is no ideal person who has the final say in the real world. But for god's sake please don't turn this into an argument! :D
It doesn't matter so much who it is that makes the decision, as long as there is someone.
What you are facing seems to be the situation where there is noone responsible for these decisions, so everyone is making decisions back and forth, and noone has the final say.
It's natural that the client should have the final say, however it's common that the client doesn't really know what they want or how to describe what they want. In that case someone else has to make the decisions on what should be presented to the client for approval.
Even if there is a democratic process during the development (which is usually a good way to progress), a company is not a democracy. In the end someone has to take the fruits of the process and decide what's good for the final product and what's not.
Ultimately it would be the business owner or the person elected to represent the business owners.
However, they would normally expect to be able to delegate the responsibility for such decisions to a single person further down the hierarchy. This person should have the authority to coordinate the required people and departments and make a decision. In an ideal world this person has the humility to realise that they aren't a web or UI designer and will allow the experts to advise them.
It sounds like the decision making hasn't been delegated to an individual but to a number of people (or the individual who has the authority isn't able or willing to use it). These seems to be relatively common.
Who's paying for it?
The Client! However its the job of the service provider to shape the clients ideas - However, there is always an odd client who has very firm ideas of what they want, in this case not alot you can do other than produce it.
Edit:
In the case of In-House - It works from the top-down then, the Owner -> MD -> Managers etc etc
However you would like to think that the top level managment would know that a designer is a designer and know how to design.
Edit 2: When I did work in such a place it was the job of the Analyst.
I would say that the final say on how a site should look (in terms of design, screen flow and usability) should come from the client.
However designers and developers should try to frame their ideas from a business sense. Remember that most (ok some) designers have spent a large amount of time at university studying elements of graphical design and know what colours work together and what gets people’s attention. Good developers will have some understanding on usability and we all know that usability is important. Whilst the customer should always have the last say we (as service providers) should always be willing to provide input (if warranted).
most programmers find it convenient to just do what their client or employer say. that's where code monkey term comes from I guess.
I could never work like that but I don't expect client to automatically trust me as well. Trust is something that has to be earned by showing confidence, diligence and good ethics. if client doesn't trust his supplier, relationship is doomed. that's why code monkeys can never deliver good results.
The end-user. If its client, then he should decide. Else if its internal website, then the actual user.
Also see the book by Joel about UI. I don't remember the name though.
Everyone has a client - even in-house.
The 'person' who gets the final say are the ones who will be eventually using it.

How do you perform address validation? [closed]

Closed. This question does not meet Stack Overflow guidelines. It is not currently accepting answers.
We don’t allow questions seeking recommendations for books, tools, software libraries, and more. You can edit the question so it can be answered with facts and citations.
Closed 2 years ago.
The community reviewed whether to reopen this question 11 months ago and left it closed:
Original close reason(s) were not resolved
Improve this question
Is it even possible to perform address (physical, not e-mail) validation? It seems like the sheer number of address formats, even in the US alone, would make this a fairly difficult task. On the other hand it seems like a task that would be necessary for several business requirements.
Here's a free and sort of "outside the box" way to do it. Not 100% perfect, but it should reject blatantly non-existent addresses.
Submit the entire address to Google's geocoding web service. This service attempts to return the exact coordinates of the location you feed it, i.e. latitude and longitude.
In my experience if the address is invalid you will get a result of 602 from the service. There's definitely a possibility of false positives or false negatives, but used in conjunction with other consistency checks it could be useful.
(Yahoo's geocoding web service, on the other hand, will return the coordinates of the center of the town if the town exists but the rest of the address is bogus. Potentially useful as long as you pay close attention to the "precision" field in the result).
There are a number of good answers in here but most of them make the assumption that the user wants an "API" solution where they must write code to connect to a 3rd-party service and/or screen scrape the USPS. This is all well and good, but should be factored into the business requirements and costs associated with the implementation and then weighed against the desired benefits.
Depending upon the business requirements and the way that the data is received into the system, a real-time address processing solution may be the best bet. If a real-time solution is required, you will want to consider the license agreement and technical limitations of the Google Maps/Bing/Yahoo APIs. They typically limit the number of calls you can make each day. The USPS web tools API is the same in additional they restrict how/why you can use their system and how you are allowed to use the data thereafter.
At the same time, there are a handful of great service providers that can easily process a static list of addresses. Essentially, you give the service provider a CSV file or Excel file, they clean it up and get it back to you. It's a one-time deal with no long-term commitment or obligation—usually.
Full disclosure: I'm the founder of SmartyStreets. We do address verification for addresses within the United States. We are easily able to CASS certify a list and we also offer a address verification web service API. We have no hidden fees, contracts, or anything. You use our service until you no longer need it and you can walk away. (Unlike cell phone companies that require a contract.)
USPS has an address cleaner online, which someone has screen scraped into a poor man's webservice. However, if you're doing this often enough, it'd be a better idea to apply for a USPS account and call their own webservice.
I will refer you to my blog post - A lesson in address storage, I go into some of the techniques and algorithms used in the process of address validation. My key thought is "Don't be lazy with address storage, it will cause you nothing but headaches in the future!"
Also, there is another StackOverflow question that asks this question entitled How should international geographic addresses be stored in a relational database.
In the course of developing an in-house address verification service at a German company I used to work for I've come across a number of ways to tackle this issue. I'll do my best to sum up my findings below:
Free, Open Source Software
Clearly, the first approach anyone would take is an open-source one (like openstreetmap.org), which is never a bad idea. But whether or not you can really put this to good and reliable use depends very much on how much you need to rely on the results.
Addresses are an incredibly variable thing. Verifying U.S. addresses is not an easy task, but bearable, but once you're going for Europe, especially the U.K. with their extensive Postal Code system, the open-source approach will simply lack data.
Web Services / APIs
Enterprise-Class Software
Money gets it done, obviously. But not every business or developer can spend ~$0.15 per address lookup (that's $150 for 1,000 API requests) - a very expensive business model the vast majority of address validation APIs have implemented.
What I ended up integrating: streetlayer API
Since I was not willing to take on the programmatic approach of verifying address data manually I finally came to the conclusion that I was in need of an API with a price tag that would not make my boss want to fire me and still deliver solid and reliable international verification results.
Long story short, I ended up integrating an API built by apilayer, called "streetlayer API". I was easily convinced by a simple JSON integration, surprisingly accurate validation results and their developer-friendly pricing. Also, 100 requests/month are entirely free.
Hope this helps!
I have used the services of http://www.melissadata.com Their "address object" works very well. Its pricey, yes. But when you consider costs of writing your own solutions, the cost of dirty data in your application, returned mailers - lost sales, and the like - the costs can be justified.
For us-based address data my company has used GeoStan. It has bindings for C and Java (and we created a Perl binding). Note that it is a commercial product and isn't cheap. It is quite fast though (~300 addresses per second) and offers features like CASS certification (USPS bulk mail discount), DPV (Delivery point verification) flagging, and LON/LAT geocoding.
There is a Perl module Geo::PostalAddress, but it uses heuristics and doesn't have the other features mentioned for GeoStan.
Edit: some have mentioned 'doing it yourself', if you do decide to do this, a good source of information to start with is the US Census Tiger Data Set, which contains a lot of information about the US including address information.
As seen on reddit:
$address = urlencode('1600 Pennsylvania Avenue, Washington, DC');
$json = json_decode(file_get_contents("http://where.yahooapis.com/geocode?q=$address&flags=J"));
print_r($json);
Fixaddress.com service is available that provides following services,
1) Address Validation.
2) Address Correction.
3) Address spell correcting.
4) Correct addresses phonetic mistakes.
Fixaddress.com uses USPS and Tiger data as reference data.
For more detail visit below link,
http://www.fixaddress.com/
One area where address lookups have to be performed reliably is for VOIP E911 services. I know companies reliably using the following services for this:
Bandwidth.com 9-1-1 Access API MSAG Address Validation
MSAG = Master Street Address Guide
https://www.bandwidth.com/9-1-1/
SmartyStreet US Street Address API
https://smartystreets.com/docs/cloud/us-street-api
There are companies that provide this service. Service bureaus that deal with mass mailing will scrub an entire mailing list to that it's in the proper format, which results in a discount on postage. The USPS sells databases of address information that can be used to develop custom solutions. They also have lists of approved vendors who provide this kind of software and service.
There are some (but not many) packages that have APIs for hooking address validation into your software.
However, you're right that its a pretty nasty problem.
http://www.usps.com/ncsc/ziplookup/vendorslicensees.htm
As mentioned there are many services out there, if you are looking to truly validate the entire address then I highly recommend going with a Web Service type service to ensure that changes can quickly be recognized by your application.
In addition to the services listed above, webservice.net has this US Address Validation service. http://www.webservicex.net/WCF/ServiceDetails.aspx?SID=24
We have had success with Perfect Address.
Their database has all the US street names and street number ranges. Also acts as a pretty decent parser for free-form address fields, if you are lucky enough to have that kind of data.
Validating it is a valid address is one thing.
But if you're trying to validate a given person lives at a given address, your only almost-guarantee would be a test mail to the address, and even that is not certain if the person is organised or knows somebody at that address.
Otherwise people could just specify an arbitrary random address which they know exists and it would mean nothing to you.
The best you can do for immediate results is request the user send a photographed / scanned copy of the head of their bank statement or some other proof-of-recent-residence, because at least then they have to work harder to forget it, and forging said things show up easily with a basic level of image forensic analysis.
There is no global solution. For any given country it is at best rather tricky.
In the UK, the PostOffice controlls postal addresses, and can provide (at a cost) address information for validation purposes.
Government agencies also keep an extensive list of addresses, and these are centrally collated in the NLPG (National Land and Property Gazetteer).
Actually validating against these lists is very difficult. Most people don't even know exactly how their address as it is held by the PostOffice. Some businesses don't even know what number they are on a particular street.
Your best bet is to approach a company that specialises in this kind of thing.
Yahoo has also a Placemaker API. It is good only for locations but it has an universal id for all world locations.
It look that there is no standard in ISO list.
You could also try SAP's Data Quality solutions which are available in both a server platform is processing a large number of requests or as an embeddable SDK if you wanted to run it in process with your application. We use it in our application and it's very robust and scalable.
NAICS.com is coming out with an API that will add all kinds of key business data including street address. This would happen on the fly as your site's forms are processed. https://www.naics.com/business-intelligence-api/
You can try Pitney Bowes “IdentifyAddress” Api available at - https://identify.pitneybowes.com/
The service analyses and compares the input addresses against the known address databases around the world to output a standardized detail. It corrects addresses, adds missing postal information and formats it using the format preferred by the applicable postal authority. I also uses additional address databases so it can provide enhanced detail, including address quality, type of address, transliteration (such as from Chinese Kanji to Latin characters) and whether an address is validated to the premise/house number, street, or city level of reference information.
You will find a lot of samples and sdk available on the site and i found it extremely easy to integrate.
For US addresses you can require a valid state, and verify that the zip is valid. You could even check that the zip code is in the right state, but beyond that I don't think there are many tests you could run that wouldn't provide a lot of false negatives.
What are you trying to do -- prevent simple mistakes or enforcing some kind of identity check?

Resources