Related
First, I want to apologize. I am complete noob in this area and many of my thoughts are probably misleading.
I need to verify that a user of my app is on a specific place in order to be authirized to perform an action. I want to use NFC for this purpose. The user have to put his smartphone by a NFC tag in order to be authorized to perform the action. Easy but I need it to be reasonably hackerproof. It means that the NFC tag must be impossible to clone without physical damage to the plastics around the NFC chip. It also means that the NFC chip must not contain only static data. The NFC chip must contain an app, that can receive some data (cryptographic challenge) and signs them using secure built-in private key (which must be unreadable through NFC interface). When the user wants to perform the action, he will ask server for the challenge, then he lets the chip to sign it, and then he sends the signed challenge back to the server which will verify the signature using known public key. This should be achievable using NFC JavaCard. But do these NFC JavaCards actually exist? I wasn't able to find a company which would be able to produce such NFC tags for me. When I try to explain my requirements to a NFC tags producer he looks like he has never heard of NFC JavaCards. I have tried about 10 producers without luck.
Can a commonly available chip meet my requirements? I mean a chip from the Mifare familly. I suspect that Mifare DESfire might be able to meet my requirements, but I am not sure.
Feel free to respond with an advertisement, because relevant advertisement is exactly what I look for :)
I try to collect some useful facts:
NFC is a very broad term, just finding that on both sides does not ensure interoperability.
Any ISO 14443 (one of the NFC flavours) compliant smart card with crypto functionality should be usable. Note, that a card with native OS may be a viable alternative to a JavaCard, since the functionality to sign a random number is pretty standard.
Any smart phone sporting a NFC chip can address such a card in principle. Unfortunately this is strongly dependent on the OS of the smart phone, for Android the relevant class to use is IsoDep, which gives you the APDU interface. After triggering the "card enters field" event, then the app receives a handle, via which further communication can take place.
Real smart cards can't be cloned, since you are not able to dump them; especially keys can't be read.
Now some things to consider:
Your approach looks unusual, which might become a problem. (To have a portable card somehow fixed to a wall, just to get the location; so you know where somebody is, but not who? While I don't consider cloning to be an issue, you somehow must ensure destruction in case of a theft attempt, which may collide with the distance topic below.)
I don't see, where the server comes into play. If not involved in the authorized action, provision of a random number is not sufficient reason.
Asymmetric key operations have a comparatively high power consumption, and this power has to be supplied via the electric field. This severely limits the distance between card and phone and may even require direct touch. While a power supply of its own would solve the issue in principle, it is not what ISO-14443 was designed for.
Yes JavaCards do exist.
https://github.com/OpenJavaCard/openjavacard-ndef is a project makes these JavaCards to output standard NDEF messages (thought note issue 4 in that there example uses the wrong APDU but that is easily changed)
This project also give a number of cards it is fully working and tested for
ACS ACOSJ - fully working
NXP JCOP J3D040/J3D081/J2E145 etc - fully working
Both ACS and Cardlogic do cards (just google the model numbers)
e.g.
https://www.acs.com.hk/en/products/405/acosj-java-card-combi/
https://www.smartcardfocus.com/shop/ilp/id~707/j3a081-80k/p/index.shtml
The answer a was looking for is not a chip which runs a custom code. Although this might be possible it is definitely not the best way to achieve the target.
I was looking for a solution that enables strong authentication using NFC data. There might be multiple chips that offers this, but probably the most available chip is NTAG 424 DNA TT. It works like this:
The chip has a memory, which is not readable through NFC. Private key is stored there.
The chip has a read counter. It increments everytime the data are read through NFC.
The chip can generate an AES-128 signature of string UID (chip serial number) + counter using the private key in the inaccessible part of the memory.
The chip can dynamicaly inject the data above into a URL that is stored in the readable memory.
So the solution will be like (I am waiting for delivery of NFC tags right now, so I don't know for sure yet):
Read the tag UID (serial number) and the actual counter value (should be 0 on an unused tag)
Generate the key-pair
Load private key to the chip
Load some data (URL, eg: https://my.app/) to the chip
Store UID, public-key, last-counter on the server
Configure the chip to inject UID, counter, signature to the URL stored on the chip
When a client reads the data, they should contain required variables, eg: https://my.app/?counter=1&uid=ff:ff:ff:ff&signature=xyz. Then on the server:
Fetch stored info (public-key, last-counter) using uid as a primary key
verifies the signature
verifies the counter that must be > last-counter
stores counter as the last-counter
successfully authorized
Is anyone able to hack this without reading the hidden memory of the chip which would require physical tampering with the chip?
I'm writing an application to protect passwords from key sniffers and screen retrievers. I have the user type in an easy-to-remember keyword or phrase (i.e, "password123", "amazon.com", "gmail") and I use that string to create a longer and stronger password which is loaded into the clipboard. I want the application to be completely anonymous, so I don't save any information. To generate the passwords, I use a random number generator. I need a way for the user to carry around their seed that isn't vulnerable to key sniffers or screen retrievers. I'm thinking a hardware token like a YubiKey, but I would like something more easier and more mainstream. I tried using behavioral biometrics, but I managed to replicate them with a program too easily. Any better ideas?
What you are suggesting is a more than vulnerable approach.
First of all, there are open source and proven-correct algorithms and applications for the problem you are targeting. In security questions it is never a good idea to go and develop applications for critical operations (and handling passwords is always a critical operation) on your own, especially reinventing the wheel is almost in every case an endeavor doomed to fail.
Your approach is problematic in several points:
To be anonymous the app needs to copy/paste or in-place-generate the password needed for some action. You will have a hard time avoiding screen retrievers capture that if you do not do some magic on OS level.
Using one(!) random seed to protect several passwords makes each of them weaker than it was before.
Carrying this random seed on a usb key and freely plugging it into all kinds of computers that you cannot control is a problem as each of them may be potentially malicious. The random seed could be silently retrieved, altered or deleted.
To give you some things to get paranoid about, google e.g. blue pill and you will see that the real problems dwell on another machine layer than the application you are talking about.
Instead have a look at the following approaches:
2 factor authentication (2FA) against malicious software and hardware stealing your passwords on type-in. See e.g. Google Authenticator.
Secure operating systems against such software entering your system and retrieving your passwords. See e.g. QubesOS
Read-only drives with secure / anonymous OS for usage on foreign and potentially dangerous machines even for very critical tasks such as banking. See e.g. Tails OS on a dvd (not a usb key!)
Virtual machines to capsule potentially malicious tasks. See e.g. VirtualBox
Trustable password safes like KeyPassX
In a nutshell: You can write such an application but it will most likely not be practical nor secure nor by so usable. Sorry about that.
As read on topic here How to find the unique serial number of a flash device? and especially here How to get manufacturer serial number of an USB flash drive? I know it is possible to get properties of hardware devices (particularly hard drives and usb drives...) using WMI Win32_PhysicalMedia and Win32_DiskDrive, which I'm getting done successfully.
However, I really want to know about the safety of these informations.
PhysicalMedia property SerialNumber returns the actual serial number of the main hard drive, while using other Win32_LogicalDisk and other calls we can map the drive letter of flash storage to actual Win32_DiskDrive device, and from there read properties like Name, Model, FirmwareRevision, SerialNumber, DeviceID, Manufacturer...
Now, DeviceID is generated by Windows / Pc itself, while SerialNumber should be the one that manufacturer added to the physical flash drive.
Manufacturer in most cases returns "Standard" something, Name is also of no use, while SerialNumber actually gets me a something that looks like unique ID, (I've read that in some cases this is not returned, so PNPDeviceID should be used instead? , Model gives the actual model of the flash drive, and FirmwareRevision just a number that could be used to add safety switch to the licensing, but is not vital.
However, the only one of these that seems / should be actually safe to use is SerialNumber, right?
So, the question here goes: Which level is Win32_DiskDrive actually reading this info from? Is it possible to fake that at all (Ok, letalone the actual lowlevel hacking stuff or driver injection etc...(??)), and if so, how hard it is?
If there's a known way / guide / example, I'd be also happy to read it. (not necessary info looking for here though.)
This is not for intention of bypassing some licensing. I'm making licensing for my SW, and am curious, whether it would be safe enough to use USB drive's SerialNumber property, and lock license against the presence of that USB flash, for which the license was bought for? Basically to use it as kind of a dongle, but not like the dongles actually work (using communication with the actual hardware inside the dongle...)
I know it may not seem as a safe solution, as flash drives dies quite often these days, or get lost etc, but this is just to add an option to my licensing from "Per PC" to "Portable - per USB device".
Thanks for any info!!!
EDIT:
I am completely aware that bypassing these kind of safety switches is very possible. Of course, even Windows itself is not licensed in a way that couldn't be hacked, nor Adobe, ProTools etc, (software that is widely used and costs a lot!).
But that wasn't a real question, and also, that's not the case for me -> the software will not be that expensive and not used by that much people, that I'd be afraid to drag interest in someone who will do extensive programming to make a patch/crack for it. Regular debugger use and workaround is pretty unlikely to be used by regular client who would need the software, ( and also, since it is something to be used in business environment, where stability is vital, I doubt they will really play around that...).
Main point here:
It is possible for sure, but: HOW hard is it to do for a regular person? (I know, the answer is: depending on your code.)
Main question of the post: Is it possible to change the ID on the USB itself, OR to make an app that will fake that data to my app? If it is, I'm sure it might be easier than making a crack/patch, that's why I wanted to know, whether WMI reads explicitly from hardware, or could one make an app that would pass fake data to it?
WMI just returns what the hardware tells it. It's as unique as the hardware. Which ultimately depends on the vendor.
But...
If someone has an administrator account to the computer†, then there are very few things that can be done to keep them from just hooking up the kernel debugger to your program and overriding your checks, or recording the raw USB communication session and replaying it on an unauthorized system. The real dongles do some to mitigate this, by having the hardware generate a response to a particular challenge. The challenge/response changes for each request, so it's not as susceptible to replay attacks, but the debugger tricks still work.
This is the real problem with the serial number approach. Uniqueness is not the primary concern for dongled software. The primary concern is unpredictability.
An illustrative example-
Let's say that I'm a bouncer at an exclusive night club. We're so exclusive that you have to answer a question to get in. You really want to get in, but no one will tell you the answer to the question. One night, you hatch a plan. You hang out in the alley and listen to the conversations that I'm having with the patrons trying to enter the club. It doesn't take you long to realize that I'm asking everyone the exact same question, and you're in. (This is the serial number approach)
After a while, I notice that there are a lot of people coming into the club that I've never seen before, and begin to suspect something. The people we really want to allow in are all given a card with a formula‡ on it. Whenever they come to the door of the club, I give them a number and they apply their formula and tell me the result. Since I also know the formula, I can tell if they are really allowed in. Now, even if you hear the entire challenge and response, without the formula, you aren't getting in. (This is one common approach taken by dongles.)
But what about the debugger? The debugger just made herself the club's owner, fired me, and can come and go as she pleases.
†Or has physical access to the machine and a password reset disk.
‡Stop laughing, this could totally happen. :)
Photo credit: Guillaume Paumier, CC-BY. Found on the Wikimedia Commons 7-Oct-15
Edit to address the question edit:
HOW hard is it to do for a regular person? (I know, the answer is: depending on your code.)
The question is how skilled is the 'regular person'? If you're talking about software/electrical engineers, then this is a trivial task. If you're talking about sales/marketing then it's a challenging task.
Is it possible to change the ID on the USB itself, OR to make an app that will fake that data to my app?
It depends and Yes. Changing the ID on the device itself is possible with some devices, and impossible with others. Software to spoof/man-in-the-middle the USB communication, or to create a virtual USB device is possible.
If it is, I'm sure it might be easier than making a crack/patch, that's why I wanted to know, whether WMI reads explicitly from hardware, or could one make an app that would pass fake data to it?
As I led with above, WMI reads from the hardware. This can be intercepted or bypassed.
Some ways to bypass the check:
Make a virtual USB device
Modify the USB MSD device driver to report the same serial number for all devices.
Build hardware using commercially available cheap host controllers that identifies with the same information as the authorized device. ($10 worth of raw components and a little bit of time.)
Redirect the system calls to/from USB to a compromised library.
Note also that:
Some places have restrictions on USB storage devices, ranging from discouraging their use, to outright bans. This would prevent your software from being used in sensitive computing environments processing private data, like credit cards, PII, trade secrets, classified information, etc. (In the US many governmental agencies have outright bans on USB storage devices, and block the install of any MSD.)
The Mass Storage specification doesn't require serial numbers. They are usually there, but they don't have to be, and many low-cost vendors
A USB PKI token costs a little bit more, but would probably do what you want. Here's an example from Safenet (Disclaimer: I am in no way affiliated with Safenet Inc, and you should evaluate all the possible options from all vendors. I suggested this because it was the first thing that came up through CDW, and the price was ~$30)
Systems have to sometimes accommodate the possibility of real world bad data. Consider that some data originates with paper forms. And forms inherently have a limited means of validating data.
Example 1: On one form users are expected to enter an integer distance (in miles) into a blank. We capture the information as written as a string since we don't always end up getting integer values.
Example 2: On another form we capture a code. That code should map to one of the codes in our system. However, sometimes the code written on the form is incorrect. We capture the code and allow it to exist with an invalid value until some future time of resolution. That is, we temporarily allow bad data since it's important to record the record even if some of it is invalid.
I'm interested in learning more about how systems accommodate bad data, that is, human error. Databases are supposed to be bastions of data integrity, but the real world is messy and people make mistakes. Systems must allow us to reflect those mistakes.
What are some ways systems you've developed accommodate human error? What practices have you used? What lessons have you learned?
Any further reading on the topic? (I had trouble Googling it.)
I agree with you, whatever we do there's no guarantee that we can get rid of bad or incorrect data. Especially, but not only, if it comes to user input. In my experience the same problems exist in complex integration projects, in which you have to integrate and merge (often inconsistent) data retrieved from different systems.
A good strategy is to decouple the input from the operational system itself. First, place user (or external system) provided data in a separate datastore (e.g. different schema). In a second step load this data into your operational datastore, but only if it confirms to strict rules (e.g. use address verification software to verify a given address). This Extract, Transform, Load (ETL) approach is fairly common in Data Warehousing (DWH) solutions, but can be applied programmatically in transactional systems as well (in my experience).
The above approach often leads to asynchronous processes in which the input is subitted first and (maybe) at a later time the external entity (user or system) retrives feedback whether its data was correct or not.
EDIT: For further readings I recommend to have a look at DWH concepts. Alhtough, you may not want to build such a thing, you could partially apply those concepts:
http://en.wikipedia.org/wiki/Extract,_transform,_load
http://en.wikipedia.org/wiki/Data_warehouse
http://en.wikipedia.org/wiki/Data_cleansing
A government department I worked in does a lot of surveys, most of which are (were) still paper based.
All the results were OCR'd into the system.
As part of the OCR process a digital scan of the forms is kept.
Data is then validated, data that is undecipherable or which fails validation is flagged.
When a human operator reviews the digital data they can modify the data if they are confident that they can correctly interpret what the code could not; they (here's the cool bit) can also bring up the scan of the paper based original, and use that to determine what the user was trying to say.
On a different thread; at some point you want to validate the data coming in against any expected data ranges that you want it to conform to; buy rejecting it at the point of entry you give the user a chance to correct it - the trade off is that every time you reject it you increase the chance of them abandoning the whole process.
At some point in your system you need to specify the rules which will be used for validation. At the end of the day a system is only going to be as smart as those rules. You can develop these yourself into the code (probably the business logic) or you might use a 3rd party component.
having flexible control over the validation is pretty important as they are likely to change overtime.
To be honest with you, one point of migrating from paper-based systems to IT is to remove these errors and make sure all data is always correct. I doubt any correctly planned and developed IT system (especially business financial systems) would allow such errors. Not in the company I am working for anyway...
There are lots of software tools that address the kinds of problems you mention. There are platforms and tools that let you define rules for scrubbing and transforming data and handling validation errors. Those techniques are widely used for Data Integration and Business Intelligence applications. Google for "Data Quality" or "Data Integration".
The easiest thing to do is to (this is not always possible) design the interface where users enter the data to limit as much as possible the amount of text that they need to enter. In my experience this seems to be where a lot of problems come from. One simple example of this is to provide a select, or auto-complete select field
One thing that you could do is do everything possible to determine if the data is correct before going into the db. I try to give the user entering the data as much feedback as possible so they can (ideally) fix some of the issues before the data gets persisted. For example, it is a very quick check to determine if the data being entered is of the correct type.
I got started in legal systems before the PC era. Litigation support databases routinely have to accommodate factually incorrect, incomplete, and contradictory information. It takes a different way of thinking.
The short version . . .
Instead of recording a single fact, you record multiple assertions about a fact. It boils down to designing a database to store data from assertions like these.
In an interview at 2011-01-03 08:13, Neil Rimes told Officer Cane
that he was at home from 2011-01-02 20:00 until 2011-01-03 08:13.
In an interview at 2011-01-03 08:25, Liza Nevers told Officer Cane
that Neil Rimes came home at 2011-01-02 23:45.
In a deposition at 2011-05-13 10:22, Cody Maxon told attorney Kurt
Schlagel that he saw Neil Rimes at Kroger at 2011-01-03 03:00
I was wondering if anyone had come across any techniques to reduce the chances of data exposed through JSON type services on the server (intended to supply AJAX functions) from being harvested by external agents.
It seems to me that the problem is not so difficult if you had say a Flash client consuming the data. Then you could send encrypted data to the client, which would know how to decrypt it. The same method seems impossible with AJAX though, due to the open nature of the Javascript source.
Has anybody implemented a clever technique here?
Whatever the method, it should still allow a genuine AJAX function to consume the data.
Note that I'm not really talking about protecting 'sensitive' information here, the odd record leaking out is not a problem. Rather I am thinking about stopping a situation where the whole DB is hoovered up by bots (either in one go, or gradually over time).
Thanks.
First, I would like to clear on this:
It seems to me that the problem is not
so difficult if you had say a Flash
client consuming the data. Then you
could send encrypted data to the
client, which would know how to
decrypt it. The same method seems
impossible with AJAX though, due to
the open nature of the Javascrip
source.
It will be pretty obvious the information is being sent encrypted to the flash client & it won't be that hard for the attacker to find out from your flash compiled program what's being used for this - replicate & get all that data.
If the data does happens to have the value you are thinking, you can count on the above.
If this is public information, embrace that & don't combat it - instead find ways to capitalize on it.
If this is information that you are only exposing to a set of users, make sure you have the corresponding authentication / secure communication. Track usage as others have said, and have measures that act on it,
The first thing to prevent bots from stealing your data is not technological, it's legal. First, make sure you have the right language in your site's Terms of Use that what you're trying to prevent is actually disallowed and defensible from a legal standpoint. Second, make sure you design your technical strategy with legal issues in mind. For example, in the US, if you put data behind an authentication barrier and an attacker steals it, it's likely a violation of the DMCA law. Third, find a lawyer who can advise you on IP and DMCA issues... nice folks on StackOverflow aren't enough. :-)
Now, about the technology:
A reasonable solution is to require that users be authenticated before they can get access to your sensitive Ajax calls. This allows you to simply monitor per-user usage of your Ajax calls and (manually or automatically) cancel the account of any user who makes too many requests in a particular time period. (or too many total requests, if you're trying to defend against a trickle approach).
This approach of course is vulnerable to sophisticated bots who automatically sign up new "users", but with a reasonably good CAPTCHA implementation, it's quite hard to build this kind of bot. (see "circumvention" section at http://en.wikipedia.org/wiki/CAPTCHA)
If you are trying to protect public data (no authentication) then your options are much more limited. As other answers noted, you can try IP-address-based limits (and run afoul of large corporate proxy users) but sophisticated attackers can get around this by distributing the load. There's also likley sophisticated software which watches things like request timing, request patterns, etc. and tries to spot bots. Poker sites, for example, spend a lot of time on this. But don't expect these kinds of systems to be cheap. One easy thing you can do is to mine your web logs (e.g. using Splunk) and find the top N IP addresses hitting your site, and then do a reverse-IP lookup on them. Some will be legitimate corporate or ISP proxies. But if you recognize a compeitor's domain name among the list, you can block their domain or follow up with your lawyers.
In addition to pre-theft defense, you might also want to think about inserting a "honey pot": deliberately fake information that you can track later. This is how, for example, maps manufacturers catch plaigarism: they insert a fake street in their maps and see which other maps show the same fake street. While this doesn't prevent determined folks from sucking out all your data, it does let you find out later who's re-using your data. This can be done by embedding unique text strings in your text output, and then searching for those strings on Google later (assuming your data is re-usable on another public website). If your data is HTML or images, you can include an image which points back to your site, and you can track who is downloading it, and look for patterns you can use to bust the freeloaders.
Note that the javascript encryption approach noted in one of the other answers won't work for non-authenticated sessions-- an attacker can simply download the javascript and run it just like a regular browser would. Moral of the story: public data is essentially indefensible. If you want to keep data protected, put it behind an authentication barrier.
This is obvious, but if your data is publicly searchable by search engines, you'll both need a non-AJAX solution for them (Google won't read your ajax data!) and you'll want to mark those pages NOARCHIVE so your data doesn't show up in Google's cache. You'll also probably want a white list of search engine crawler IP addreses which you allow into your search-engine-crawlable pages (you can work with Google, Bing, Yahoo, etc. to get these), otherwise malicious bots could simply impersonate Google and get your data.
In conclusion, I want to echo #kdgregory above: make sure that the threat is real enough that it's worth the effort required. Many companies overestimate the interest that other people (both legitimate customers and nefarious actors) have in their business. It might be that yours is an oddball case where you have particularly important data, it's particularly valuable to obtain, it must be publicly accessible without authentication, and your legal recourses will be limited if someone steals your data. But all those together is admittedly an unusual case.
P.S. - another way to think about this problem which may or may not apply in your case. Sometimes it's easier to change how your data works which obviates securing it. For example, can you tie your data in some way to a service on your site so that the data isn't very useful unless it's being used in conjunction with your code. Or can you embed advertising in it, so that wherever it's shown you get paid? And so on. I don't know if any of these mitigations apply to your case, but many businesses have found ways to give stuff away for free on the Internet (and encourage rather than prevent wide re-distribution) and still make money, so a hybrid free/pay strategy may (or may not) be possible in your case.
If you have an internal Memcached box, you could consider using a technique where you create an entry for each IP that hits your server with an hour expiration. Then increment that value each time the IP hits your AJAX endpoint. If the value gets over a particular threshold, fry the connection. If the value expires in Memcached, you know it isn't getting "hoovered away".
This isn't a concrete answer with a proof of concept, but maybe a starting point for you. You could create a javascript function that provides encryption/decryption functions. The javascript would need to be built dynamically, and you would include an encryption key that is unique to the session. On the server side, you'd have an encryption service that uses the key from the session to encrypt your JSON before delivering it.
This would at least prevent someone from listening to your web traffic, pulling information out of your database.
I'm with kdgergory though, it sounds like your data is too open.
Some techniques are listed in Further thoughts on hindering screen scraping.
If you use PHP, Bad behavior is a nice tool to help. If you don't use PHP, it can give some ideas on how to filter (see How it works page).
Incredibill's blog is giving nice tips, lists of User-agents/IP ranges to block, etc...
Here are a variety of suggestions:
Issue tokens required for redemption along with each AJAX request. Expire the tokens.
Track how many queries are coming from each client, and throttle excessive usage based on expected normal usage of your site.
Look for patterns in usage such as sequential queries, spikes in requests, or queries that occur faster than a human could conduct.
Check user-agents. Many bots don't completely replicate the user agent info of a browser, and you can eliminate programatic scraping of your data using this method.
Change the front-end component of your website to redirect to a captcha (or some other human verifying mechanism) once a request threshold is exceeded.
Modify your logic so the respsonse data is returned in a few different ways to complicate the code required to parse.
Obsfucate your client-side javascript.
Block IPs of offending clients.
Bots usually doesn't parse Javascript, so your ajax code won't be instantly executed. And if they even do, bots usually doesn't maintain sessions/cookies as well. Knowing that, you could reject the request if it is invoked without a valid session/cookie (which is obviously set on the server side beforehand by the request on the parent page).
This does not protect you from human hazard though. The safest way is to restrict access to users with a login/password. If that is not your intent, well, then you have to live with the fact that it's a public application. You could of course scan logs and maintian blacklists with IP addresses and useragents, but that goes extreme.