I want to find if two images are the picture of the same art work (below are two pics of a work from japanese artist Yoshitomo Nara)
I aligned the two pics using feature match and affine transformation. So the images are now of identical size and supposedly well aligned.
but in some cases (e.g. in this one), the two pictures have different brightnesses.
directly comparing greyscale images doesn't work.
I tried equal hist, which makes the situation even worse
I also tried some other methods, including:
number of matches in feature match;
ssim https://en.wikipedia.org/wiki/Structural_similarity
phash https://www.phash.org/
none of them work very well.
My guess is I need to first get rid of the brightness difference. But I'm not sure how to do it. Any ideas? Thanks
Related
I want to identify all occurrences of a list of images in a screenshot and get the label of the emoji (e.g. "smile").
The list of images holds all emojis (full list):
and so on…
And this is the screenshot (show large):
The screenshots can have different resolutions and different heights where the emoji occur.
My ideas were:
Using OpenCV and a variety of filtering and iterate all emoji images over it (maybe using template matching)
Using neural networks like tensorflow, training your own model with the emojis
How would you do it ?
There are several classic ways to answer your problem:
Simple regular correlation: https://en.wikipedia.org/wiki/Cross-correlation.
The simple correlation is used when you have exactly the image you are looking for, with no change in intensity.
Normalized correlation (math behind template matching): https://en.wikipedia.org/wiki/Template_matching.
The simple correlation is used when you have exactly the image you are looking for, with no change in intensity.
If you have different intensities between your screenshot and your emoji base picture, you should use normalized correlation.
Both these methods will give you an image with peaks, and your emojis will be localized at the local maxima of this image.
As your emojis can be very similar to one another, you will have to use a threshold on the correlation image in order to discriminate between the emoji you are testing and the ones that look nearly like him.
This method can be time consuming, but can be easily speed-up by using an image pyramid.
An image pyramid is a set of image where the first one is your image, the second one is a subsampling of the first by a factor of 2, and so on:
https://en.wikipedia.org/wiki/Pyramid_(image_processing).
Then the correlation is applyed on the top level of the pyramid to find an approximate location, then on the top - 1 level around the approxiamte location and so on.
About the neural network, or other machine learning methods you want to try, they are really heavy solutions and you have a pretty simple problem, so you should normaly don't need them.
You have the exact picture you are looking for, without rotation, deformation or intensity change, and the template matching should be very effective.
I remember a story about someone filtering images with a spam filter which he fed with some training data.
I come to the point where I exactly need something like this.
I have a lot different types of images (mainly people, e.g. selfies, group pictures, portraits, ..) but I only want a certain type (e.g. only male) of them.
With the right algorithm and training data I think it's possible to get it to the point where I can pass an image to it and i get true or false whether it matches my type or not.
I had a look at a few Face/Gender Detection APIs, but none of them worked for me that's why I want to try the approach with the spam-filter - seems like a funny idea.
Here's what I need:
a trainable spam-filter algorithm/code sample/API
has to work offline
preferably for C# or Java
I already spent a few hours trying different things and googling, now I'm here and I'd like to get your opinion on this problem and the solution you think is appropriate.
Buddha
There is a simple image comparison algorithm that you can read about here: compareImages php class.
Basically the way it works is this:
it takes an image (a cropped image would be best), scales it down to a 8x8 pixels image, converts it to a BW / Greyscale image, and then it calculates the mean value of the pixels (which is the average value).
Then it goes over all the pixels of the scaled image (64 pixels), and in every pixel where the pixel's value >= the mean value, it puts "1", and if the pixel's value < the mean value, it puts "0", resulting in a 64bit "signature" value of 0s and 1s.
This signature value is what identifies the image, and then you can save this signature value in some kind of a database, as your "learned" filter.
Then if an email arrives with some images.. you can just crop them, and scan them, produce a signature, and see if it matches any known signature in your database.
The good things about this algorithm are:
It is very fast and scalable (scaling an image down to 8x8 is fast, and scanning the pixels as described is fast too).
Because it converts the image to greyscale & resizes it down, it means it can detect any color variations or sizes of the same image.
Because you use 64bit signatures, it doesn't take alot of space in your database as well.
Hope this helps.
I've checked methods like Phasher to get similar images. Basically to resize images to 8x8, grayscale, get average pixel and create a binary hash of each pixel comparing if it's above or below the average pixel.
This method is very well explained here:
http://hackerfactor.com/blog/index.php?/archives/432-Looks-Like-It.html
Example working:
- image 1 of a computer on a table
- image 2, the same, but with a coin
This would work, since, using the hash of a very reduced, grayscale image, both of them will be almost the same, or even the same. So you can conclude they are similar when 90% of more of the pixels are the same (in the same place!)
My problem is in images that are taken from the same point of view but different angle, for example this ones:
In this case, the hashes "fingerprint" generated are so shifted each other, that we can not compare the hashes bit by bit, it will be very different.
The pixels are "similar", but they are not in the same place, since in this case there's more sky, and the houses "starts" more below than the first one.
So the hash comparison results in "they are different images".
Possible solution:
I was thinking about creating a larger hash for the first image, then get 10 random "sub hashes" for the second one, and try to see if the 10 sub hashes are or are not in "some place" of the first big hash (if a substring is contained into another bigger).
Problem here I think is the CPU/time when working with thousands of images, since you have to compare 1 image to 1000, and in each one, compare 10 sub hashes with a big one.
Other solutions ? ;-)
One option is to detect a set of "interesting" points for each image and store that alongside your hash. It's somewhat similar to the solution you suggested.
We want those points be unlikely to vary between images like yours that have shifts in perspective. These lecture slides give a good overview of how to find points like that with fairly straightforward linear algebra. I'm using Mathematica because it has built in functions for a lot of this stuff. ImageKeypoints does what we want here.
After we have our interesting points we need to find which ones match between the images we're comparing. If your images are very similar, like the ones in your examples, you could probably just take an 8x8 greyscale image for each interesting point and compare each from one image with the ones for the nearby interesting points on the other image. I think you could use your existing algorithm.
If you wanted to use a more advanced algorithm like SIFT you'd need to have a look at ImageKeypoint's properties like scale and orientation.
The ImageKeypoints documentation has this example you can use to get a small piece of the image for each interesting point (it uses the scale property instead of a fixed size):
MapThread[ImageTrim[img, {#1}, 2.5 #2] &,
Transpose#
ImageKeypoints[img, {"Position", "Scale"},
"KeypointStrength" -> .001]]
Finding a certain number of matching points might be enough to say that the images are similar, but if not you can use something like RANSAC to figure out the transformation you need to align your hash images (the 8x8 images you're already able to generate) enough that your existing algorithm works.
I should point out that Mathematica has ImageCorrespondingPoints, which does all of this stuff (using ImageKeypoints) much better. But I don't know how you could have it cache the intermediate results so that scales for what you're trying to do. You might want to look into its ability to constrain matching points to a perspective transform, though.
Here's a plot of the matching points for your example images to give you an idea of what parts end up matching:
So you can precalculate the interesting points for your database of images, and the greyscale hashes for each point. You'll have to compare several hash images for each image in your database, rather than just two, but it will scale to within a constant factor of your current algorithm.
You can try an upper bound if the hashes doesn't match compare how many pixels match from the 8x8 grid. Maybe you can try to match the colors like in photo mosaic:Photo Mosaic Algorithm. How to create a mosaic photo given the basic image and a list of tiles?.
Here's my problem: A folder of 20,000 images- many of them are photographs but lots of them are basically scans of book text. I want to delete those.
Sorting by size helps a little, but since the images are of different resolutions some of the text ones are in fact bigger than the photo ones.
I'm thinking if I can find a program that can sort by color depth that might work? However a simply 'Number of colors' sort might not work because many of the photos i want to keep are drawings/black and white
Any ideas?
I have access to windows, mac, linux
You could use a OCR Library for a Language of your choice. And if you get a String with more than 50 chars it's a textscan.
That's how I would try it.
Edit: 50 is just an example. You need to test what fits the best
Have you try to do something like Histogram analyse? Maybe it is not possible because of your black and white images, but they should have more different gray scales then that one with text. You could check which range of gray colors the images cover.
I am wondering if there is a pre-existing algorithm/library/framework to compare two images to see if one is a re-sized version of the other? The programming language doesn't matter at this stage.
If there is nothing out there, I'd need to write something up. What I have thought of so far:
(Expensive) Resize the larger to the smaller and compare pixel by pixel.
Better yet, just resize a few random "areas" on the picture and compare. If they match, convert more, etc...
Break the image into a number of rows and columns and do some sort of parity math on the color values.
The problem I see with the first two ideas especially, is that there are different ways to re-size a picture in the first place, so the math will likely not work out the same at all. Some re-sizing adds blur, etc....
If anyone could point me to some good literature on this subject, that would be great. My googling turns up mostly shareware applications which is not what I want.
The goal is to have this running in the back of a webserver.
The best approach depends on the characteristics of the images you are comparing, what percentage of probability it is that the images are the same, and when they are different, are they typically off by a lot or could it be as minute as a single pixel difference?
If the answers to the above is that the images you need to compare will be completely random then going with the expensive solution, or some available package might be the best bet.
If it is that you know that the images are different more often than not, and that the images typically differ quite a lot, and you really want to hand-roll a solution you could implement some initial 'quick compare' steps that would be less expensive and that would quickly identify a lot of the cases where the images are different.
For example you could resize the larger image, then either compare pixel-by-pixel (or calculate a hash of the pixel values) only a 'diagonal line' of the image (top left pixel to bottom right pixel) and by doing so exclude differing images and only do the more expensive comparison for those that pass this test.
Or take a pre-set number of points at whatever is a 'good distribution' depending on the type of image and only do the more expensive comparison for those that pass this test.
If you know a lot about the images you will be comparing, they have known characteristics and they are different more often than they are the same, implementing a cheap 'quick elimination compare' along the lines of the above could be worthwhile.
You need to look into dHash algorithm for this.
I wrote a pure java library just for this few days back. You can feed it with directory path(includes sub-directory), and it will list the duplicate images in list with absolute path which you want to delete. Alternatively, you can use it to find all unique images in a directory too.
It used awt api internally, so can't be used for Android though. Since, imageIO has problem reading alot of new types of images, i am using twelve monkeys jar which is internally used.
https://github.com/srch07/Duplicate-Image-Finder-API
Jar with dependencies bundled internally can be downloaded from, https://github.com/srch07/Duplicate-Image-Finder-API/blob/master/archives/duplicate_image_finder_1.0.jar
The api can find duplicates among images of different sizes too.