[cross-posted from stats and datascience]
I'm working on an animal classification problem, with the data extracted from a video feed. The recording was made in a pen, so the problem is quite challenging with a dark background and many shadows:
Initially I tried scikit-image, but then someone helped me with an advanced tool called crf-rnn (http://crfasrnn.torr.vision/) that does a great job segmenting and labelling objects in an image. I did the following:
import caffe
net = caffe.Segmenter(MODEL_FILE, PRETRAINED)
IMAGE_FILE = '0045_crop2.png'
input_image = caffe.io.load_image(IMAGE_FILE)
from PIL import Image as PILImage
image = PILImage.fromarray(np.uint8(input_image))
image = np.array(image)
mean_vec = [np.mean(image[:,:,vals]) for vals in range(image.shape[2])]
im = image[:, :, ::-1]
im = im - reshaped_mean_vec
cur_h, cur_w, cur_c = im.shape
pad_h = 750 - cur_h
pad_w = 750 - cur_w
print(pad_h, pad_w, "999")
im = np.pad(im, pad_width=((0, max(pad_h,0)), (0, max(pad_w,0)), (0, 0)), mode = 'constant', constant_values = 255)
segmentation = net.predict([im])
segmentation2 = segmentation[0:cur_h, 0:cur_w]
The resulting image segmentation is rather poor (although two cows are recognized correctly):
I use a trained crf-rnn (MODEL_FILE, PRETRAINED), which works well for other problems, but this one is harder. I would appreciate any suggestions on how to pre-process this sort of image to extract the shape of most cows.
Related
I was trying to find snr for a set of images that I have but my two methodologies of doing so creates two different answers and I'm not sure which is right. I was wondering if one of them is just straight up the wrong way of doing this or if neither way is correct?
I am trying to characterize the snr of a set of images that I'm processing. I have 1 set of data with images and darkfields. From these pieces of data I subtracted the darkfield from the image and got "corrected_images".
So since I know snr is (mean of signal)/(std of noise), in my first methodology I was working with the corrected image and background noise image and I just took the mean of every pixel on the spectrum (from the corrected image) with a value greater than 1 for signal and the general std for the background noise image as my values. the plot for this methodology is in blue.
In my second methodology I used a single uncorrected image and basically considered every pixel above 50 as signal and every pixel below 50 as noise.This gives us the orange values for snr.
# -*- coding: utf-8 -*-
"""
Spyder Editor
This is a temporary script file.
"""
from PIL import Image
from matplotlib import pyplot as plt
import numpy as np
import os
signals=[]
name = r"interpolating_streaks/corrected"
name2 = r"interpolating_streaks/averages"
file = os.listdir(name)
file2 = os.listdir(name2)
wv1=[]
signal = []
snr = []
noise = []
x=0
for i in file:
wv=(i[:3])
wv1.append(wv)
corrected_image = Image.open(name+"/"+i) #opens the image
streak= np.array(corrected_image)
dark_image = Image.open(name2+'/d'+wv+'_averaged.tif')
dark = np.array(dark_image)
darkavg = dark[:][:].mean(axis=0)
avg= streak[:][:].mean(axis=0)
for i in avg:
if i >= 1:
signal.append(i)
noiser = np.std(darkavg)
signalr = np.mean(signal)
snr.append(signalr/noiser)
plt.plot(wv1,snr)
signal = []
noise = []
snr = []
for i in file2:
if(i[0] !='d'):
image = Image.open(name2+'/' + i )
im = np.array(image)
im_avg = im[:][:].mean(axis=0)
for i in im_avg:
if i <= 50:
noise.append(i)
else:
signal.append(i)
snr.append(np.mean(signal)/np.std(noise))
plt.plot(wv1,snr)
I would expect the snr values to be the same , and I know for my camera the snr has to be below 45 dB (but also I'm pretty sure this methodology for snr doesnt output decibels)
here are my current results
![1]: https://imgur.com/a/Vgecyp1
I'm trying to build a simple image classifier using scikit-learn. I'm hoping to avoid having to resize and convert each image before training.
Question
Given two different images that are different formats and sizes (1.jpg and 2.png), how can I avoid a ValueError while fitting the model?
I have one example where I train using only 1.jpg, which fits successfully.
I have another example where I train using both 1.jpg and 2.png and a ValueError is produced.
This example will fit successfully:
import numpy as np
from sklearn import svm
import matplotlib.image as mpimg
target = [1, 2]
images = np.array([
# target 1
[mpimg.imread('./1.jpg'), mpimg.imread('./1.jpg')],
# target 2
[mpimg.imread('./1.jpg'), mpimg.imread('./1.jpg')],
])
n_samples = len(images)
data = images.reshape((n_samples, -1))
model = svm.SVC()
model.fit(data, target)
This example will raise a Value error.
Observe the different 2.png image in target 2.
import numpy as np
from sklearn import svm
import matplotlib.image as mpimg
target = [1, 2]
images = np.array([
# target 1
[mpimg.imread('./1.jpg'), mpimg.imread('./1.jpg')],
# target 2
[mpimg.imread('./2.png'), mpimg.imread('./1.jpg')],
])
n_samples = len(images)
data = images.reshape((n_samples, -1))
model = svm.SVC()
model.fit(data, target)
# ValueError: setting an array element with a sequence.
1.jpg
2.png
For this, I would really recommend using the tools in Keras that are specifically designed to preprocess images in a highly scalable and efficient way.
from keras.preprocessing.image import ImageDataGenerator
from PIL import Image
import matplotlib.pyplot as plt
import numpy as np
1 Determine the target size of your new pictures
h,w = 150,150 # desired height and width
batch_size = 32
N_images = 100 #total number of images
Keras works in batches, so batch_size just determines how many pictures at once will be processed (this does not impact your end result, just the speed).
2 Create your Image Generator
train_datagen = ImageDataGenerator(
rescale=1./255)
train_generator = train_datagen.flow_from_directory(
'Pictures_dir',
target_size=(h, w),
batch_size=batch_size,
class_mode = 'binary')
The object that is going to do the image extraction is ImageDataGenerator. It has the method flow_from_directory which I believe might be useful for you here. It will read the content of the folder Pictures_dir and expect your images to be in folders by class (eg: Pictures_dir/class0 and Pictures_dir/class1). The generator, when called, will then create images from these folders and also import their label (in this example, 'class0' and 'class1').
There are plenty of other arguments to this generator, you can check them out in the Keras documentation (especially if you want to do data augmentation).
Note: this will take any image, be it PNG or JPG, as you requested
If you want to get the mapping from class names to label indices, do:
train_generator.class_indices
# {'class0': 0, 'class1': 1}
You can check what is going on with
plt.imshow(train_generator[0][0][0])
3 Extract all resized images from the Generator
Now you are ready to extract the images from the ImageGenerator:
def extract_images(generator, sample_count):
images = np.zeros(shape=(sample_count, h, w, 3))
labels = np.zeros(shape=(sample_count))
i = 0
for images_batch, labels_batch in generator: # we are looping over batches
images[i*batch_size : (i+1)*batch_size] = images_batch
labels[i*batch_size : (i+1)*batch_size] = labels_batch
i += 1
if i*batch_size >= sample_count:
# we must break after every image has been seen once, because generators yield indifinitely in a loop
break
return images, labels
images, labels = extract_images(train_generator, N_images)
print(labels[0])
plt.imshow(images[0])
Now you have your images all at the same size in images, and their corresponding labels in labels, which you can then feed into any scikit-learn classifier of your choice.
Its difficult because of the math operations behind the scene, (the details are out of scope) if you manage do so, lets say you build your own algorithm, still you would not get the desired result.
i had this issue once with faces with different sizes. maybe this piece of code give you starting point.
from PIL import Image
import face_recognition
def face_detected(file_address = None , prefix = 'detect_'):
if file_address is None:
raise FileNotFoundError('File address required')
image = face_recognition.load_image_file(file_address)
face_location = face_recognition.face_locations(image)
if face_location:
face_location = face_location[0]
UP = int(face_location[0] - (face_location[2] - face_location[0]) / 2)
DOWN = int(face_location[2] + (face_location[2] - face_location[0]) / 2)
LEFT = int(face_location[3] - (face_location[3] - face_location[2]) / 2)
RIGHT = int(face_location[1] + (face_location[3] - face_location[2]) / 2)
if UP - DOWN is not LEFT - RIGHT:
height = UP - DOWN
width = LEFT - RIGHT
delta = width - height
LEFT -= int(delta / 2)
RIGHT += int(delta / 2)
pil_image = Image.fromarray(image[UP:DOWN, LEFT:RIGHT, :])
pil_image.thumbnail((50, 50), Image.ANTIALIAS)
pil_image.save(prefix + file_address)
return True
pil_image = Image.fromarray(image)
pil_image.thumbnail((200, 200), Image.ANTIALIAS)
pil_image.save(prefix + file_address)
return False
Note : i wrote this long time ago maybe not a good practice
I have these two functions in my program:
def depict_ph_increase(x,y,color, imobject):
program_print(color)
draw = PIL.ImageDraw.Draw(imobject)
draw.text((x, y),color,(255,255,255))
imobject.save('tmp-out.gif')
im_temp = PIL.Image.open("tmp-out.gif")#.convert2byte()
im_temp = im_temp.resize((930, 340), PIL.Image.ANTIALIAS)
MAP_temp = ImageTk.PhotoImage(im_temp)
map_display_temp = Label(main, image=MAP_temp)
map_display_temp.image = MAP_temp # keep a reference!
map_display_temp.grid(row=4,column=2, columnspan=3)
def read_temp_pixels(temperature_file, rngup, rngdown):
temp_image_object = PIL.Image.open(temperature_file)
(length, width) = get_image_size(temp_image_object)
(rngxleft, rngxright) = rngup
(rngyup,rngydown) = rngdown
print 'the length and width is'
print length, width
hotspots = 5;
for hotspot in range(0,hotspots):
color = "#ffffff"
while color == "#ffffff" or color == "#000000" or color == "#505050" or color == "#969696":
yc = random.randint(rngxleft, rngxright)
xc = random.randint(rngyup,rngydown)
color = convert_RGB_HEX(get_pixel_color(temp_image_object, xc, yc))
depict_ph_increase(xc,yc,color, temp_image_object)
The bottom one calls the top one. Their job is to read in this image:
It then randomly selects a few pixels, grabs their colors, and writes the hex values of the colors on top. But, when it redisplays the image, it gives me this garbage:
Those white numbers up near the upper right corner are the hex values its drawing. Its somehow reading the values from the corrupted image, despite the fact that I don't collect the values until AFTER I actually call the ImageDraw() method. Can someone explain to me why it is corrupting the image?
Some background--the get_pixel_color() function is used several other times in the program and is highly accurate, its just reading the pixel data from the newly corrupted image somehow. Furthermore, I do similar image reading (but not writing) at other points in my code.
If there is anything I can clarify, or any other part of my code you want to see, please let me know. You can also view the program in its entirety at my github here: https://github.com/jrfarah/coral/blob/master/src/realtime.py It should be commit #29.
Other SO questions I have examined, to no avail: Corrupted image is being saved with PIL
Any help would be greatly appreciated!
I fixed the problem by editing this line:
temp_image_object = PIL.Image.open(temperature_file)
to be
temp_image_object = PIL.Image.open(temperature_file).convert('RGB')
I wish to skeletonize this image
To do so i am using matlab's bwmorph function, Here is the snippet :
bw = bwmorph(img_bw,'skel',Inf);
However the output is not as expected. Here is the output.
Could someone suggest a better way to achieve proper results ?
EDIT: here is a stripped down relevant code
img = imread(name);
img = rgb2gray(img*4);
img_bw = img > 50;
img_bw = medfilt2(img_bw,[10 10]);
bw = bwmorph(img_bw,'skel',Inf);
What you see is aliasing, the imshow function can not display the full image because it is to large to fit the screen. To fit the screen some rows and columns are skipped, which cause the lines to be disconnected. To display an image at full resolution using a scrollpanel, use imscrollpanel
hFig = figure('Toolbar','none', 'Menubar','none');
hIm = imshow(bw);
hSP = imscrollpanel(hFig,hIm);
Below is the current working code in python using PIL for highlighting the difference between the two images. But rest of the images is blacken.
Currently i want to show the background as well along with the highlighted image.
Is there anyway i can keep the show the background lighter and just highlight the differences.
from PIL import Image, ImageChops
point_table = ([0] + ([255] * 255))
def black_or_b(a, b):
diff = ImageChops.difference(a, b)
diff = diff.convert('L')
# diff = diff.point(point_table)
h,w=diff.size
new = diff.convert('RGB')
new.paste(b, mask=diff)
return new
a = Image.open('i1.png')
b = Image.open('i2.png')
c = black_or_b(a, b)
c.save('diff.png')
!https://drive.google.com/file/d/0BylgVQ7RN4ZhTUtUU1hmc1FUVlE/view?usp=sharing
PIL does have some handy image manipulation methods,
but also a lot of shortcomings when one wants
to start doing serious image processing -
Most Python lterature will recomend you to switch
to use NumPy over your pixel data, wich will give
you full control -
Other imaging libraries such as leptonica, gegl and vips
all have Python bindings and a range of nice function
for image composition/segmentation.
In this case, the thing is to imagine how one would
get to the desired output in an image manipulation program:
You'd have a black (or other color) shade to place over
the original image, and over this, paste the second image,
but using a threshold (i.e. a pixel either is equal or
is different - all intermediate values should be rounded
to "different) of the differences as a mask to the second image.
I modified your function to create such a composition -
from PIL import Image, ImageChops, ImageDraw
point_table = ([0] + ([255] * 255))
def new_gray(size, color):
img = Image.new('L',size)
dr = ImageDraw.Draw(img)
dr.rectangle((0,0) + size, color)
return img
def black_or_b(a, b, opacity=0.85):
diff = ImageChops.difference(a, b)
diff = diff.convert('L')
# Hack: there is no threshold in PILL,
# so we add the difference with itself to do
# a poor man's thresholding of the mask:
#(the values for equal pixels- 0 - don't add up)
thresholded_diff = diff
for repeat in range(3):
thresholded_diff = ImageChops.add(thresholded_diff, thresholded_diff)
h,w = size = diff.size
mask = new_gray(size, int(255 * (opacity)))
shade = new_gray(size, 0)
new = a.copy()
new.paste(shade, mask=mask)
# To have the original image show partially
# on the final result, simply put "diff" instead of thresholded_diff bellow
new.paste(b, mask=thresholded_diff)
return new
a = Image.open('a.png')
b = Image.open('b.png')
c = black_or_b(a, b)
c.save('c.png')
Here's a solution using libvips:
import sys
from gi.repository import Vips
a = Vips.Image.new_from_file(sys.argv[1], access = Vips.Access.SEQUENTIAL)
b = Vips.Image.new_from_file(sys.argv[2], access = Vips.Access.SEQUENTIAL)
# a != b makes an N-band image with 0/255 for false/true ... we have to OR the
# bands together to get a 1-band mask image which is true for pixels which
# differ in any band
mask = (a != b).bandbool("or")
# now pick pixels from a or b with the mask ... dim false pixels down
diff = mask.ifthenelse(a, b * 0.2)
diff.write_to_file(sys.argv[3])
With PNG images, most CPU time is spent in PNG read and write, so vips is only a bit faster than the PIL solution.
libvips does use a lot less memory, especially for large images. libvips is a streaming library: it can load, process and save the result all at the same time, it does not need to have the whole image loaded into memory before it can start work.
For a 10,000 x 10,000 RGB tif, libvips is about twice as fast and needs about 1/10th the memory.
If you're not wedded to the idea of using Python, there are a few really simple solutions using ImageMagick:
“Diff” an image using ImageMagick