How to do I apply the same augmentations to two sets of similar data - tensorflow-datasets

I am trying to train an image translation model with TensorFlow, the dataset consists of two equal length list of images, how do I apply a map both of them so they receive the same augmentations.
I have tried to create a variable to keep track of which iteration it is, as well as using a seeded random number.
def load_and_preprocess_color(path):
image = tf.read_file(path)
image = tf.image.decode_png(image, channels=3)
image = tf.image.resize(image, [224, 224])
image = tf.image.random_flip_left_right(image)
image /= 255.0 # normalize to [0,1] range
return image
def load_and_preprocess_gray(path):
image = tf.read_file(path)
image = tf.image.decode_png(image, channels=1)
image = tf.image.resize(image, [224, 224])
image = tf.image.random_flip_left_right(image)
image /= 255.0 # normalize to [0,1] range
return image
input_data_root = pathlib.Path('Images/Gray/') output_data_root =
pathlib.Path('Images/Color/')
input_image_paths = [str(item) for item in input_data_root.iterdir()]
output_image_paths = [str(item) for item in
output_data_root.iterdir()] image_count = len(input_image_paths)
input_path_ds = tf.data.Dataset.from_tensor_slices(input_image_paths)
output_path_ds =
tf.data.Dataset.from_tensor_slices(output_image_paths)
input_ds =
input_path_ds.map(load_and_preprocess_gray,num_parallel_calls=AUTOTUNE)
output_ds =
output_path_ds.map(load_and_preprocess_color,num_parallel_calls=AUTOTUNE`
I expect two pieces of data at the same position to be either flipped of not flipped, but its flipped state to be the same as the other

Related

Image pixel value normalized for tf.image.decode_jpeg and tf.train.shuffle_batch?

I am trying to use the tf.train.shuffle_batch function from tensorflow, then I need to first load the images using tf.image.decode_jpeg(or other similar functions to load png and jpg). But I just found out that the images are loaded as probability map, which means the max of the value of pixel is 1, and the min of the value of the pixel is 0. Below is my code updated from a github repo. I don't know why the values of pixels are normalized to [0,1], and I don't find related documentation on tensorflow. Could anyone help me? Thanks.
def load_examples(self, input_dir, flip, scale_size, batch_size, min_queue_examples):
input_paths = get_image_paths(input_dir)
with tf.name_scope("load_images"):
path_queue = tf.train.string_input_producer(input_paths)
reader = tf.WholeFileReader()
paths, contents = reader.read(path_queue)
# note this is important for truncated images
raw_input = tf.image.decode_jpeg(contents,try_recover_truncated = True, acceptable_fraction=0.5)
raw_input = tf.image.convert_image_dtype(raw_input, dtype=tf.float32)
raw_input.set_shape([None, None, 3])
# break apart image pair and move to range [-1, 1]
width = tf.shape(raw_input)[1] # [height, width, channels]
a_images = preprocess(raw_input[:, :width // 2, :])
b_images = raw_input[:, width // 2:, :]
inputs, targets = [a_images, b_images]
def transform(image):
r = image
r = tf.image.resize_images(r, [self.image_height, self.image_width], method=tf.image.ResizeMethod.AREA)
return r
def transform_gaze(image):
r = image
r = tf.image.resize_images(r, [self.gaze_height, self.gaze_width], method=tf.image.ResizeMethod.AREA)
return r
with tf.name_scope("input_images"):
input_images = transform(inputs)
with tf.name_scope("target_images"):
target_images = transform(targets)
total_image_count = len(input_paths)
# target_images = tf.image.per_image_standardization(target_images)
target_images = target_images[:,:,0]
target_images = tf.expand_dims(target_images, 2)
inputs_batch, targets_batch = tf.train.shuffle_batch([input_images, target_images],
batch_size=batch_size,
num_threads=1,
capacity=min_queue_examples + 3 * batch_size,
min_after_dequeue=min_queue_examples)
# inputs_batch, targets_batch = tf.train.batch([input_images, target_images],batch_size=batch_size)
return inputs_batch, targets_batch, total_image_count
Values are into [0,1] because is what tf.image.decode_* methods do.
In general, when a method returns a float tensor, its values are supposed to be in the [0,1] range, whilst if the returned tensor is a uint8 the values are supposed to be in the [0,255] range.
Also, when you use the tf.image.convert_image_dtype method, to convert the dtype of the input image, you're applying that conversion rules.
If your input image is a uint8 image and you convert it to a float32, the values are scaled in the [0,1] range. If your image is already a float, its values are supposed to be in that range and nothing is done.

Resize a polygon image in matlab

I have two polygon images defined by 25 control points. I want to replace one polygon by another one in matlab. Below is an example of TC and BP.
I have added the code. I am not happy with the output texture in the replaced area. Also, I found the that if the polygon shape of the second image is smaller than the first image polygon shape then the output looks very bad.
clc;clear all;close all
im_original = imread('tc.jpg');
im_original=im2double(im_original);
%% ROI (X,Y) coordinates, variable name (pt_original)
load('tc.mat');
im_morphed = imread('bp.jpg');
img_morphed=im2double(im_morphed);
%% ROI (X,Y) coordinates, variable name (pt_morphed)
load('bp.mat');
%% Replace Face
[img_proc,mask] = defineRegion(im_original,pt_original);
img_morphed_proc = histeq_rgb(img_morphed, im_original, mask, mask);
sigma = 5;
se = strel('square',sigma);
mask = imerode(mask,se);
w = fspecial('gaussian',[50 50],sigma);
mask = imfilter(double(mask),w);
img_result = bsxfun(#times,double(img_morphed_proc),double(mask)) + bsxfun(#times,double(im_original),double(1-mask));
imshow(img_result)
function [img_proc,mask] = defineRegion(img, landmark)
sz = size(img);
k =convhull(landmark(:,2),landmark(:,1));
[YY,XX] = meshgrid(1:sz(1),1:sz(2));
in = inpolygon(XX(:),YY(:),landmark(k,1),landmark(k,2));
mask = reshape(in,[sz(2),sz(1)])';
img_proc = bsxfun(#times,im2double(img),double(mask));
end
function img_proc = histeq_rgb(img_src, img_dest, mask_src, mask_dest)
img_proc = img_src;
for i = 1 : 3
tmp_src = img_src(:,:,i);
tmp_src = tmp_src(mask_src);
tmp_dest = img_dest(:,:,i);
tmp_dest = tmp_dest(mask_dest);
t = histeq(tmp_src,imhist(tmp_dest));
tmp_proc = img_proc(:,:,i);
tmp_proc(mask_src) = t;
img_proc(:,:,i) = tmp_proc;
end
end
Output Image

automatically finding length of object in binary image (MATLAB)

I have the following binary image:
http://www.4shared.com/download/BozvHQcHba/untitled2.jpg?lgfp=3000
I manually select the start and end points using the ruler in imtool to get the length. Is there a way to automatically get the length i.e the first white pixel to last white pixel (longest length) without doing it manually.
Code
%%// Get the binary image
img = imread(filename1);
BW = im2bw(img);
%%// Find biggest blob
[L,num] = bwlabel( BW );
count_pixels_per_obj = sum(bsxfun(#eq,L(:),1:num));
[~,ind] = max(count_pixels_per_obj);
biggest_blob = (L==ind);
%%// Find row and column info for all edge pixels
BW1 = edge(biggest_blob,'canny');
[row1,col1] = find(BW1);
%%// Get the distance matrix and thus find the largest length separating
%%// them which is the length of the object/blob
%dist_mat = pdist2([row1 col1],[row1 col1]);
dist_mat = dist2s([row1 col1],[row1 col1]); %// If you do not have pdist2
length_blob = max(dist_mat(:))
Associated function
function out = dist2s(pt1,pt2)
out = NaN(size(pt1,1),size(pt2,1));
for m = 1:size(pt1,1)
for n = 1:size(pt2,1)
if(m~=n)
out(m,n) = sqrt( (pt1(m,1)-pt2(n,1)).^2 + (pt1(m,2)-pt2(n,2)).^2 );
end
end
end

How to find more than one matching pattern using Normalized Correalation

I'm using normxcorr2 to find the area that exactly match with my pattern and i also want to find the other area(in the red rectangle) that is look like the pattern. I think it will be works if i can find the next maximum and so on and that value must not in the first maximum area or the first one that it has been detected but i can't do it. Or if you have any idea that using normxcorr2 to find the others area please advise me, I don't have any idea at all.
Here's my code. I modified from this one http://www.mathworks.com/products/demos/image/cross_correlation/imreg.html
onion = imread('pattern103.jpg'); %pattern image
peppers = imread('rsz_1jib-159.jpg'); %Original image
onion = rgb2gray(onion);
peppers = rgb2gray(peppers);
%imshow(onion)
%figure, imshow(peppers)
c = normxcorr2(onion,peppers);
figure, surf(c), shading flat
% offset found by correlation
[max_c, imax] = max(abs(c(:)));
[ypeak, xpeak] = ind2sub(size(c),imax(1));
corr_offset = [(xpeak-size(onion,2))
(size(onion,1)-ypeak)]; %size of window show of max value
offset = corr_offset;
xoffset = offset(1);
yoffset = offset(2);
xbegin = round(xoffset+1); fprintf(['xbegin = ',num2str(xbegin)]);fprintf('\n');
xend = round(xoffset+ size(onion,2));fprintf(['xend = ',num2str(xbegin)]);fprintf('\n');
ybegin = round(yoffset+1);fprintf(['ybegin = ',num2str(ybegin)]);fprintf('\n');
yend = round(yoffset+size(onion,1));fprintf(['yend = ',num2str(yend)]);fprintf('\n');
% extract region from peppers and compare to onion
extracted_onion = peppers(ybegin:yend,xbegin:xend,:);
if isequal(onion,extracted_onion)
disp('pattern103.jpg was extracted from rsz_org103.jpg')
end
recovered_onion = uint8(zeros(size(peppers)));
recovered_onion(ybegin:yend,xbegin:xend,:) = onion;
figure, imshow(recovered_onion)
[m,n,p] = size(peppers);
mask = ones(m,n);
i = find(recovered_onion(:,:,1)==0);
mask(i) = .2; % try experimenting with different levels of
% transparency
% overlay images with transparency
figure, imshow(peppers(:,:,1)) % show only red plane of peppers
hold on
h = imshow(recovered_onion); % overlay recovered_onion
set(h,'AlphaData',mask)

Matlab - displaying a background image

I'm trying to find the median values for the R,G & B channels of each pixel for each 10th image in a set of 100, to find the background image. My values all seem correct but when i try to display the background at the end of my code it's always white, please help
%// list all the files in some folder
folder = '~/V&R/1/';
filelist = dir(folder);
images = zeros(480,640,3,100);
% images = [];
%// the first two in filelist are . and ..
count = 1;
for i=3:size(filelist,1)
if filelist(i).isdir ~= true
fname = filelist(i).name;
%// if file extension is jpg
if strcmp( fname(size(fname,2)-3:size(fname,2)) ,'.jpg' ) == 1
tmp = imread([folder fname]);
images(:,:,:,count) = tmp;
count = count +1;
end
end
end
background = zeros(480,640,3);
for j=1:480
for i=1:640
tmpR = zeros(1,10);
tmpG = zeros(1,10);
tmpB = zeros(1,10);
for k=1:10
tmpR(k) = images(j,i,1,k*10);
tmpG(k) = images(j,i,2,k*10);
tmpB(k) = images(j,i,3,k*10);
end
background(j,i,1) = floor(median(tmpR));
background(j,i,2) = floor(median(tmpG));
background(j,i,3) = floor(median(tmpB));
end
end
imshow(background)
thanks
The first step is to vectorize your code. Instead of the following block of code:
background = zeros(480,640,3);
for j=1:480
for i=1:640
tmpR = zeros(1,10);
tmpG = zeros(1,10);
tmpB = zeros(1,10);
for k=1:10
tmpR(k) = images(j,i,1,k*10);
tmpG(k) = images(j,i,2,k*10);
tmpB(k) = images(j,i,3,k*10);
end
background(j,i,1) = floor(median(tmpR));
background(j,i,2) = floor(median(tmpG));
background(j,i,3) = floor(median(tmpB));
end
end
write:
subimages = images(:, :, :, 1:10:end);
background = median(subimages, 4);
now as said before, use imshow with the [] option to show your image:
imshow(background, []);
if you still see a white image, then it's possible that you are dealing with a matrix of double values that are not between [0, 1]. Images in Matlab are usually of class double or single with values between 0 and 1, or of class uint8 or uint16 with values between 0, 255 or 0, 65535 respectively. If your values are between 0 and 255 but class(subimages) returns double or single, do the following before using imshow():
subimages = uint8(subimages);
Try
imshow(background,[])
When using imshow, MATLAB needs to set a display range. For single or double grayscale images, the default display range is [0 1]. This means that any value larger than 1 will be represented as white. You can fix this by setting your own display range manually, say
imshow(background,[0 100],
or you can let MATLAB calculate a new display range by doing
imshow(background,[])
which is the same as
imshow(background,[min(background(:)) max(background(:))])
You can rewrite your code as:
%# get filenames of all JPG images in some folder
folder = '~/V&R/1/';
filelist = dir( fullfile(folder,'*.jpg') );
filelist = strcat(folder, filesep, {filelist.name});
%# read files, and store as 'double' images in a 4D matrix
images = zeros(480,640,3, numel(filelist), 'double');
for i=1:numel(filelist)
images(:,:,:,i) = im2double( imread(filelist{i}) );
end
%# estimate background using median
subimages = images(:,:,:,1:10:end);
background = median(subimages, 4);
imshow(background)

Resources