I want to to data augmentation on an 8*8-pixel greayscale image through the codes below on Keras (the pixel values are only 0 and 1):
from ctypes import sizeof
from re import X
from turtle import shape
from keras.preprocessing.image import ImageDataGenerator
from skimage import io
import numpy as np
from PIL import Image
datagen = ImageDataGenerator(
rotation_range=45, #Random rotation between 0 and 45
width_shift_range=0.2, #% shift
height_shift_range=0.2,
shear_range=0.2,
zoom_range=0.2,
horizontal_flip=True,
fill_mode='nearest') #Also try nearest, constant, reflect, wrap
# forming a binary 8*8 array
array = np.array([[0,0,0,0,0,0,0,0],[0,0,0,0,0,0,0,0],[0,0,1,1,1,0,0,0],
[0,0,0,1,1,1,0,0],[0,0,0,0,0,0,0,0],[0,0,0,0,0,0,0,0],[0,0,0,0,0,0,1,0],[0,0,0,0,0,0,0,0]])
# scale values to uint8 maximum 255, and convert it to greyscale image
array = ((array) * 255).astype(np.uint8)
x = Image.fromarray(array)
i = 0
for batch in datagen.flow(x, batch_size=16,
save_to_dir='augmented',
save_prefix='aug',
save_format='png'):
i += 1
if i > 20:
break # otherwise the generator would loop indefinitely
But I get this error in the output (when I have .flow function):
ValueError: ('Input data in `NumpyArrayIterator` should have rank 4. You passed an array with shape', (8, 8))
Could anyone give me some hands please?
ImageDataGenerator accepts input as 4-dimensional tensor, where first dimension is sample number and last dimension are color channels. In your code you should convert this (8,8) tensor to (1,8,8,1) tensor. This can be done by
array = np.expand_dims(array, (0, -1))
Also you should not convert array to image before passing it to generator as you did it here
x = Image.fromarray(array)
you should simply pass array to generator.
Related
I am trying to convert an 8-by-8 numpy array of binary format (0 represents black and 1 represents white). This is what I run:
from PIL import Image
data = im.fromarray(array) # array is an 8*8 binary numpy
data.save('dummy_pic.png')
But in the output I get a fully black square. Could anyone give me a hand please?
Black square is probably very dark gray, because you may have np.array with uint8 datatype, which has range of 0-255, not 0-1 like binary array. Try changing array datatype to bool, or scale values to 0-255 range.
Here is code snippet in which binary array is generated and displayed. If you scale by smaller value, circle will become slightly darker.
from PIL import Image
import numpy as np
# Generating binary array which represents circle
radius = 0.9
size = 100
x,y = np.meshgrid(np.linspace(-1,1,size),np.linspace(-1,1,size))
f = np.vectorize(lambda x,y: ( 1.0 if x*x + y*y < radius*radius else 0.0))
array = f(x,y)
# Solution 1:
# convert np.array to bool datatype
array = (array).astype(np.bool)
# Solution 2:
# scale values to uint8 maximum 255
array = ((array) * 255).astype(np.uint8)
img = Image.fromarray(array)
display(img)
Result: White circle
Slic can implement segmentation under binarized masks, as shown in the figure below
from https://scikit-image.org/docs/dev/auto_examples/segmentation/plot_mask_slic.html
But if I need to divide the superpixels of different adjacent regions, what should I do?
Each color represents an area, each region requires independent superpixel segmentation
There is not currently any way to handle a mask with multiple regions in a single call. For your use case you will have to split each region into a separate mask and then call slic once per mask. You can combine the multiple segmentations into one by incrementing the labels appropriately.
Pasted below is a concrete example of this for two separate masked regions (adapted from the existing example you referenced):
import matplotlib.pyplot as plt
import numpy as np
from skimage import data
from skimage import color
from skimage import morphology
from skimage import segmentation
# Input data
img = data.immunohistochemistry()
# Compute a mask
lum = color.rgb2gray(img)
mask = morphology.remove_small_holes(
morphology.remove_small_objects(
lum < 0.7, 500),
500)
mask1 = morphology.opening(mask, morphology.disk(3))
# create a second mask as the inverse of the first
mask2 = ~mask1
segmented = np.zeros(img.shape[:-1], dtype=np.int64)
max_label = 0
# replace [mask2, mask1] with a list of any number of binary masks
for mask in [mask2, mask1]:
# maskSLIC result
m_slic = segmentation.slic(img, n_segments=100, mask=mask, start_label=1)
if max_label > 0:
# offset the labels by the current maximum label
m_slic += max_label
# add the label into the current combined segmentation
segmented += m_slic
# increment max label
max_label += m_slic.max()
# Display result
fig, ax_arr = plt.subplots(2, 2, sharex=True, sharey=True, figsize=(10, 10))
ax1, ax2, ax3, ax4 = ax_arr.ravel()
ax1.imshow(img)
ax1.set_title('Original image')
ax2.imshow(mask, cmap='gray')
ax2.set_title('Mask')
ax3.imshow(segmentation.mark_boundaries(img, m_slic))
ax3.contour(mask, colors='red', linewidths=1)
ax3.set_title('maskSLIC (mask1 only)')
ax4.imshow(segmentation.mark_boundaries(img, segmented))
ax4.contour(mask, colors='red', linewidths=1)
ax4.set_title('maskSLIC (both masks)')
for ax in ax_arr.ravel():
ax.set_axis_off()
plt.tight_layout()
plt.show()
The basic approach I am suggesting is in the for loop above. Most of the other code is just generating the data and plots.
An exciting animation was posted on twitter recently: https://twitter.com/thomas_rackow/status/1392509885883944960.
One of the authors explained in this Jupyter Notebook https://nbviewer.jupyter.org/github/koldunovn/FESOM_SST_shaded_by_U/blob/main/FESOM_SST_shaded_by_U.ipynb
how a frame is created.
Related to the simple code displayed by this notebook, my question is: when we call imshow twice for the same ax:
ax.imshow(np.flipud(sst.sst.values), cmap=cm.RdBu_r, vmin=12, vmax=24)
ax.imshow(np.flipud(u.u_surf.values), alpha=0.3, cmap=cm.gray, vmin=-.3, vmax=0.3)
what operations performs matplotlib, behind the scenes, to get a layered image?
I worked with alpha blending in Open CV - Python, but here it starts with two arrays of the same shape (1000, 1000), and via ax.imshow, called twice for the two arrays, it displays the resulting image. I'd like to know how is it possible. What arithmetic operations between images are involved?
I searched the matplotlib github repository to understand what's going on, but I couldn't find something relevant.
I succeeded to illustrate that the two imshow(s) hide the alpha-blending of the two images.
import numpy as np
import xarray as xr
import matplotlib.pyplot as plt
import matplotlib.cm as cm
sst = xr.open_dataset('GF_FESOM2_testdata/sst.nc')
u = xr.open_dataset('GF_FESOM2_testdata/u_surf.nc')
v = xr.open_dataset('GF_FESOM2_testdata/v_surf.nc')
#Define the heatmap from SST-data and extract the array representing it as an image:
fig1, ax1 = plt.subplots(1, 1,
constrained_layout=True,
figsize=(10, 10))
f1 = ax1.imshow(np.flipud(sst.sst.values), cmap=cm.RdBu_r, vmin=12, vmax=24)
ax1.axis('off');
arr1 = f1.make_image('notebook')[0] #array representing the above image
#Repeat the same procedure for u-data set:
fig2, ax2 = plt.subplots(1, 1,
constrained_layout=True,
figsize=(10, 10))
f2 = ax2.imshow(np.flipud(u.u_surf.values), cmap=cm.gray, vmin=-0.3, vmax=0.3)
ax2.axis('off');
arr2 = f2.make_image("notebook")[0]
#alpha blending of the two images amounts to a convex combination of the associated arrays
alpha1= 1 # background image alpha
alpha2 = 0.3 #foreground image alpha
arr = np.asarray((alpha2*arr2 + alpha1*(1-alpha2)*arr1)/(alpha2+alpha1*(1-alpha2)), dtype=np.uint8)
fig, ax = plt.subplots(1, 1,
constrained_layout=True,
figsize=(10, 10))
ax.imshow(np.flipud(arr))
ax.axis('off');
I've loaded an image into the scikit-image library in Python.
I've also created a mask of the same dimensions as the image, and now I want to intersect the mask with the image to only show pixels from the image where the mask is non-zero. How can I do that without iterating pixel by pixel?
Do I just apply the mask to the image, like:
new_image = image[mask]
Or can I just multiply the two image arrays together to do bitwise multiplication pixel by pixel?
Element-wise multiplication indeed works perfectly:
from skimage import data
from matplotlib import pyplot as plt
image = data.coins()
mask = image > 128
masked_image = image * mask
fig, (ax0, ax1) = plt.subplots(nrows=1, ncols=2)
ax0.imshow(image, cmap='gray')
ax1.imshow(masked_image, cmap='gray')
Note 1: your code example is not a scikit-image question but a NumPy indexing question, and will not do what you want, but rather return a linear array of all the pixels where mask is True. For more information, see the NumPy documentation on boolean indexing.
Note 2: you can also use scikit-image to save images:
from skimage import io
io.imsave('masked_image.png', masked_image)
You can do it like this:
from skimage import data
import numpy as np
from PIL import Image
# Load coins data-set
im = data.coins()
# Make mask of where image is less than mid-grey
mask = im<128
# Set image black everywhere it was less than mid-grey
im[mask] = 0
# Set image mid-grey everywhere it was mid-grey or brighter
im[~mask] = 128
# Convert to PIL image and save
Image.fromarray(im).save('result.png')
Starting image of coins:
Resulting image:
If I have a list of images represented by 3D ndarray such as [[x,y,color],...], what operations can I use to output an image with values that are median of all values? I am using a for loop and find it too slow.
This is my vectorized implementation using NumPy:
For my test I used these five images:
The relevant parts:
import numpy as np
import scipy.ndimage
# Load five images:
ims = [scipy.ndimage.imread(str(i + 1) + '.png', flatten=True) for i in range(5)]
# Stack the reshaped images (rows) vertically:
ims = np.vstack([im.reshape(1,im.shape[0] * im.shape[1]) for im in ims])
# Compute the median column by column and reshape to the original shape:
median = np.median(ims, axis=0).reshape(100, 100)
The complete script:
import numpy as np
import scipy.ndimage
import matplotlib.pyplot as plt
ims = [scipy.ndimage.imread(str(i + 1) + '.png', flatten=True) for i in range(5)]
print ims[0].shape # (100, 100)
ims = np.vstack([im.reshape(1,im.shape[0] * im.shape[1]) for im in ims])
print ims.shape # (5, 10000)
median = np.median(ims, axis=0).reshape(100, 100)
fig = plt.figure(figsize=(100./109., 100./109.), dpi=109, frameon=False)
ax = fig.add_axes([0, 0, 1, 1])
ax.axis('off')
plt.imshow(median, cmap='Greys_r')
plt.show()
The median (numpy.median) result of the five images looks like this:
Fun part: The mean (numpy.mean) result looks like this:
Okay, science meets art. :-)
You said your images in color, formatted as a list of 3d ndarrays. Let's say there are n images:
imgs = [img_1, ..., img_n]
Where imgs is a list and each img_i is a an ndarray with shape (nrows, ncols, 3).
Convert the list to a 4d ndarray, then take the median over the dimension that corresponds to images:
import numpy as np
# Convert images to 4d ndarray, size(n, nrows, ncols, 3)
imgs = np.asarray(imgs)
# Take the median over the first dim
med = np.median(imgs, axis=0)
This gives the pixel-wise median. The value of each color channel of each pixel is the median of the corresponding pixel/channel in all images.
The documentation for asarray() says "no copy is performed if the input is already an ndarray". This suggests that the operation would be faster if you stored the original list of images as a 4d ndarray instead of a list. In that case, it wouldn't be necessary to copy the information in memory (or to run asarray())
Can you put an example of your data ?
Else, I think that you could maybe use numpy with numpy.mean ?
You have the doc here ;)