I have a code to augment images like the following,
# Augmentation
train_datagen = ImageDataGenerator(rotation_range=5, # rotation
width_shift_range=0.2, # horizontal shift
zoom_range=0.2, # zoom
horizontal_flip=True, # horizontal flip
brightness_range=[0.2,0.8]) # brightness
# Epochs
epochs = 25
# Batch size
batch_size = 32
history = model.fit(train_datagen.flow(x_train,y_train,
batch_size=batch_size,
seed=27,
shuffle=False),
epochs=epochs,
steps_per_epoch=x_train.shape[0] // batch_size,
validation_data=(x_test,y_test),
verbose=1)
I am trying to understand exactly how many extra images will be created in the training process as a result of augmentation.
The second question is how can I create extra 50K images on the fly for the training?
The datagenerator doesn't create new images it rather just transforms it for each epoch.
if you have x_train ) = [x1,x2,x3] images in your entire training set, upon training in each epoch the model should see the same x_train BUT your x_train is so small (just 3 images) so the thing is for each epoch the datagen will feed the model the whole x_train slightly transformed (according to the parameters you put in ImageDataGenerator) e.g.:
for epoch 1 x_train: [x1,x2,x3]
for epoch 2 x_train: [x1_t1,x2_t1,x3_t1]
for epoch 3 x_train: [x1_t2,x2_t2,x3_t2]
etc...
Related
I have a Geodataframe ("mhg") in which the index are months (i.e. "2019-01-01", "2019-02-01", ...), and the GDF have a column that is the geometry of certain regions (i.e. POLYGON(...)), and finally another column that is the population corresponding to that geometry at that month.
sample data (with onyl two months) could be created by:
import geopandas as gpd
data = [['2019-01-01', 'POLYGON(123...)', 1000], ['2019-01-01', 'POLYGON(456...)', 1500], ['2019-01-01', 'POLYGON(789...)', 1400], ['2019-02-01', 'POLYGON(123...)', 1100], ['2019-02-01', 'POLYGON(456...)', 1600], ['2019-02-01', 'POLYGON(789...)', 1300]]
mhg = gpd.GeoDataFrame(data, columns=['month','geometry', 'population'])
mhg.set_index('month')
I can make a multicolor plot of the users living in each region (all periods) with:
mhg.plot(column='population',cmap='jet')
and I can make the same, but filtering by month, using:
mhg.loc['2019-01-01'].plot(column='population',cmap='jet')
I would like to get some kind of ""animation" or animated gif where I can see the temporal evolution of the population, by using this kind of pseudocode:
for all the plots in
mhg.loc['2019-01-01'].plot(column='population',cmap='jet')
mhg.loc['2019-02-01'].plot(column='population',cmap='jet')
mhg.loc['2019-03-01'].plot(column='population',cmap='jet')
...
then merge all plots into 1 animated gif
But I dont' know how to do it: the number of months can be up to hundreds, I don't how how to make the for loop, and I don't know even how to start...
Any suggestions?
EDIT: I tried the following (following https://linuxtut.com/en/c089c549df4d4a6d815c/):
months = np.sort(np.unique(mhg.month.values))
from matplotlib.animation import FuncAnimation
from matplotlib.animation import PillowWriter
fig, ax = plt.subplots()
ims = []
def update_fig(month):
if len(ims) > 0:
ims[0].remove()
del ims[0]
geos = mhg['geometry'].values
users = mhg[(mhg.month==month)].population
apl = gpd.plotting.plot_polygon_collection(ax, geos, population, True, cmap='jet')
ims.append(apl)
ax.set_title('month = ' + str(month))
return ims
anim = FuncAnimation(fig, update_fig, interval=1000, repeat_delay=3000, frames=months)
plt.show()
But I got a UserWarning: animation was deleted without rendering anything...
So I am stuck again.
I managed to do it this way:
mhg = mhg.reset_index()
groups = mhg.groupby('month')
for month, grp in groups:
grp.plot(column='users',cmap='jet',legend=True,figsize=(10, 10),norm=matplotlib.colors.LogNorm(vmin=mhg.users.min(), vmax=mhg.users.max()))
plt.title({month})
plt.xlim([-20, 5])
plt.ylim([25, 45])
plt.savefig("plot{month}.png".format(month=month), facecolor='white')
plt.close()
And then I joined all the png's with convert (imagemagick tool):
convert -delay 50 -loop 0 *.png aniamtion.gif
I am learning RNN with pytorch from this github.
Pytorch RNN Tutorial
I'm a little bit confused, because the code didn't show result of the training. Only show the accuracy. Now I want to show image that output from model.
I try to save image from output, but the result I get is a compressed representation (below is the image I got from output of the model)
# Train the model
total_step = len(train_loader)
for epoch in range(num_epochs):
for i, (images, labels) in enumerate(train_loader):
images = images.reshape(-1, sequence_length, input_size).to(device)
labels = labels.to(device)
# Forward pass
outputs = model(images)
loss = criterion(outputs, labels)
# Backward and optimize
optimizer.zero_grad()
loss.backward()
optimizer.step()
if (i+1) % 100 == 0:
print ('Epoch [{}/{}], Step [{}/{}], Loss: {:.4f}'
.format(epoch+1, num_epochs, i+1, total_step, loss.item()))
I am trying to show the image of the output by adding this code
plt.imshow(outputs.reshape(28, 28))
But I got this error
RuntimeError: shape '[28, 28]' is invalid for input of size 1000
So, I assume that the output is 1000 size. but I want it to show as a mnist image (784 size (28*28)).
I was trying to find snr for a set of images that I have but my two methodologies of doing so creates two different answers and I'm not sure which is right. I was wondering if one of them is just straight up the wrong way of doing this or if neither way is correct?
I am trying to characterize the snr of a set of images that I'm processing. I have 1 set of data with images and darkfields. From these pieces of data I subtracted the darkfield from the image and got "corrected_images".
So since I know snr is (mean of signal)/(std of noise), in my first methodology I was working with the corrected image and background noise image and I just took the mean of every pixel on the spectrum (from the corrected image) with a value greater than 1 for signal and the general std for the background noise image as my values. the plot for this methodology is in blue.
In my second methodology I used a single uncorrected image and basically considered every pixel above 50 as signal and every pixel below 50 as noise.This gives us the orange values for snr.
# -*- coding: utf-8 -*-
"""
Spyder Editor
This is a temporary script file.
"""
from PIL import Image
from matplotlib import pyplot as plt
import numpy as np
import os
signals=[]
name = r"interpolating_streaks/corrected"
name2 = r"interpolating_streaks/averages"
file = os.listdir(name)
file2 = os.listdir(name2)
wv1=[]
signal = []
snr = []
noise = []
x=0
for i in file:
wv=(i[:3])
wv1.append(wv)
corrected_image = Image.open(name+"/"+i) #opens the image
streak= np.array(corrected_image)
dark_image = Image.open(name2+'/d'+wv+'_averaged.tif')
dark = np.array(dark_image)
darkavg = dark[:][:].mean(axis=0)
avg= streak[:][:].mean(axis=0)
for i in avg:
if i >= 1:
signal.append(i)
noiser = np.std(darkavg)
signalr = np.mean(signal)
snr.append(signalr/noiser)
plt.plot(wv1,snr)
signal = []
noise = []
snr = []
for i in file2:
if(i[0] !='d'):
image = Image.open(name2+'/' + i )
im = np.array(image)
im_avg = im[:][:].mean(axis=0)
for i in im_avg:
if i <= 50:
noise.append(i)
else:
signal.append(i)
snr.append(np.mean(signal)/np.std(noise))
plt.plot(wv1,snr)
I would expect the snr values to be the same , and I know for my camera the snr has to be below 45 dB (but also I'm pretty sure this methodology for snr doesnt output decibels)
here are my current results
![1]: https://imgur.com/a/Vgecyp1
I am testing the adversarial sample attack using deepfool and sparsefool on mnist dataset. It did an attack on the preprocessed image data. However, when I save it into an image and then load it back, it fails attack.
I have test it using sparsefool and deepfool, and I think there are some precision problems when I save it into images. But I cannot figure it out how to implement it correctly.
if __name__ == "__main__":
# pic_path = 'testSample/img_13.jpg'
pic_path = "./hacked.jpg"
model_file = './trained/'
image = Image.open(pic_path)
image_array = np.array(image)
# print(np.shape(image_array)) # 28*28
shape = (28, 28, 1)
projection = (0, 1)
image_norm = tf.cast(image_array / 255.0 - 0.5, tf.float32)
image_norm = np.reshape(image_norm, shape) # 28*28*1
image_norm = image_norm[tf.newaxis, ...] # 1*28*28*1
model = tf.saved_model.load(model_file)
print(np.argmax(model(image_norm)), "nnn")
# fool_img, r, pred_label, fool_label, loops = SparseFool(
# image_norm, projection, model)
print("pred_label", pred_label)
print("fool_label", np.argmax(model(fool_img)))
pert_image = np.reshape(fool_img, (28, 28))
# print(pert_image)
pert_image = np.copy(pert_image)
# np.savetxt("pert_image.txt", (pert_image + 0.5) * 255)
pert_image += 0.5
pert_image *= 255.
# shape = (28, 28, 1)
# projection = (0, 1)
# pert_image = tf.cast(((pert_image - 0.5) / 255.), tf.float32)
# image_norm = np.reshape(pert_image, shape) # 28*28*1
# image_norm = image_norm[tf.newaxis, ...] # 1*28*28*1
# print(np.argmax(model(image_norm)), "ffffnnn")
png = Image.fromarray(pert_image.astype(np.uint8))
png.save("./hacked.jpg")
It should attack 4 to 9, however, the saved image is still predicted into 4.
The full code project is shared on
https://drive.google.com/open?id=132_SosfQAET3c4FQ2I1RS3wXsT_4W5Mw
Based on my research and also this paper as reference https://arxiv.org/abs/1607.02533
You can see in real life when you converted to images, all of the adversarial attack samples generated from attack will not work on in real world. it can explain as below "This could be explained by the fact that iterative methods exploit more subtle kind of
perturbations, and these subtle perturbations are more likely to be destroyed by photo transformation"
As example, your clean image has 127,200,55,..... you dividing into 255 (as it is 8bit png) and sending to you ML as (0.4980,0.78431,0.2156,...) . And deepfool is advanced attack method it added small perturb and change it to (0.4981,0.7841,0.2155...). Now this is adversarial sample which can fool your ML. but if you try to save it to 8bit png you will get again 127,200,55.. as you will multiply it by 255. So adversarial information is lost.
Simple put, you use deep fool method it added some perturb so small which essential not possible in real world 8bit png.
I am using an ImageDataGenerator to augment my images. I need to get the y labels from the generator.
Example : I have 10 training images, 7 are label 0 and 3 are label 1. I want to increase training set size to 100.
total_training_images = 100
total_val_images = 50
model.fit_generator(
train_generator,
steps_per_epoch= total_training_images // batch_size,
epochs=epochs,
validation_data=validation_generator,
validation_steps= total_val_images // batch_size)
By my understanding, this trains a model on 100 training images for each epoch, with each image being augmented in some way or the other according to my data generator, and then validates on 50 images.
If I do train_generator.classes, I get an output [0,0,0,0,0,0,0,1,1,1]. This corresponds to my 7 images of label 0 and 3 images of label 1.
For these new 100 images, how do I get the y-labels?
Does this mean when I am augmenting this to 100 images, my new train_generator labels are the same thing, but repeated 10 times? Essentially np.append(train_generator.classes) 10 times?
I am following this tutorial, if that helps :
https://blog.keras.io/building-powerful-image-classification-models-using-very-little-data.html
The labels generate as one-hot-encoding with the images.Hope this helps !
training_set.class_indices
from keras.preprocessing import image
import matplotlib.pyplot as plt
x,y = train_generator.next()
for i in range(0,3):
image = x[i]
label = y[i]
print (label)
plt.imshow(image)
plt.show()
Based on what you're saying about the generator, yes.
It will replicate the same label for each augmented image. (Otherwise the model would not train properly).
One simple way to check what the generator is outputting is to get what it yields:
X,Y = train_generator.next() #or next(train_generator)
Just remember that this will place the generator in a position to yield the second element, not the first anymore. (This would make the fit method start from the second element).