I'm trying to figure out how to take a crop of an image determined dynamically in Tensorflow. Below is an example of what I am trying to accomplish, however I can't seem to make it work. Essentially, I want to feed images and the crop values for that image within the graph, and then continue on with other computations on those cropped pieces. My current attempt:
import tensorflow as tf
from matplotlib import pyplot as plt
import numpy as np
sess = tf.InteractiveSession()
img1 = np.random.random([400, 600, 3])
img2 = np.random.random([400, 600, 3])
img3 = np.random.random([400, 600, 3])
images = [img1, img2, img3]
img1_crop = [100, 100, 100, 100]
img2_crop = [200, 150, 100, 100]
img3_crop = [150, 200, 100, 100]
crop_values = [img1_crop, img2_crop, img3_crop]
def crop_image(img, crop):
tf.image.crop_to_bounding_box(img,
crop[0],
crop[1],
crop[2],
crop[3])
image_placeholder = tf.placeholder("float", [None, 400, 600, 3])
crop_placeholder = tf.placeholder(dtype=tf.int32)
sess.run(tf.global_variables_initializer())
cropped_image = tf.map_fn(lambda img, crop: crop_image(img, crop), elems=[image_placeholder, crop_placeholder])
result = sess.run(cropped_image, feed_dict={image_placeholder: images, crop_placeholder:crop_values})
plt.imshow(result)
plt.show()
/Users/p111/anaconda/bin/python /Users/p111/PycharmProjects/analysis_code/testing.py
Traceback (most recent call last):
File "/Users/p111/PycharmProjects/analysis_code/testing.py", line 31, in
cropped_image = tf.map_fn(lambda img, crop: crop_image(img, crop), elems=[image_placeholder, crop_placeholder])
File "/Users/p111/anaconda/lib/python3.5/site-packages/tensorflow/python/ops/functional_ops.py", line 390, in map_fn
swap_memory=swap_memory)
File "/Users/p111/anaconda/lib/python3.5/site-packages/tensorflow/python/ops/control_flow_ops.py", line 2636, in while_loop
result = context.BuildLoop(cond, body, loop_vars, shape_invariants)
File "/Users/p111/anaconda/lib/python3.5/site-packages/tensorflow/python/ops/control_flow_ops.py", line 2469, in BuildLoop
pred, body, original_loop_vars, loop_vars, shape_invariants)
File "/Users/p111/anaconda/lib/python3.5/site-packages/tensorflow/python/ops/control_flow_ops.py", line 2419, in _BuildLoop
body_result = body(*packed_vars_for_body)
File "/Users/p111/anaconda/lib/python3.5/site-packages/tensorflow/python/ops/functional_ops.py", line 380, in compute
packed_fn_values = fn(packed_values)
TypeError: () missing 1 required positional argument: 'crop'
Edit: It appears that elems will only accept a single tensor. Which means I would need to somehow combine my two tensors into one, and then unpack it in my function to get the values out. I'm not sure how I would perform that kind of tensor manipulation. I have found the glimpse method already and that does work, however I am wondering if the same can be done with this specific method. Mostly, I'm wondering how you would combine and then split a pair of tensors so it can be used in this method.
I saw this code from here.
elems = (np.array([1, 2, 3]), np.array([-1, 1, -1]))
alternate = map_fn(lambda x: x[0] * x[1], elems, dtype=tf.int64)
# alternate == [-1, 2, -3]
It is possible to use a tuple or a list to pack several elements into one so I tried this.
import tensorflow as tf
from matplotlib import pyplot as plt
import numpy as np
sess = tf.InteractiveSession()
img1 = np.random.random([400, 600, 3])
img2 = np.random.random([400, 600, 3])
img3 = np.random.random([400, 600, 3])
images = np.array([img1, img2, img3])
# images = tf.convert_to_tensor(images) # it can be uncommented.
img1_crop = [100, 100, 100, 100]
img2_crop = [200, 150, 100, 100]
img3_crop = [150, 200, 100, 100]
crop_values = np.array([img1_crop, img2_crop, img3_crop])
# crop_values = tf.convert_to_tensor(crop_values) # it can be uncommented.
def crop_image(img, crop):
return tf.image.crop_to_bounding_box(img,
crop[0],
crop[1],
crop[2],
crop[3])
fn = lambda x: crop_image(x[0], x[1])
elems = (images, crop_values)
cropped_image = tf.map_fn(fn, elems=elems, dtype=tf.float64)
result = sess.run(cropped_image)
print result.shape
plt.imshow(result[0])
plt.show()
It works on my machine with tf version 0.11 and python2. Hope this can help you.
Couple of things :
You do not have a return statement in the crop_image function.
map_fn accepts a single argument.
I strongly advise you to separate the graph definition and the session usage.
--
# Graph def
def crop_image(img, crop):
return tf.image.crop_to_bounding_box(img,
crop[0],
crop[1],
crop[2],
crop[3])
image_placeholder = tf.placeholder(tf.float32, [None, 400, 600, 3])
crop_placeholder = tf.placeholder(dtype=tf.int32)
cropped_image = tf.map_fn(lambda inputs: crop_image(*inputs), elems=[image_placeholder, crop_placeholder], dtype=tf.float32)
# Session
sess = tf.InteractiveSession()
img1 = np.random.random([400, 600, 3])
img2 = np.random.random([400, 600, 3])
img3 = np.random.random([400, 600, 3])
images = [img1, img2, img3]
img1_crop = [100, 100, 100, 100]
img2_crop = [200, 150, 100, 100]
img3_crop = [150, 200, 100, 100]
crop_values = [img1_crop, img2_crop, img3_crop]
sess.run(tf.global_variables_initializer())
result = sess.run(cropped_image, feed_dict={image_placeholder: images, crop_placeholder:crop_values})
plt.imshow(result[0])
plt.show()
tf.map_fn(f, l) runs function f for every tensor in list l. In your case, your function expects 2 arguments, but since you supply a flat list, map_fn() sends them one by one. According to docs, map_fn() supports variable arity, so what you should do is something like this
tf.map_fn(lambda img, crop: crop_image(img, crop),
elems=([image_placeholder, crop_placeholder], ))
so the list you pass to map_fn contains pairs of arguments.
Related
I´m trying to plot a chart and I have some problems to solve, sorry but I´m new in program language.
First one:
How to plot only one chart? I got that example from the internet and when a plot there is two figure for each code and two of them is blank.
The second one:
Is it possible to plot only a positive error bar?
Third one:
Is it possible to plot these two charts side by side in one figure?
import numpy as np
import pandas as pd
import seaborn as sns
import matplotlib.pyplot as plt
Treat1 =pd.DataFrame({'Treatment': 1, 'weight': np.random.randint(low=1, high=100, size=40)})
Treat2 =pd.DataFrame({'Treatment': 2, 'weight': np.random.randint(low=1, high=100, size=40)})
df = pd.concat([Treat1, Treat2])
Treat3 =pd.DataFrame({'Treatment': 1, 'weight': np.random.randint(low=100, high=300, size=40)})
Treat4 =pd.DataFrame({'Treatment': 2, 'weight': np.random.randint(low=100, high=300, size=40)})
df2 = pd.concat([Treat3, Treat4])
sns.set(style="ticks")
fig, ax = plt.subplots()
color_map = dict(pos="indianred", neg="steelblue")
g = sns.catplot(x= "Treatment", y="weight", hue="Treatment", capsize=.07, ci ="sd",
data=df, kind="bar", palette = 'coolwarm', edgecolor="white")
plt.text(-0.22,99, "B")
plt.text(1.18,99, "A")
plt.ylabel('weight, kg')
plt.xticks([-0.2, 1.2], ['Group 1', 'Group 2'])
plt.ylim(0, 100)
color_map = dict(pos="indianred", neg="steelblue")
g = sns.catplot(x= "Treatment", y="weight", hue="Treatment", capsize=.07, ci ="sd",
data=df2, kind="bar", palette = 'coolwarm', edgecolor="white")
plt.text(-0.22,300, "B")
plt.text(1.18,300, "A")
plt.ylabel('weight, kg')
plt.xticks([-0.2, 1.2], ['Group 1', 'Group 2'])
plt.ylim(0, 300)
Thank you so much!
A seaborn catplot is a figure level plot, which creates and occupies a new figure. To have such a plot as a subplot, sns.barplot can be called directly. Supplying an ax tells into which subplot the barplot should go.
The barplot gets a legend, which in this case is superfluous, but it can be removed.
To only have the upper error bar visible, the rectangles of the bars can be plot on top of them. A zorder larger than the zorder of the lines of the errorbar (2) takes care of this.
import numpy as np
import pandas as pd
import seaborn as sns
import matplotlib.pyplot as plt
Treat1 = pd.DataFrame({'Treatment': 1, 'weight': np.random.randint(low=1, high=100, size=40)})
Treat2 = pd.DataFrame({'Treatment': 2, 'weight': np.random.randint(low=1, high=100, size=40)})
df1 = pd.concat([Treat1, Treat2])
Treat3 = pd.DataFrame({'Treatment': 1, 'weight': np.random.randint(low=100, high=300, size=40)})
Treat4 = pd.DataFrame({'Treatment': 2, 'weight': np.random.randint(low=100, high=300, size=40)})
df2 = pd.concat([Treat3, Treat4])
sns.set(style="ticks")
fig, axs = plt.subplots(ncols=2, figsize=(10, 4))
for ax, df, height in zip(axs, [df1, df2], [100, 300]):
color_map = {1: "indianred", 2: "steelblue"}
g = sns.barplot(x="Treatment", y="weight", hue="Treatment", capsize=.07, ci="sd",
data=df, palette=color_map, edgecolor="white", ax=ax)
g.legend_.remove()
for bar in g.patches:
bar.set_zorder(3)
ax.text(-0.2, height * 0.95, "B", ha='center')
ax.text(1.2, height * 0.95, "A", ha='center')
ax.set_ylabel('weight, kg')
ax.set_xticks([-0.2, 1.2])
ax.set_xticklabels(['Group 1', 'Group 2'])
ax.set_ylim(0, height)
plt.tight_layout()
plt.show()
PS: Note that the code can be simplified somewhat if you don't use hue=. This also puts the bars in a more logical position.
fig, axs = plt.subplots(ncols=2, figsize=(10, 4))
for ax, df, height in zip(axs, [df1, df2], [100, 300]):
color_map = {1: "indianred", 2: "steelblue"}
g = sns.barplot(x="Treatment", y="weight", capsize=.07, ci="sd",
data=df, palette=color_map, edgecolor="white", ax=ax)
for bar in g.patches:
bar.set_zorder(3)
ax.text(0, height * 0.97, "B", ha='center', va='top')
ax.text(1, height * 0.97, "A", ha='center', va='top')
ax.set_ylabel('weight, kg')
ax.set_ylim(0, height)
ax.set_xticklabels(['Group 1', 'Group 2'])
plt.tight_layout()
plt.show()
For the prediction I need an image of the shape [null,7,7,256].
const image = tf.reshape(tf.fromPixels(loadedImage).resizeBilinear([?,?]), [null, 7, 7, 256]);
But I don't know how to resize the image to be exactly 7*7*256 big.
Error: Size(37632) must match the product of shape ,7,7,256
Edit: The code for the prediction is:
tf.loadModel(tf.io.browserFiles([uploadJSONInput.files[0], uploadWeightsInput.files[0]])).then(model => {
console.log("model loaded");
return model;
}).then(pretrainedModel => {
return loadImage2('http://localhost/myimg.jpeg', (src) => {
const loadedImage = document.createElement("img");
loadedImage.src = src;
loadedImage.width = "275"
loadedImage.height = "183"
console.log("image loaded");
const image = tf.fromPixels(loadedImage)
resized = tf.image.resizeBilinear(image, [7, 7])
const padded = resized.pad([[0, 0], [0, 0], [126, 127]])
const pretrainedModelPrediction = pretrainedModel.predict(padded);
const modelPrediction = model.predict(pretrainedModelPrediction);
const prediction = modelPrediction.as1D().argMax().dataSync()[0];
console.log(prediction);
});
})
Error:
Error: Error when checking : expected flatten_Flatten1_input to have 4 dimension(s), but got array with shape [7,7,256]
ResizeBilinear will resize the height and the width of the image, meaning that it does not affect the number of channel which is the last dimension of the shape of an image.
If your image has 256 as it last channel, then the following will work
tf.fromPixels(loadedImage).resizeBilinear([7,7])
Reshaping a tensor will only work if both sizes matches.
const image = tf.ones([183, 275, 3 ])
resized = tf.image.resizeBilinear(image, [7, 7])
console.log(resized.pad([[0, 0], [0, 0], [126, 127]]).shape);
An image is generally of shape [h, w, 3].
resize = tf.fromPixels(loadedImage).resizeBilinear([7,7]) // [7, 7, 3]
And then use tf.pad for the last dimension
const image = tf.ones([183, 275, 3 ])
resized = tf.image.resizeBilinear(image, [7, 7])
console.log(resized.pad([[0, 0], [0, 0], [126, 127]]).shape);// [7,7,256]
// reshape the tensor to be a 4d
resized.reshape([1,7,7,256])
Here's how to do it with a Uint8Array
const canvas: any = document.getElementById('canvas')
const context = canvas.getContext('2d')
const imageData: ImageData = context.getImageData(0, 0, canvas.width, canvas.height)
const uint8array = new Uint8Array(imageData.data.buffer)
const rgbaTens3d = tf.tensor3d(uint8array, [canvas.height, canvas.width, 4])
const rgbTens3d= tf.slice3d(rgbaTens3d, [0, 0, 0], [-1, -1, 3]) // strip alpha channel
const smallImg = tf.image.resizeBilinear(rgbTens3d, [192, 192]); // 192,192 is dictated by my model
I try to train a CNN model, 2 classes, which is based on tensorflow to do the image classification.
I have tried much modification about epochs, learning rate, batch size and the CNN size, but nothing works.
about data
86(label: 0) + 63(label: 1) images
shape: (128, 128)
about current parameters
learning_rate = 0.00005(I have tried from 0.00000001 to 0.8...)
batch size = 30(I also have tried from 5 to 130)
epoch = 20
about network
def weight_variable(shape):
initial = tf.truncated_normal(shape, stddev = 0.1, dtype = tf.float32)
return tf.Variable(initial)
def bias_variable(shape):
initial = tf.constant(0.1, shape = shape, dtype = tf.float32)
return tf.Variable(initial)
def conv2d(x, W):
#(input, filter, strides, padding)
#[batch, height, width, in_channels]
return tf.nn.conv2d(x, W, strides=[1, 1, 1, 1], padding='SAME')
def max_pool_2x2(x):
#(value, ksize, strides, padding)
return tf.nn.max_pool(x, ksize=[1, 2, 2, 1], strides=[1, 2, 2, 1], padding='SAME')
def cnn_model():
epochs = 20
batch_size = 30
learning_rate = 0.00005
hidden = 2
cap_c = 86
cap_h = 63
num = cap_c + cap_h
image_size = 128
label_size = 2
print ((num//(batch_size)) * epochs)
train_loss = np.empty((num//(batch_size)) * epochs)
train_acc = np.empty((num//(batch_size)) * epochs)
x = tf.placeholder(tf.float32, shape = [None, image_size, image_size])
y = tf.placeholder(tf.float32, shape = [None, label_size])
weight_balance = tf.constant([0.1])
X_train_ = tf.reshape(x, [-1, image_size, image_size, 1])
#First layer
W_conv1 = weight_variable([5, 5, 1, 4])
b_conv1 = bias_variable([4])
h_conv1 = tf.nn.relu(conv2d(X_train_, W_conv1) + b_conv1)
h_pool1 = max_pool_2x2(h_conv1)
# #Second layer
# W_conv2 = weight_variable([5, 5, 4, 8])
# b_conv2 = bias_variable([8])
#
# h_conv2 = tf.nn.relu(conv2d(h_pool1, W_conv2) + b_conv2)
# h_pool2 = max_pool_2x2(h_conv2)
#
# Third layer
# W_conv3 = weight_variable([5, 5, 8, 16])
# b_conv3 = bias_variable([16])
#
# h_conv3 = tf.nn.relu(conv2d(h_pool2, W_conv3) + b_conv3)
# h_pool3 = max_pool_2x2(h_conv3)
#Full connect layer
W_fc1 = weight_variable([64 * 64 * 4, hidden])
b_fc1 = bias_variable([hidden])
h_pool2_flat = tf.reshape(h_pool1, [-1, 64 * 64 * 4])
h_fc1 = tf.nn.relu(tf.matmul(h_pool2_flat, W_fc1) + b_fc1)
keep_prob = tf.placeholder(tf.float32)
h_fc1_drop = tf.nn.dropout(h_fc1, keep_prob)
#Output_Softmax
W_fc2 = weight_variable([hidden, label_size])
b_fc2 = bias_variable([label_size])
y_conv = tf.nn.softmax(tf.matmul(h_fc1_drop, W_fc2) + b_fc2)
print y_conv.shape
#Train
loss = tf.reduce_mean(tf.nn.weighted_cross_entropy_with_logits(y, y_conv, weight_balance))
optimize = tf.train.AdamOptimizer(learning_rate).minimize(loss)
correct_prediction = tf.equal(tf.argmax(y_conv, 1), tf.argmax(y, 1))
accuracy = tf.reduce_mean(tf.cast(correct_prediction, tf.float32))
about result
The loss is not convergent and also the accuracy.
I don't know if my CNN model is not suitable for my data?
or
The Activate function and loss function of the network is not suitable?
Really thank you
There are couple of problems with the code:
You are applying softmax on the last layer and then calling tf.nn.weighted_cross_entropy_with_logits which in turn applies sigmoid activation, so you are applying activation twice.
For initialisation of the weights, use Xavier or Variance_scaling for faster convergence. Better to use tf.layers API in implementing your model as its default settings follows best practices.
how to build 152-layer resnet and extract the penultimate hidden layer's image feature.
now I got a pre-trained ResNet 152-layer model http://download.tensorflow.org/models/resnet_v1_152_2016_08_28.tar.gz
I just want to use this 152-layer model to extract the image feature, now i want to extract the penultimate hidden layer's image feature
(just as show in the code).
The main question is how to build 152-layer ResNet model ? (i just see that set n = 18 then resnet
is 110 layers.). Or how to build 50-layer ResNet model?
my extract the penultimate hidden layer's image feature code is right?
from __future__ import division, print_function, absolute_import
import tflearn
from PIL import Image
import numpy as np
# Residual blocks
# 32 layers: n=5, 56 layers: n=9, 110 layers: n=18
n = 18
# Data loading
from tflearn.datasets import cifar10
(X, Y), (testX, testY) = cifar10.load_data()
Y = tflearn.data_utils.to_categorical(Y, 10)
testY = tflearn.data_utils.to_categorical(testY, 10)
# Real-time data preprocessing
img_prep = tflearn.ImagePreprocessing()
img_prep.add_featurewise_zero_center(per_channel=True)
# Real-time data augmentation
img_aug = tflearn.ImageAugmentation()
img_aug.add_random_flip_leftright()
img_aug.add_random_crop([32, 32], padding=4)
# Building Residual Network
net = tflearn.input_data(shape=[None, 32, 32, 3],
data_preprocessing=img_prep,
data_augmentation=img_aug)
net = tflearn.conv_2d(net, 16, 3, regularizer='L2', weight_decay=0.0001)
net = tflearn.residual_block(net, n, 16)
net = tflearn.residual_block(net, 1, 32, downsample=True)
net = tflearn.residual_block(net, n-1, 32)
net = tflearn.residual_block(net, 1, 64, downsample=True)
net = tflearn.residual_block(net, n-1, 64)
net = tflearn.residual_block(net, 1, 64, downsample=True)
net = tflearn.residual_block(net, n-1, 64)
net = tflearn.batch_normalization(net)
net = tflearn.activation(net, 'relu')
output_layer = tflearn.global_avg_pool(net)
# Regression
net = tflearn.fully_connected(output_layer, 10, activation='softmax')
mom = tflearn.Momentum(0.1, lr_decay=0.1, decay_step=32000, staircase=True)
net = tflearn.regression(net, optimizer=mom,
loss='categorical_crossentropy')
# Training
model = tflearn.DNN(net, checkpoint_path='resnet_v1_152.ckpt',
max_checkpoints=10, tensorboard_verbose=0,
clip_gradients=0.)
model.fit(X, Y, n_epoch=200, validation_set=(testX, testY),
snapshot_epoch=False, snapshot_step=500,
show_metric=True, batch_size=128, shuffle=True,
run_id='resnet_cifar10')
model.save('./resnet_v1_152.ckpt')
#---------------
# now extract the penultimate hidden layer's image feature
img = Image.open(file_path)
img = img.resize((32, 32), Image.ANTIALIAS)
img = np.asarray(img, dtype="float32")
imgs = np.asarray([img])
model_test = tflearn.DNN(output_layer, session = model.session)
model_test.load('resnet_v1_152.ckpt', weights_only = True)
predict_y = model_test.predict(imgs)
print('layer\'s feature'.format(predict_y))
I'd like to create 2 animations and save them to one file, but get the error:
TypeError: 'FuncAnimation' object is not iterable
Separately each animation runs fine.
Here's a minimal-ish example that demonstrates the problem. The cosmetic stuff like ax.text and ax.axhline may help show what I'm trying to do, but is not essential to the problem. I'm running the code in a Jupyter notebook.
Thanks for any help.
import numpy as np
import matplotlib.pyplot as plt
from matplotlib import animation
xdata = np.array([0.2, 0.4, 0.6, 0.8])
ydata = np.array([0, 0, 0, 0])
fig = plt.figure(figsize=(8,1.25))
ax = fig.add_subplot(111)
ax.set_xlim(0, 1)
ax.set_ylim(-0.5, 0.5)
ax.barh(0, 1, align='center', height=0.7, color='.95', ec='None', zorder=-20)
ax.axhline(0, 0, 1, color='0.25', linewidth=.5)
ax.text(0, -0.45, "0", verticalalignment = "center", horizontalalignment="center", fontsize=10)
ax.text(1, -0.45, "1", verticalalignment = "center", horizontalalignment="center", fontsize=10)
ax.text(-0.01, 0, "0", verticalalignment = "center", horizontalalignment="center", fontsize=10)
ax.text(0.75, -0.45, "3/4", verticalalignment = "center", horizontalalignment="center", fontsize=10)
ax.axvline(0.75, 0.15, 0.85, color='.5', linestyle='dotted', linewidth=0.75)
line, = ax.plot(xdata, ydata, 'ro', markersize=7)
def init0(): # initialization function for first animation
line.set_data(xdata, ydata)
return line,
def init1(): # second animation
ax.set_xlim(0.5, 1)
ax.text(0.5, -0.45, "1/2", verticalalignment = "center", horizontalalignment="center", fontsize=10)
line.set_data(xdata, ydata)
return line,
def animate(i): # update function
ax.figure.canvas.draw()
return line,
# call animators
anim0 = animation.FuncAnimation(fig, animate, init_func=init0, frames=150, interval=20, blit=True)
anim1 = animation.FuncAnimation(fig, animate, init_func=init1, frames=150, interval=20, blit=True)
fig.set_size_inches(16, 2.5, True) # improve resolution in animation
plt.axis('off')
anim0.save('anims.mp4', fps=30, extra_args=['-vcodec', 'libx264'], extra_anim=anim1)
plt.show()