I saved train and validation set as tfrecord file. Inference gives input images and returns logits. loss and accuracy compute loss and accuracy as well. Using this code network is trained well(train set accuracy increases and loss decreases). But accuracy of validation set is almost fixed. By tensorboard, I found that computing accuracy of validation set create new graph that doesn't use the main graph's weights. How can I predict accuracy on validation set simultaneously?
def run_training():
train_images,train_labels = read_and_decode_tfrecord_train(train_data_path)
val_images,val_labels = read_and_decode_tfrecord_validation(validation_data_path)
train_images = tf.cast(train_images,tf.float32)/255.
val_images = tf.cast(train_images,tf.float32)/255.
batch_Xs,batch_Ys=tf.train.shuffle_batch([train_images,train_labels],batch_size=500,capacity=500,min_after_dequeue=100)
batch_xs,batch_ys=tf.train.shuffle_batch([val_images,val_labels],batch_size=500,capacity=500,min_after_dequeue=100)
logits=inference(batch_Xs,1)
total_loss = loss(logits,batch_Ys)
train_op = training(total_loss,learning_rate=LEARNING_RATE)
accuracy = evaluation(logits,batch_Ys)
val_logits=inference(batch_xs,1)
val_accuracy = evaluation(val_logits,batch_ys)
saver = tf.train.Saver(tf.all_variables(), max_to_keep=4,)
sess = tf.Session()
init = tf.initialize_all_variables()
sess.run(init)
tf.train.start_queue_runners(sess=sess)
for i in range(NUM_ITER):
_,loss_value,acc=sess.run([train_op,total_loss,accuracy])
if i%10==0:
val_acc,testing_summary_accuracy=sess.run([val_accuracy,testing_summary])
print 'Iteration:',i, ' Loss:',loss_value,' Train Accuracy:',acc,' Validation Accuracy:',v
Related
I'm working in an application where there is a lot of reporting of external events. One of the metrics that is used often is the event rate as a function of time. For example, measuring the sample-rate of some external asynchronous sensor.
Currently the way I'm calculating the frequency of events like this is to keep a queue of event timestamps. When the event occurs we push a current timestamp onto the queue, then pop until the oldest timestamp is less than a predefined age. Then, the event frequency is proportional to the size of the queue. In pseudo-code the method usually looks something like this:
def on_event():
var now = current_time()
time_queue.push(now)
while((now - time_queue.front()) > QUEUE_DEPTH_SECONDS):
time_queue.pop()
frequency = time_queue.size() / QUEUE_DEPTH_SECONDS
Now this approach is obviously not optimal:
Memory requirement and computation time is proportional to event rate.
The queue duration has to be manually adjusted based on the expected data rate in order to tune low-frequency performance vs memory requirements.
The response-time of the frequency measurement is also dependent on the queue duration. Longer durations lower the response time of the calculation.
The frequency is only updated when a new event occurs. If the events stop occurring, then the frequency measurement will remain at the value calculated when the last event was received.
I'm curious if there are any alternative algorithms that can be used to calculate the rate of an event, and what trade-offs they have in relation to computational complexity, space requirements, response-time, etc.
https://en.wikipedia.org/wiki/Exponential_smoothing is very efficient and uses only a small and bounded amount of memory. You could try exponential smoothing of the inter-arrival times. When retrieving the smoothed inter-arrival time you could look at the time to the last event, and mix that in if it is larger than the smoothed inter-arrival time.
This is different enough that I would in fact start by collecting a sample of timestamps in current use, so that I could use it to test the result of this or other schemes off-line.
One alternative is a local timer that fires at a constant rate (e.g. once per second):
When an external event occurs, it increments a counter.
When the local timer fires, the counter value is added to a queue, and the count is reset to zero.
Here's how the method compares to yours:
Memory requirement and computation time is independent of the external event rate. It is determined by the local timer rate, which you control.
The queue size depends on how much averaging you want to do. A queue size of 1 (i.e. no queue) with a timer rate of once per second results in a raw events-per-second reading, with no averaging. The larger the queue, the more averaging you get.
The response time is determined by the amount of averaging desired. More averaging results in slower response times.
The frequency is updated at the rate that the local timer fires, regardless of whether external events occur.
I've implemented something similar to calculate the rate/concentration of particles in an air stream. They come at random events (probably poisson distributed) and I want to know the average rate in particles per time. My approach is as follows (taken from the docstring of my code):
Event timestamps are placed in a buffer of a fixed maximum size.
Whenever a concentration estimate is requested, all timestamps up to
a set maximum age are filtered out. If the remaining number of
recent timestamps is below some threshold, the reported
concentration is zero. Otherwise, the concentration is calculated
as the number of remaining events divided by the time difference
between the oldest timestamp in the buffer and now.
I'll attach the Python implementation below for reference. It is a part of a much larger project, so I had to slightly modify it to get rid of references to external code:
particle_concentration_estimator_params.py
#!/usr/bin/env python3
"""This module implements a definition of parameters for the particle
concentration estimator.
"""
__author__ = "bup"
__email__ = "bup#swisens.ch"
__copyright__ = "Copyright 2021, Swisens AG"
__license__ = "GNU General Public License Version 3 (GPLv3)"
__all__ = ['ParticleConcentrationEstimatorParams']
import dataclasses
#dataclasses.dataclass
class ParticleConcentrationEstimatorParams:
"""This provides storage for the parameters used for the particle
concentration estimator.
"""
timestamp_buffer_max_size: int
"""Maximum size of the buffer that is used to keep track of event
timestamps. The size of this buffer mainly affects the filtering
of the reported data.
Unit: - (count)
"""
timestamp_buffer_max_age: float
"""Maximum age of events in the timestamp buffer which are
considered for the concentration calculation. This value is a
tradeoff between a smooth filtered value and the dynamic response
to a changed concentration.
Unit: s
"""
min_number_of_timestamps: int
"""Minimum number of timestamps to use for the concentration
estimation. If less timestamps are available, the concentration is
reported as zero.
Unit: - (count)
"""
particle_concentration_estimator.py
#!/usr/bin/env python3
"""This module implements the particle concentration estimation.
"""
__author__ = "bup"
__email__ = "bup#swisens.ch"
__copyright__ = "Copyright 2021, Swisens AG"
__license__ = "GNU General Public License Version 3 (GPLv3)"
__all__ = ['ParticleConcentrationEstimator']
import logging
import time
from typing import Optional
import numpy as np
from .particle_concentration_estimator_params import ParticleConcentrationEstimatorParams
logger = logging.getLogger(__name__)
class ParticleConcentrationEstimator:
"""An object of this class implements the Poleno particle
concentration estimator. Particle concentration is basically just
a number of particles per time unit. But since the particle events
arrive irregularly, there are various ways to filter the result, to
avoid too much noise especially when the concentration is low. This
class implements the following approach:
Event timestamps are placed in a buffer of a fixed maximum size.
Whenever a concentration estimate is requested, all timestamps up to
a set maximum age are filtered out. If the remaining number of
recent timestamps is below some threshold, the reported
concentration is zero. Otherwise, the concentration is calculated
as the number of remaining events divided by the time difference
between the oldest timestamp in the buffer and now.
"""
def __init__(self, params: ParticleConcentrationEstimatorParams):
"""Initializes the object with no events.
Args:
est_params: Initialized PolenoParams object which includes
information describing the estimator's behaviour.
"""
self.params = params
"""Stored params for the object."""
n_rows = self.params.timestamp_buffer_max_size
self._rb = np.full((n_rows, 2), -1e12)
self._rb_wp = 0
self._num_timestamps = 0
self._concentration_value = 0.0
self._concentration_value_no_mult = 0.0
def tick(self, now: float) -> float:
"""Recalculates the current concentration value.
Args:
now: Current timestamp to use to filter out old entries
in the buffer.
Returns:
The updated concentration value, which is also returned
using the concentration attribute.
"""
min_ts = now - self.params.timestamp_buffer_max_age
min_num = self.params.min_number_of_timestamps
used_rows = self._rb[:, 0] >= min_ts
filt_ts = self._rb[used_rows]
num_ts = round(np.sum(filt_ts[:, 1]))
self._num_timestamps = num_ts
num_ts_no_mult = round(np.sum(used_rows))
if num_ts < min_num:
self._concentration_value = 0.0
self._concentration_value_no_mult = 0.0
else:
t_diff = now - np.min(filt_ts[:, 0])
if t_diff >= 1e-3:
# Do not change the reported value if all events in the
# buffer have the same timestamp.
self._concentration_value = num_ts / t_diff
self._concentration_value_no_mult = num_ts_no_mult / t_diff
return self._concentration_value
def got_timestamp(self,
ts: Optional[float] = None,
multiplier: float = 1.0) -> None:
"""Passes in the most recent event timestamp. Timestamps need
not be ordered.
Calling this method does not immediately update the
concentration value, this is deferred to the tick() method.
Args:
ts: Event timestamp to use. If None, the current time is
used.
multiplier: Optional multiplier, which makes ts to be
counted that many times.
"""
if ts is None:
ts = time.time()
self._rb[self._rb_wp] = (ts, float(multiplier))
self._rb_wp = (self._rb_wp + 1) % self._rb.shape[0]
self._num_timestamps += round(multiplier)
#property
def concentration(self) -> float:
"""The calculated concentration value.
Unit: 1/s
"""
return self._concentration_value
#property
def concentration_no_mult(self) -> float:
"""The calculated concentration value without taking into
account the timestamp multipliers, i.e. as if all timestamps
were given with the default multiplier of 1.
Unit: 1/s
"""
return self._concentration_value_no_mult
#property
def num_timestamps(self) -> int:
"""Returns the number of timestamps which currently are in
the internal buffer.
"""
return self._num_timestamps
I trained a CNN in Tensorflow and it tested with 92% accuracy. I saved it as a typical ckpt file.
session = tf.Session(config=tf.ConfigProto(log_device_placement=True))
session.run(tf.global_variables_initializer())
<TRAINING ETC>
saver.save(session, save_path_name)
In a different file, I want to run inference, so I called the meta-graph as explained in the documentation:
face_recognition_session = tf.Session()
saver = tf.train.import_meta_graph(<PATH TO META FILE>, clear_devices=True)
saver.restore(face_recognition_session, <PATH TO CKPT FILE>)
graph = tf.get_default_graph()
x = graph.get_tensor_by_name('input_variable_00:0')
y = graph.get_tensor_by_name('output_variable_00:0')
When performing inference or testing it anew, the accuracy drops to 3%.
Am I overlooking anything?
You are assigning the wrong method to saver. From the TF Guide you can see that you want to init session and then upload through tensorflow.train.Saver().
tf.reset_default_graph()
# Create some variables.
x = tf.get_variable("input_variable_00:0", [x_shape])
y = tf.get_variable("output_variable_00:0", [y_shape])
saver = tf.train.Saver()
# Use the saver object normally after that.
with tf.Session() as sess:
# Initialize v1 since the saver will not.
saver.restore(sess, <PATH TO CKPT FILE>)
print("x : %s" % x.eval())
print("y : %s" % y.eval())
I would also recommend looking into freezing and exporting your graphs as a GraphDef if you want to have consistent inference results.
I am working for a project of semantic segmentation of retinal blood vessels with Tensorflow with the MobileUNet model and I have received this error:
InvalidArgumentError (see above for traceback): logits and labels must
be broadcastable: logits_size=[82944,2] labels_size=[90000,2]
[[Node: softmax_cross_entropy_with_logits_sg = SoftmaxCrossEntropyWithLogits[T=DT_FLOAT,
_device="/job:localhost/replica:0/task:0/device:CPU:0"](softmax_cross_entropy_with_logits_sg/Reshape,
softmax_cross_entropy_with_logits_sg/Reshape_1)]]
Here my code is as follows:
network=network = build_mobile_unet(net_input, preset_model = args.model, num_classes=num_classes)
net_input = tf.placeholder(tf.float32,shape=[None,None,None,3])
net_output = tf.placeholder(tf.float32,shape=[None,None,None,num_classes])
losses = tf.nn.softmax_cross_entropy_with_logits(logits=network, labels=net_output)
cost = tf.reduce_mean(losses)
opt = tf.train.AdamOptimizer(0.001).minimize(cost)
init = tf.initialize_all_variables()
_,current=sess.run([opt,cost],feed_dict={net_input:input_image_batch, net_output:segmented_image_batch})
The input image is 300x300, and is in the RGB colour-space. The output is a binary image with the same size as input.
Can someone help me?
We answered this problem which is also related to architecture Understand this in following link
Input to reshape is a tensor with 37632 values, but the requested shape has 150528
Let us know if you face any issue
The same problem occured with me.This comes when we use label_size more than number of the classes in our dataset.
In the last fully connected layer(Dense) I had used 46 but in my dataset there was only 38 classes.So when I used 38 instead of 46,problem solved.
I want to training a image classifier using inception model.
Now, I have a dishes called chicken rice.
Suppose i want to create rice and chicken meat class.
So can i design output ground true probability as [0.5,0.5,0,0,0...]?
In other words, If the target image contains two classes' content, what should I do to make it reasonable?
Do somebody has tried this?
I have tried to train the image separately, and google did this, too.
keycnt = 0
imagcnt = 0
TestNumber_byclass = np.zeros([keycount],np.int32)
for key in TestKeys:
TestNumber_byclass[keycnt] = len(json_data_test[key])
for imagedata in json_data_test[key]:
imgdata = tf_resize_images(imagdir + imagedata + '.jpg')
imgdata = np.array(imgdata, dtype = np.uint8)
# make image center at 0 in the range of (-1,1]
#imgdata = (imgdata - mean - 128) / 128
h5f = h5py.File(h5filedir_test + str(imagcnt) + ".h5", "w")
h5f.create_dataset('image', data=imgdata)
h5f.create_dataset('label', data=keycnt)
h5f.create_dataset('name' , data=key)
h5f.close()
imagcnt = imagcnt + 1
keycnt =keycnt +1
message = '\r[%d/%d] progress...' % (keycnt,keycount)
sys.stdout.write(message)
sys.stdout.flush()
Many thanks.
What you're trying to do is a multiclass classification, where M out of N classes will be predicted. This is usually done by setting the flag to 1 if the object appears in the image and setting it to 0 if that's not the case.
The really important piece of information is that the last activation function needs to be a sigmoid instead of a softmax. That way you decouple the confidence for each class from the other classes and the sum will be between 0 and N.
I'm writing a piece of code that has to transform from an RGB image to an rgb normalized space. I've got it working with a for format but it runs too slow and I need to evaluate lots of images. I'm trying to vectorize the full function in order to faster it. What I have for the moment is the following:
R = im(:,:,1);
G = im(:,:,2);
B = im(:,:,3);
r=reshape(R,[],1);
g=reshape(G,[],1);
b=reshape(B,[],1);
clear R G B;
VNormalizedRed = r(:)/(r(:)+g(:)+b(:));
VNormalizedGreen = g(:)/(r(:)+g(:)+b(:));
VNormalizedBlue = b(:)/(r(:)+g(:)+b(:));
NormalizedRed = reshape(VNormalizedRed,height,width);
NormalizedGreen = reshape(VNormalizedGreen,height,width);
NormalizedBlue = reshape(VNormalizedBlue,height,width);
The main problem is that when it arrives at VNormalizedRed = r(:)/(r(:)+g(:)+b(:)); it displays an out of memory error (wich is really strange because i just have freed three vectors of the same size). Were is the error? (solved)
Its possible to do the same process in a more efficiently way?
Edit:
After using Martin sugestions I found the reshape function was not necessary, being able to do the same with a simple code:
R = im(:,:,1);
G = im(:,:,2);
B = im(:,:,3);
NormalizedRed = R(:,:)./sqrt(R(:,:).^2+G(:,:).^2+B(:,:).^2);
NormalizedGreen = G(:,:)./sqrt(R(:,:).^2+G(:,:).^2+B(:,:).^2);
NormalizedBlue = B(:,:)./sqrt(R(:,:).^2+G(:,:).^2+B(:,:).^2);
norm(:,:,1) = NormalizedRed(:,:);
norm(:,:,2) = NormalizedGreen(:,:);
norm(:,:,3) = NormalizedBlue(:,:);
I believe you want
VNormalizedRed = r(:)./(r(:)+g(:)+b(:));
Note the dot in front of the /, which specifies an element-by-element divide. Without the dot, you're solving a system of equations -- which is likely not what you want to do. This probably also explains why you're seeing the high memory consumption.
Your entire first code can be rewritten in one vectorized line:
im_normalized = bsxfun(#rdivide, im, sum(im,3,'native'));
Your second slightly modified version as:
im_normalized = bsxfun(#rdivide, im, sqrt(sum(im.^2,3,'native')));
BTW, you should be aware of the data type used for the image, otherwise one can get unexpected results (due to integer division for example). Therefore I would convert the image to double before performing the normalization calculations:
im = im2double(im);