Keras doesn't train using fit_generator() - performance

I am using Keras 2.0.4 (TensorFlow backend) for an image classification task.
I am trying to train my own network (without any pretrained parameters).
As my data is huge I cannot load all into memory.
For this reason I use ImageDataGenerator(), flow_from_directory() and fit_generator().
Creating ImageDataGenerator object:
train_datagen = ImageDataGenerator(preprocessing_function = my_preprocessing_function) # only preprocessing; no augmentation; static data set
my_preprocessing_function rescales images to domain [0,255] and centers data by mean reduction (similar to preprocessing of VGG16 or VGG19)
Use method flow_from_directory() from the ImageDataGenerator object:
train_generator = train_datagen.flow_from_directory(
path/to/training/directory/with/five/subfolders,
target_size=(img_width, img_height),
batch_size=64,
classes = ['class1', 'class2', 'class3', 'class4', 'class5'],
shuffle = True,
seed = 1337,
class_mode='categorical')
(The same is done in order to create a validation_generator.)
After defining and compiling the model (loss function: categorical crossentropy, optimizer: Adam), I train the model using fit_generator():
model.fit_generator(
train_generator,
steps_per_epoch=total_amount_of_train_samples/batch_size,
epochs=400,
validation_data=validation_generator,
validation_steps=total_amount_of_validation_samples/batch_size)
Problem:
There is no error message, but training doesn't perform well.
After 400 epochs, accuracy still oscillates around 20% (which is as good as randomly choosing one of those classes). Indeed, the classifier always predicts 'class1'.
The same holds true after only one epoch of training. Why is this the case although I am initializing random weights?
What is wrong? What am I missing?
U S E D M O D E L
x = Input(shape=input_shape)
# Block 1
x = Conv2D(16, (3, 3), activation='relu', padding='same', name='block1_conv1')(x)
x = Conv2D(16, (5, 5), activation='relu', padding='same', name='block1_conv2')(x)
x = MaxPooling2D((2, 2), strides=(2, 2), name='block1_pool')(x)
# Block 2
x = Conv2D(64, (3, 3), activation='relu', padding='same', name='block2_conv1')(x)
x = Conv2D(64, (5, 5), activation='relu', padding='same', name='block2_conv2')(x)
x = MaxPooling2D((2, 2), strides=(2, 2), name='block2_pool')(x)
# Block 3
x = Conv2D(16, (1, 1), activation='relu', padding='same', name='block3_conv1')(x)
# Block 4
x = Conv2D(256, (3, 3), activation='relu', padding='valid', name='block4_conv1')(x)
x = Conv2D(256, (5, 5), activation='relu', padding='valid', name='block4_conv2')(x)
x = MaxPooling2D((2, 2), strides=(2, 2), name='block4_pool')(x)
# Block 5
x = Conv2D(1024, (3, 3), activation='relu', padding='valid', name='block5_conv1')(x)
x = MaxPooling2D((2, 2), strides=(2, 2), name='block5_pool')(x)
# topping
x = Dense(1024, activation='relu', name='fc1')(x)
x = Dense(1024, activation='relu', name='fc2')(x)
predictions = Dense(5, activation='softmax', name='predictions')(x)
E D I T
terminal output

Related

Filling in Julia matrix with nested for loops

I have two arrays and an empty matrix, I need to perform a function such that the resulting matrix includes every combination of the two arrays.
Unfortunately I cannot run the arrays separately as they are both optional parameters for the function. I thought that the best way to do this was through nested loops but now I am unsure...
I've tried multiplying one of the matrices so that it includes the necessary duplicates, but I struggled with that as the real data is somewhat larger.
I've tried many versions of these nested loops.
a = [ 1 2 3 ]
b = [ 4 5 6 7 ]
ab = zeros(3,4)
for i = 1:length(a)
for j = 1:length(b)
ab[??] = function(x = a[??], y = b[??])
end
end
ab = [1x4 1x5 1x6 1x7, 2x4 2x5 2x6 2x7, 3x4 3x5 3x6 3x7]
Your problem can be solved by broadcasting:
julia> f(x, y) = (x,y) # trivial example
f (generic function with 1 method)
julia> f.([1 2 3]', [4 5 6 7])
3×4 Array{Tuple{Int64,Int64},2}:
(1, 4) (1, 5) (1, 6) (1, 7)
(2, 4) (2, 5) (2, 6) (2, 7)
(3, 4) (3, 5) (3, 6) (3, 7)
The prime in a' transposes a to make the shapes work out correctly.
But note that a = [ 1 2 3 ] constructs a 1×3 Array{Int64,2}, which is a matrix. For a vector (what you probably call "array"), use commas: a = [ 1, 2, 3 ] etc. If you have your data in that form, you have to transpose the other way round:
julia> f.([1,2,3], [4,5,6,7]')
3×4 Array{Tuple{Int64,Int64},2}:
(1, 4) (1, 5) (1, 6) (1, 7)
(2, 4) (2, 5) (2, 6) (2, 7)
(3, 4) (3, 5) (3, 6) (3, 7)
BTW, this is called an "outer product" (for f = *), or a generalization of it. And if f is an operator ∘, you can use dotted infix broadcasting: a' ∘. b.
Isn't that just
a'.*b
?
Oh, now I have to write some more characters to get past the minimum acceptable answer length but I don't really have anything to add, I hope the code is self-explanatory.
Also a list comprehension:
julia> a = [1,2,3];
julia> b = [4,5,6,7];
julia> ab = [(x,y) for x in a, y in b]
3×4 Array{Tuple{Int64,Int64},2}:
(1, 4) (1, 5) (1, 6) (1, 7)
(2, 4) (2, 5) (2, 6) (2, 7)
(3, 4) (3, 5) (3, 6) (3, 7)

What is prolog query for finding possible values

This is my prolog program:
par(0,0).
par(0,1).
par(0,2).
par(1,0).
par(1,2).
par(1,1).
par(2,1).
par(2,0).
par(2,2).
gp(X,Y):- par(X,Z),par(Z,Y).
ggp(X,Y) :- par(X,Z), par(Z,W), par(W,Y).
What query must be used to obtain he possible set of tuples to satisfy the rules gp and ggp
I tried using gp (X,Y) but doesn't give me the tuples.
Also gp and ggp are not related. The tuples that satisfy ggp does not have to necessarily satisfy the rule gp
You only need to "pack" the X and Y together into a tuple, like:
tuple_gp((X,Y)) :-
gp(X,Y).
tuple_ggp((X,Y)) :-
ggp(X,Y).
This then can answer with:
?- tuple_gp(T).
T = (0, 0) ;
T = (0, 1) ;
T = (0, 2) ;
T = (0, 0) ;
T = (0, 2) ;
T = (0, 1) ;
T = (0, 1) ;
T = (0, 0) ;
T = (0, 2) ;
T = (1, 0) ;
T = (1, 1) ;
T = (1, 2) ;
T = (1, 1) ;
T = (1, 0) ;
T = (1, 2) ;
T = (1, 0) ;
T = (1, 2) ;
T = (1, 1) ;
T = (2, 0) ;
T = (2, 2) ;
T = (2, 1) ;
T = (2, 0) ;
T = (2, 1) ;
T = (2, 2) ;
T = (2, 1) ;
T = (2, 0) ;
T = (2, 2).
If you want to generate a list of all possible tuples, you can use findall/3:
?- findall((X,Y),gp(X,Y),L).
L = [ (0, 0), (0, 1), (0, 2), (0, 0), (0, 2), (0, 1), (0, 1), (0, 0), (..., ...)|...].
?- findall((X,Y),ggp(X,Y),L).
L = [ (0, 0), (0, 1), (0, 2), (0, 0), (0, 2), (0, 1), (0, 1), (0, 0), (..., ...)|...].
If you want only to obtain unique tuples, you can use setof/3:
?- setof((X,Y),X^Y^gp(X,Y),S).
S = [ (0, 0), (0, 1), (0, 2), (1, 0), (1, 1), (1, 2), (2, 0), (2, 1), (..., ...)].

Sort by key then value which will then be grouped up...pyspark

So I'm trying to sort data in this format...
[((0, 4), 3), ((4, 0), 3), ((1, 6), 1), ((3, 2), 3), ((0, 5), 1)...
Ascending by key and then descending by value. I'm able to achieve this via...
test = test.sortBy(lambda x: (x[0], -x[1]))
which would give me based on shortened version above...
[((0, 4), 3), ((0, 5), 1), ((1, 6), 1), ((3, 2), 3), ((4, 0), 3)...
The problem I'm having is that after the sorting I no longer want the value but do need to retain the sort after grouping the data. So...
test = test.map(lambda x: (x[0][0],x[0][1]))
Gives me...
[(0, 4), (0, 5), (1, 6), (3, 2), (4, 0)...
Which is still in the order I need it but I need the elements to be grouped up by key. I then use this command...
test = test.groupByKey().map(lambda x: (x[0], list(x[1])))
But in the process I lose the sorting. Is there any way retain?
I managed to retain the order by changing the format of the tuple...
test = test.map(lambda x: (x[0][0],(x[0][1],x[1]))
test = test.groupByKey().map(lambda x: (x[0], sorted(list(x[1]), key=lambda x: (x[0],-x[1]))))
[(0, [(4, 3), (5, 1)] ...
which leaves me with the value (2nd element in the tuple) that I want to get rid of but took care of that too...
test = test.map(lambda x: (x[0], [e[0] for e in x[1]]))
Feels a bit hacky but not sure how else it could be done.

quick sort list of tuple with python

I am trying to do this in its operation algorithm quicksort to sort though the elements of a list of tuples. Or if I have a list of this type [(0,1), (1,1), (2,1), (3,3), (4,2), (5,1), (6,4 )] I want to sort it in function of the second element of each tuple and obtain [(6,4), (3,3), (4,2), (0,1), (1,1), (2,1 ), (5,1)]. I have tried using the following algorithm:
def partition(array, begin, end, cmp):
pivot=array[end][1]
ii=begin
for jj in xrange(begin, end):
if cmp(array[jj][1], pivot):
array[ii], array[jj] = array[jj], array[ii]
ii+=1
array[ii], array[end] = pivot, array[ii]
return ii
enter code hedef sort(array, cmp=lambda x, y: x > y, begin=0, end=None):
if end is None: end = len(array)
if begin < end:
i = partition(array, begin, end-1, cmp)
sort(array, cmp, i+1, end)
sort(array, cmp, begin, i)
The problem is that the result is this: [4, (3, 3), (4, 2), 1, 1, 1, (5, 1)]. What do I have to change to get the correct result ??
Complex sorting patterns in Python are painless. Python's sorting algorithm is state of the art, one of the fastest available in real-world cases. No algorithm design needed.
>>> from operator import itemgetter
>>> l = [(0,1), (1,1), (2,1), (3,3), (4,2), (5,1), (6,4 )]
>>> l.sort(key=itemgetter(1), reverse=True)
>>> l
[(6, 4), (3, 3), (4, 2), (0, 1), (1, 1), (2, 1), (5, 1)]
Above, itemgetter returns a function that returns the second element of its argument. Thus the key argument to sort is a function that returns the item on which to sort the list.
Python's sort is stable, so the ordering of elements with equal keys (in this case, the second item of each tuple) is determined by the original order.
Unfortunately the answer from #wkschwartz only works due to the peculiar start ordering of the terms. If the tuple (5, 1) is moved to the beginning of the list then it gives a different answer.
The following (first) method works in that it gives the same result for any initial ordering of the items in the initial list.
Python 3.4.2 |Continuum Analytics, Inc.| (default, Oct 22 2014, 11:51:45) [MSC v
.1600 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> l = [(0,1), (1,1), (2,1), (3,3), (4,2), (5,1), (6,4 )]
>>> sorted(l, key=lambda x: (-x[1], x[0]))
[(6, 4), (3, 3), (4, 2), (0, 1), (1, 1), (2, 1), (5, 1)]
>>> from operator import itemgetter
>>> sorted(l, key=itemgetter(1), reverse=True)
[(6, 4), (3, 3), (4, 2), (0, 1), (1, 1), (2, 1), (5, 1)]
>>> # but note:
>>> l2 = [(5,1), (1,1), (2,1), (3,3), (4,2), (0,1), (6,4 )]
>>> # Swapped first and sixth elements
>>> sorted(l2, key=itemgetter(1), reverse=True)
[(6, 4), (3, 3), (4, 2), (5, 1), (1, 1), (2, 1), (0, 1)]
>>> sorted(l2, key=lambda x: (-x[1], x[0]))
[(6, 4), (3, 3), (4, 2), (0, 1), (1, 1), (2, 1), (5, 1)]
>>>

Mapping sort indexes

I encountered and solved this problem as part of a larger algorithm, but my solution seems inelegant and I would appreciate any insights.
I have a list of pairs which can be viewed as points on a Cartesian plane. I need to generate three lists: the sorted x values, the sorted y values, and a list which maps an index in the sorted x values with the index in the sorted y values corresponding to the y value with which it was originally paired.
A concrete example might help explain. Given the following list of points:
((3, 7), (15, 4), (7, 11), (5, 0), (4, 7), (9, 12))
The sorted list of x values would be (3, 4, 5, 7, 9, 15), and the sorted list of y values would be (0, 4, 7, 7, 11, 12).
Assuming a zero based indexing scheme, the list that maps the x list index to the index of its paired y list index would be (2, 3, 0, 4, 5, 1).
For example the value 7 appears as index 3 in the x list. The value in the mapping list at index 3 is 4, and the value at index 4 in the y list is 11, corresponding to the original pairing (7, 11).
What is the simplest way of generating this mapping list?
Here's a simple O(nlog n) method:
Sort the pairs by their x value: ((3, 7), (4, 7), (5, 0), (7, 11), (9, 12), (15, 4))
Produce a list of pairs in which the first component is the y value from the same position in the previous list and the second increases from 0: ((7, 0), (7, 1), (0, 2), (11, 3), (12, 4), (4, 5))
Sort this list by its first component (y value): ((0, 2), (4, 5), (7, 0), (7, 1), (11, 3), (12, 4))
Iterate through this list. For the ith such pair (y, k), set yFor[k] = i. yFor[] is your list (well, array) mapping indices in the sorted x list to indices in the sorted y list.
Create the sorted x list simply by removing the 2nd element from the list produced in step 1.
Create the sorted y list by doing the same with the list produced in step 3.
I propose the following.
Generate the unsorted x and y lists.
xs = [3, 15, 7, 5, 4, 9 ]
ys = [7, 4, 11, 0, 7, 12]
Transform each element into a tuple - the first of the pair being the coordinate, the second being the original index.
xs = [(3, 0), (15, 1), ( 7, 2), (5, 3), (4, 4), ( 9, 5)]
ys = [(7, 0), ( 4, 1), (11, 2), (0, 3), (7, 4), (12, 5)]
Sort both lists.
xs = [(3, 0), (4, 4), (5, 3), (7, 2), ( 9, 5), (15, 1)]
ys = [(0, 3), (4, 1), (7, 0), (7, 4), (11, 2), (12, 5)]
Create an array, y_positions. The nth element of the array contains the current index of the y element that was originally at index n.
Create an empty index_list.
For each element of xs, get the original_index, the second pair of the tuple.
Use y_positions to retrieve the current index of the y element with the given original_index. Add the current index to index_list.
Finally, remove the index values from xs and ys.
Here's a sample Python implementation.
points = ((3, 7), (15, 4), (7, 11), (5, 0), (4, 7), (9, 12))
#generate unsorted lists
xs, ys = zip(*points)
#pair each element with its index
xs = zip(xs, range(len(xs)))
ys = zip(ys, range(len(xs)))
#sort
xs.sort()
ys.sort()
#generate the y positions list.
y_positions = [None] * len(ys)
for i in range(len(ys)):
original_index = ys[i][1]
y_positions[original_index] = i
#generate `index_list`
index_list = []
for x, original_index in xs:
index_list.append(y_positions[original_index])
#remove tuples from x and y lists
xs = zip(*xs)[0]
ys = zip(*ys)[0]
print "xs:", xs
print "ys:", ys
print "index list:", index_list
Output:
xs: (3, 4, 5, 7, 9, 15)
ys: (0, 4, 7, 7, 11, 12)
index list: [2, 3, 0, 4, 5, 1]
Generation of y_positions and index_list is O(n) time, so the complexity of the algorithm as a whole is dominated by the sorting step.
Thank you for the answers. For what it's worth, the solution I had was pretty similar to those outlined, but as j_random_hacker pointed out, there's no need for a map. It just struck me that this little problem seems more complicated than it appears at first glance and I was wondering if I was missing something obvious. I've rehashed my solution into Python for comparison.
points = ((3, 7), (15, 4), (7, 11), (5, 0), (4, 7), (9, 12))
N = len(points)
# Separate the points into their x and y components, tag the values with
# their index into the points list.
# Sort both resulting (value, tag) lists and then unzip them into lists of
# sorted x and y values and the tag information.
xs, s = zip(*sorted(zip([x for (x, y) in points], range(N))))
ys, r = zip(*sorted(zip([y for (x, y) in points], range(N))))
# Generate the mapping list.
t = N * [0]
for i in range(N):
t[r[i]] = i
index_list = [t[j] for j in s]
print "xs:", xs
print "ys:", ys
print "index_list:", index_list
Output:
xs: (3, 4, 5, 7, 9, 15)
ys: (0, 4, 7, 7, 11, 12)
index_list: [2, 3, 0, 4, 5, 1]
I've just understood what j_random_hacker meant by removing a level of indirection by sorting the points in x initially. That allows things to be tidied up nicely. Thanks.
points = ((3, 7), (15, 4), (7, 11), (5, 0), (4, 7), (9, 12))
N = len(points)
ordered_by_x = sorted(points)
ordered_by_y = sorted(zip([y for (x, y) in ordered_by_x], range(N)))
index_list = N * [0]
for i, (y, k) in enumerate(ordered_by_y):
index_list[k] = i
xs = [x for (x, y) in ordered_by_x]
ys = [y for (y, k) in ordered_by_y]
print "xs:", xs
print "ys:", ys
print "index_list:", index_list

Resources