Can you please tell me how to build an autoencoder using CNN and pooling layers, with a single matrix(4,4) with integer numbers?
e.g,
input data = array([[ 4, 3, 8, 6], [ 1, 1, 2, 2], [24, 18, 32, 24], [ 6, 6, 8, 8]])
autoencoder(data)
output data= array([[ 4, 3, 8, 6], [ 1, 1, 2, 2], [24, 18, 32, 24], [ 6, 6, 8, 8]])
Explanation:
https://medium.com/machine-learning-researcher/auto-encoder-d942a29c9807
The article you quoted is already well detailed, all you need to do is replaced linear units by convolution and max pool in the encoding part and deconvolution and upsampling in the decoding part.
Here is in example using Keras of a convolutional autoencoder.
But if you're new to Deep Learning I would not advise you to start here. Autoencoder are tricky, the expected output you described above could easily be obtained by setting:
def autoencoder(x):
return x
Which is not something you want.
People commonly design "undercomplete" autoencoders, with internal layer of smaller dimension to induce some compression. But "overcomplete" autoencoders with larger internal dimension are also valid architecture combined with regularization.
Related
I am working on grouping of elements for large dataset in least amount of time.
Below is small dataset example of the same:
Input:
pairs = [[1,2],[8,10],[4,8],[3,4],[2,3],[5,6],[6,7],[9,4],[11,12],[13,10],[16,1],[78,79],[61,3],[93,94]]
How to derive output:
If any of the element is present in another pair then both pair can be grouped. This way traverse all pairs and create different groups.
Output: -
[{1, 2, 3, 4, 8, 9, 10, 13, 16, 61}, {5, 6, 7}, {11, 12}, {78, 79}, {93, 94}]
Question:
What is the best way to solve this problem?
e.g.: graph database(neo4j) or any specific readily available algorithm.
any other alternatives which is not listed in example solution(neo4j, algorithm) above?
Below output is expected and solution should consume less time with large dataset.
Expected Output: -
[{1, 2, 3, 4, 8, 9, 10, 13, 16, 61}, {5, 6, 7}, {11, 12}, {78, 79}, {93, 94}]
I was curious if there is a name for encoding a list of numbers as the deltas between adjacent values. For example, given the following list:
[5, 7, -2, 8, 10, 5]
the "delta list' would be:
[2, -9, 10, 2, -5]
Perhaps a variant would be:
[5, 2, -9, 10, 2, -5]
if the first number of the original lst is to be included for a frame of reference.
Is there a specific name for either of these encodings, whether the name describes either the resulting data encoding or just the process of arriving at the given output? I'm aware of delta encodings, but I was wondering if there was a more specific term.
There is a famous card game in Germany called "Doppelkopf".
Usually, you play "Doppelkopf" with 4 players, but you can also play it with a table of 5 players, where one player is just watching.
(Where everyone "has the cards" once in a round, meaning everyone has the right to play the first card once every round.)
Every year, my family organizes a "Doppelkopf" tournament with 3 rounds (r).
Depending on the availabilty of my relatives, every year the number of participants varies.
Expecting a minimun of participant of 16 people, the number (n) in this experiment can rise up unlimited (as does the number of rounds r).
Naturally, my relatives do not want to be paired with someone twice, since they want to exchange gossip most efficiently!
There we have:
n - Participants
r- Rounds
t_total = n // 4 # Total Tables (round down of n)
t_5 = n % 4 # Tables of 5s
t_4 = t_total - t_5 # Tables of 4s
pos_pair = n * (n - 1) / 2 # possible pairs (n over 2)
nec_pair = (t_5 * 10 + t_4 * 6) * r # necessary pairs
I was instructed with the aim to minimize the encounters (if possible to set encounters == 1 for everyone)!
Since, I do not want to solve the problem for P{n={16, ..., 32}, r=3} (which I did for some cases), but to solve it with any given P{n∈N, r∈N} , there is a discrepancy between my abilities and the requirements for a solution!
Therefore, I would like to ask the community to help me with this problem, to solve it for any given P{n∈N, r∈N}!
And also to prove, if this problem is not solvable for any P{n∈N, r∈N}, which is given "if pos_pair < nec_pair".
Here are two solutions for P{n=20, r=3}:
which very much solves my "Doppelkopf" tournament problem:
('Best result was ', [[[1, 2, 3, 4], [5, 6, 7, 8], [9, 10, 11, 12], [13, 14, 15, 16], [17, 18, 19, 20]], [[16, 12, 8, 18], [13, 1, 5, 9], [15, 4, 17, 6], [2, 19, 7, 10], [3, 11, 20, 14]], [[14, 9, 17, 7], [13, 20, 8, 2], [5, 4, 12, 19], [6, 16, 11, 1], [15, 18, 10, 3]]])
('Best result was ', [[[1, 2, 3, 4], [5, 6, 7, 8], [9, 10, 11, 12], [13, 14, 15, 16], [17, 18, 19, 20]], [[19, 11, 13, 3], [2, 15, 9, 8], [1, 16, 18, 6], [14, 7, 17, 10], [4, 12, 20, 5]], [[17, 8, 3, 12], [20, 9, 16, 7], [15, 11, 6, 4], [2, 13, 10, 18], [1, 19, 14, 5]]])
But in order to solve this problem with an arbitrary n and r I have come to no conclusion.
In my opinion, there are three ways to go about this problem in a computational solution or approximation.
First, you can iterate about rounds, and assign every player to a
table without having collision, remembering pairs and appeareances
in total (not to exeed total rounds)
Secondly, you can iterate about tables, which seems to be helpful with participants, that are a multiple of 2 (see for P{n=16, r=5}
https://matheplanet.com/default3.html?call=viewtopic.php?topic=85206&ref=https%3A%2F%2Fwww.google.com%2F)
also remeber pairs and appearances, but mainly follow a certain
patters as described in the link, which I somehow can not scale to
other numbers!!
There is somehow a mathemathical way to descibe this procedure and conclude a solution
Even though, this is more of a mathematical question (and I don't know where to ask those questions), I am interested in the algorithmic solution!
Bin packing problem is to find the minimal number of bins of size v, which can contain all objects of size [s_1, s_2, s_3, ..., s_n]
I'm searching for a simple, non-trivial instance of the bin-packing problem.
A simple instance is an instance which can be solved with no more than 5 bins.
A non-trivial instance is an instance, which can't be solved by the best-fit-decreasing heuristic algorithm, but can be solved with complete search.
For example, the instance v = 20, objects = [15, 7, 14, 3, 14, 7, 9] is simple, but not non-trivial, because complete search proves that the minimal number of bins is 5:
[[15, 3], [7, 7], [14], [14], [9]]
however, best-fit heuristic also produces a 5-bin packing:
[[15], [14], [14], [9, 7, 3], [7]]
Does a simple, non-trivial instance of bin packing exist?
Indeed, such instance exists, namely:
v = 20, objects = [11, 7, 7, 6, 5, 3, 1]
Best-fit-decreasing heuristic gives:
[[11, 7], [7, 6, 5, 1], [3]]
Optimal packing is:
[[11, 6, 3], [7, 7, 5, 1]]
We are trying to create auto-correlated random values which will be used as timeseries.
We have no existing data we refer to and just want to create the vector from scratch.
On the one hand we need of course a random process with distribution and its SD.
On the other hand the autocorrelation influencing the random process has to be described. The values of the vector are autocorrelated with decreasing strengh over several timelags.
e.g. lag1 has 0.5, lag2 0.3, lag1 0.1 etc.
So in the end the vector should look something that:
2, 4, 7, 11, 10 , 8 , 5, 4, 2, -1, 2, 5, 9, 12, 13, 10, 8, 4, 3, 1, -2, -5
and so on.