Given a simple table like this:
+---------+---------+---------+
| Column1 | Column2 | Column3 |
+---------+---------+---------+
| A | J | Q |
| A | K | S |
| B | M | R |
| B | N | S |
| B | J | Q |
| C | K | R |
| D | J | R |
| D | J | Q |
| E | L | Q |
+---------+---------+---------+
Is it possible to determine whether there is a subset of N rows in this table, such that for each column, all N values are distinct?
For example, with N = 3, the answer would be yes
+---------+---------+---------+
| Column1 | Column2 | Column3 |
+---------+---------+---------+
| A | J | Q |
| B | N | S |
| C | K | R |
+---------+---------+---------+
Is there a simple algorithm to conclude on such a question?
Is there a simple algorithm to conclude on such a question?
The answer to that is strictly "yes"; you can do a brute-force search over all (R choose K) subsets of K rows, where R is the number of rows in the whole table. That algorithm is quite simple, and could be implemented in a few lines in a language like Python.
But I don't think that is the answer you're looking for; I think you want to know if there is a simple algorithm that takes less than exponential time. The answer to that is almost certainly not; the problem is NP-hard, by reduction from the maximum independent set problem, so there is no known algorithm which gives correct answers in polynomial time, and it's very likely that no such algorithm is possible.
The reduction is as follows: given a graph, construct a table with one row for each vertex. For each edge in the graph, add one column to the table; in this column write the same letter in the two rows the edge joins, and then write distinct other letters in each of the remaining rows for that column. The resulting table has V rows and E columns, so its size is polynomial in the size of the original graph, and it is constructed in polynomial time.
Then, any set of K rows which have distinct values in each column gives K vertices in the original graph not connected by any edges. This means if you can answer yes/no to whether there is such a set of K rows, in polynomial time, then you can also answer the decision form of the maximum independent set problem in polynomial time. The latter is NP-complete, so therefore your problem is NP-hard.
Simple solution would be simply search (backtracking).
But every tool / library solving CSP (Constraint satisfaction problem) can find a solution.
Since you are explicitly asking for an algorithm to solve this:
If I understand the problem correctly (you are using N two times here,the first time for rows and the second time for values, what is a bit confusing), you want to find N rows with ALL DIFFERENT values in the given table.
I would start like this:
Create a data structure for a lookup if you already found a value (e.g. a hashmap)
Create a data structure to store your result rows which fit the match-condition (all values different)
Iterate the input table rows, until you reached your desired subset size or you reached the end of the table
Start with the first row
While iterating a row, check for each value, if it is in your structure (created in 1.), if yes -> abort, else add the value to the lookup-structure. When all row-values are checked, this row is okay and can be added to your result set.
Iterate next row, if existing
But as pointed out in the comment, this algorithm is a greedy one, that will not always find the possible solution
Related
Background
In an algorithm called finite element method, a continuous region is discretized into repeated sections with consistent geometry, over which linked equations are formed based on the assumption of continuity between them.
In this case, I have chosen to divide a shape up into an arbitrary grid, and am now trying to connect the elements' values together as I iterate through the elements. Here is an example of the kind of grid I am talking about:
Indices
There are a bunch of related indices:
The element index (teal numbers), linear, row-major, range 0..ROWS*2.
The node index (brown numbers), linear, row-major, range 0..ROWS*COLS.
The local element vertex index (lavender numbers), counter-clockwise by element, range 0..2.
The coordinates of the actual point in space (stored with the element's struct, as well as the grid's struct)
Problem
In order to get to the next step in the algorithm, I need to iterate over each node and compute some sums of values indexed by local element indices and store them in another matrix. If, for example, I'm on node 8, my element lookup/access function is generically V(el_index, start_vertex, end_vertex), and my matrix of outputs is S(start_node, end_node):
S(8,8) = V(el_14, vert_1, vert_1) + V(el_15, vert_1, vert_1) + V(el_04, vert_0, vert_0) + V(el_03, vert_0, vert_0) + V(el_2, vert_2, vert_2) + V(el_13, vert_2, vert_2)
S(8,9) = V(el_15, vert_1, vert_2) + V(el_4, vert_0, vert_2)
and so on, for all of the connections (teal lines) from node 8. (The connections are symmetric, so once I compute S(7,8), I don't need to compute S(8,7).)
The problem is, the grid (and therefore everything else) is parameterized at runtime, so which node index + adjacency direction corresponds to which element index is dynamically determined. I need to tell the program, "Get me the element indices where the node index of vert_x is my current node index." That's the instruction that tells the program which element to access in V().
Is there a way I can relate these indices in a simple and transparent manner in Rust?
Attempts
I tried computing some simple arithmetic functions modulo the row stride of the node matrix, but the result is messy and hard to debug, as well as requiring verbose bounds checking.
I tried creating three HashMaps keyed by the different vertices of each triangular element, holding the values at each vertex, but the problem is that adjacent triangles share vertex numbers as well as spatial coordinates.
I considered keying a HashMap with multiple keys, but the Rust docs didn't say anything about a HashMap with multiple keys.
I think this is a possible workaround, but I hope there's something better:
Nest one HashMap in another as per this question, with the node index as the first key, the element index as the second key/first value, and the element itself as the second value.
So, the iteration can look like this:
Iterate through grid node index N
Get all elements with N as the first key
Note that the vertex indices and (relative) element indices have the following patterns, depending on where you number from:
Node index numbering beginning from matrix top-left (usual in this programming domain):
| Element (index ordinal) | Vertex |
|-------------------------|--------|
| 0 (min) | 1 |
| 1 | 2 |
| 2 | 1 |
| 3 | 2 |
| 4 | 0 |
| 5 | 0 |
Node index numbering beginning as picture, from lower-left:
| Element (index ordinal) | Vertex |
|-------------------------|--------|
| 0 (min) | 2 |
| 1 | 0 |
| 2 | 0 |
| 3 | 2 |
| 4 | 1 |
| 5 | 1 |
Since you can hardcode the relative order of elements like this, it might be better to use a HashMap<usize, Vec<Element>>, or even HashMap<usize, [Element; 6]>.
This method brings up the question of how to relate node index to element indices dynamically. How do you know which elements to insert into that HashMap? One way to accomplish this is to record the nodes the element vertices correspond to in the element struct as well, in the same order.
At that point, you can compute a list like adjacent_elements as you iterate through the matrix, and use the above patterns to figure out what to access (sadly, with bounds checking).
I'm trying to implement a feature with following requirement but having hard time come up with a algorithm.
My data contains a positive integer value and a date. The numbers of data vary from 100 ~ 10,000.
-------------------------
id | value | date
-------------------------
1 | 10 | 2015-01-01
2 | 10 | 2015-01-02
3 | 20 | 2015-01-02
....................
960 | 30 | 2015-09-10
961 | 15 | 2015-09-10
And a specified target value, says 5,000.
I would like to find a combination of the data, so their sum of the values equal to target, and they contains older data as much as possible. (The target number must match, it is okay to have a combination without using oldest data first)
Can anyone give me a direction how I can implement it ?
One approach based on Subset-Sum pseudo-polynomial solution could be :
First, sort the entries such that the oldest one is last, and the newest one is first. Then, generate the DP matrix based on the formulas:
D(i,0) = true
D(0,x) = false x!= 0
D(i,x) = D(i-1,x) OR D(i-1, x-value[i])
This matrix is of size (n+1) * (target+1).
Next, generate a solution by greedily choosing (from last to first) to take the element if it's possible:
t = target
i = n
sol = [] //empty list
while (t != 0):
if D(i-1,t-value[i] == true):
sol.append(i) //item i in the solution
t = t - value[i]
i = i-1 //either case
This guarantees:
Values of sol sums to the target
The oldest value which is in any feasible solution will be in sol.
Apparently the problem can be interpreted as the knapsack problem. Note that this problem is NP-hard. It can be solved to optimality by dynamic programming and admits an FPTAS.
The problem can be modelled in the following way. The item profits and weights are the item's value (which means that the problem can be simplified to the https://en.wikipedia.org/wiki/Subset_sum_problem). Let C denote the specified target value. Group the items by value; for value v, let keep only the floor(C/v) oldest items, where floor denotes rounding down. After the knapsack solver has generated a solution, replace the at most floor(C/v) items in the solution of value v (for each value v) with the oldest ones selected before.
Consider the following data.
Groundtruth | Dataset1 | Dataset2 | Dataset3
Datapoints|Time | Datapoints|Time | Datapoints|Time | Datapoints|Time
A |0 | a |0 | a |0 | a |0
B |10 | b |5 | b |5 | b |13
C |15 | c |12 | c |12 | c |21
D |25 | d |22 | d |14 | d |30
E |30 | e |30 | e |17 |
| | f |27 |
| | g |30 |
Visualized like this (as in number of - between each identifier):
Time ->
Groundtruth: A|----------|B|-----|C|----------|D|-----|E
Dataset1: a|-----|b|-------|c|----------|d|--------|e
Dataset2: a|-----|b|-------|c|--|d|---|e|----------|f|---|g
Dataset3: a|-------------|b|--------|c|---------|d
My goal is to compare the datasets with the groundtruth. I want to create a function that generates a similarity measurement between one of the datasets and the groundtruth in order to evaluate how good my segmentation algorithm is. Obviously I would like the segmentation algorithm to consist of equal number of datapoints(segments) as the groundtruth but as illustrated with the datasets this is not a guarantee, neither is the number of datapoints known ahead of time.
I've already created a Jacard Index to generate a basic evaluation score. But I am now looking into an evaluation method that punish the abundance/absence of datapoints as well as limit the distance to a correct datapoint. That is, b doesn't have to match B, it just has to be close to a correct datapoint.
I've tried to look into a dynamic programming method where I introduced a penalty for removing or adding a datapoint as well as a distance penalty to move to the closest datapoint. I'm struggling though, due to:
1. I need to limit each datapoint to one correct datapoint
2. Figure out which datapoint to delete if needed
3. General lack of understanding in how to implement DP algorithms
Anyone have ideas how to do this? If dynamic programming is the way to go, I'd love some link recommendation as well as some pointers in how to go about it.
Basically, you can modify the DP for Levenshtein edit distance to compute distances for your problem. The Levenshtein DP amounts to finding shortest paths in an acyclic directed graph that looks like this
*-*-*-*-*
|\|\|\|\|
*-*-*-*-*
|\|\|\|\|
*-*-*-*-*
where the arcs are oriented left-to-right and top-to-bottom. The DAG has rows numbered 0 to m and columns numbered 0 to n, where m is the length of the first sequence, and n is the length of the second. Lists of instructions for changing the first sequence into the second correspond one-to-one (cost and all) to paths from the upper left to the lower right. The arc from (i, j) to (i + 1, j) corresponds to the instruction of deleting the ith element from the first sequence. The arc from (i, j) to (i, j + 1) corresponds to the instruction of adding the jth element from the second sequence. The arc from (i, j) corresponds to modifying the ith element of the first sequence to become the jth element of the second sequence.
All you have to do to get a quadratic-time algorithm for your problem is to define the cost of (i) adding a datapoint (ii) deleting a datapoint (iii) modifying a datapoint to become another datapoint and then compute shortest paths on the DAG in one of the ways described by Wikipedia.
(As an aside, this algorithm assumes that it is never profitable to make modifications that "cross over" one another. Under a fairly mild assumption about the modification costs, this assumption is superfluous. If you're interested in more details, see this answer of mine: Approximate matching of two lists of events (with duration) .)
I have to arrange a tour. There are N peoples who want to attend the tour. But some of them are enemies with each other. (enemy of enemy may or may not be your enemy. If A is enemy of B, B is also enemy of A)
If I accommodate a person in my tour, I can not accommodate his enemy.
Now I want to accommodate maximum possible number of persons in the tour. How can I find this number?
ex: If there are 5 tourists, and let's call them A-E. A is enemy with B and D, B is enemy of E, and C is enemy of D.
A B C D E
+---+---+---+---+---+
A | - | X | | X | |
+---+---+---+---+---+
B | X | - | | | X |
+---+---+---+---+---+
C | | | - | X | |
+---+---+---+---+---+
D | X | | X | - | |
+---+---+---+---+---+
E | | X | | | - |
+---+---+---+---+---+
In this case following trips are possible: empty, A, B, C, D, E, AC, AE, BC, BD, ACE, CE, DE etc. Out of these, best tour is ACE as it accommodates 3 tourist and hence the answer 3.
My approach:
I tried looping and trying combinations with bitmaps, but they are
very slow.
I am currently using DFS but trying to find even a better
method.
I tried to work by creating friendship graph and prepare some
spanning tree. But it doesn't work as A can travel with B and B can
travel with C does not guarantee A and travel with C.
I tried by creating enemity graph and finding some weak links
but ended up clueless.
I would be thankful, if somebody can give me a hint or point to a good resource to solve this problem.
Create a graph from the tourists :
each node is a tourist
if two tourists are enemies draw an edge between them
find the maximum independent set
There are very well detailed algorithms for finding maximum independent set in this paper:
Algorithms for Maximum independent Sets
And a parallel approach has been provided in this one:Lecture Notes on a Parallel Algorithm for Generating a Maximal Independent Set
And this is a java implementation.
Firstly create an array of vectors-
vector<int> enemy[N];
and an array curr[N] telling if he is going or not.
If person x is going, curr[x]=1 else curr[x]=0
now call the given below function as f(0,0).
The answer will be in a global variable named "maxx", which is 0 initially.
void f(int i,int c)
{
// i is the index of current person we are seeing
// c is the total count of people going till now
if(i==N)
{
if(c>maxx)maxx=c;
}
else
{
int j,flag=1;
int sz;
sz=enemy[i].size();
if(sz==0)
{
//if he has no enemy, he can obviously go
curr[i]=1;
f(i+1,c+1);
}
else
{
for(j=0;j<sz;j++)
if(curr[enemy[i][j]]==1)
{
//if his enemy is already going, he cannot
flag=0;
break;
}
if(flag)
{
//if none of his enemies till now are going, he can
curr[i]=1;
f(i+1,c+1);
}
curr[i]=0;
f(i+1,c);//we will also see what if we dont take him
}
}
Try making an antagonism graph and then find a largest independent set. (Warning: this problem is NP)
What is the point of the one-comparison-per-iteration binary search? And can you explain how it works?
There are two reasons to binary search with one comparison per iteration. The
less important is performance. Detecting an exact match early using two
comparisons per iteration saves an average one iteration of the loop, whereas
(assuming comparisons involve significant work) binary searching with one
comparison per iteration almost halves the work done per iteration.
Binary searching an array of integers, it probably makes little difference
either way. Even with a fairly expensive comparison, asymptotically the
performance is the same, and the half-rather-than-minus-one probably isn't worth
pursuing in most cases. Besides, expensive comparisons are often coded as functions that return negative, zero or positive for <, == or >, so you can get both comparisons for pretty much the price of one anyway.
The important reason to do binary searches with one comparison per iteration is
because you can get more useful results than just some-equal-match. The main
searches you can do are...
First key > goal
First key >= goal
First key == goal
Last key < goal
Last key <= goal
Last key == goal
These all reduce to the same basic algorithm. Understanding this well enough
that you can code all the variants easily isn't that difficult, but I've not
really seen a good explanation - only pseudocode and mathematical proofs. This
is my attempt at an explanation.
There are games where the idea is to get as close as possible to a target
without overshooting. Change that to "undershooting", and that's what "Find
First >" does. Consider the ranges at some stage during the search...
| lower bound | goal | upper bound
+-----------------+-------------------------+--------------
| Illegal | better worse |
+-----------------+-------------------------+--------------
The range between the current upper and lower bound still need to be searched.
Our goal is (normally) in there somewhere, but we don't yet know where. The
interesting point about items above the upper bound is that they are legal in
the sense that they are greater than the goal. We can say that the item just
above the current upper bound is our best-so-far solution. We can even say this
at the very start, even though there is probably no item at that position - in a
sense, if there is no valid in-range solution, the best solution that hasn't
been disproved is just past the upper bound.
At each iteration, we pick an item to compare between the upper and lower bound.
For binary search, that's a rounded half-way item. For binary tree search, it's
dictated by the structure of the tree. The principle is the same either way.
As we are searching for an item greater-than our goal, we compare the test item
using Item [testpos] > goal. If the result is false, we have overshot (or
undershot) our goal, so we keep our existing best-so-far solution, and adjust
our lower bound upwards. If the result is true, we have found a new best-so-far
solution, so we adjust the upper bound down to reflect that.
Either way, we never want to compare that test item again, so we adjust our
bound to eliminate (only just) the test item from the range to search. Being
careless with this usually results in infinite loops.
Normally, half-open ranges are used - an inclusive lower bound and an exclusive
upper bound. Using this system, the item at the upper bound index is not in the
search range (at least not now), but it is the best-so-far solution. When you
move the lower bound up, you move it to testpos+1 (to exclude the item you just
tested from the range). When you move the upper bound down, you move it to
testpos (the upper bound is exclusive anyway).
if (item[testpos] > goal)
{
// new best-so-far
upperbound = testpos;
}
else
{
lowerbound = testpos + 1;
}
When the range between the lower and upper bounds is empty (using half-open,
when both have the same index), your result is your most recent best-so-far
solution, just above your upper bound (ie at the upper bound index for
half-open).
So the full algorithm is...
while (upperbound > lowerbound)
{
testpos = lowerbound + ((upperbound-lowerbound) / 2);
if (item[testpos] > goal)
{
// new best-so-far
upperbound = testpos;
}
else
{
lowerbound = testpos + 1;
}
}
To change from first key > goal to first key >= goal, you literally switch
the comparison operator in the if line. The relative operator and goal could be replaced by a single parameter - a predicate function that returns true if (and only if) its parameter is on the greater-than side of the goal.
That gives you "first >" and "first >=". To get "first ==", use "first >=" and
add an equality check after the loop exits.
For "last <" etc, the principle is the same as above, but the range is
reflected. This just means you swap over the bound-adjustments (but not the
comment) as well as changing the operator. But before doing that, consider the following...
a > b == !(a <= b)
a >= b == !(a < b)
Also...
position (last key < goal) = position (first key >= goal) - 1
position (last key <= goal) = position (first key > goal ) - 1
When we move our bounds during the search, both sides are being moved towards the goal until they meet at the goal. And there is a special item just below the lower bound, just as there is just above the upper bound...
while (upperbound > lowerbound)
{
testpos = lowerbound + ((upperbound-lowerbound) / 2);
if (item[testpos] > goal)
{
// new best-so-far for first key > goal at [upperbound]
upperbound = testpos;
}
else
{
// new best-so-far for last key <= goal at [lowerbound - 1]
lowerbound = testpos + 1;
}
}
So in a way, we have two complementary searches running at once. When the upperbound and lowerbound meet, we have a useful search result on each side of that single boundary.
For all cases, there's the chance that that an original "imaginary" out-of-bounds
best-so-far position was your final result (there was no match in the
search range). This needs to be checked before doing a final == check for the
first == and last == cases. It might be useful behaviour, as well - e.g. if
you're searching for the position to insert your goal item, adding it after the
end of your existing items is the right thing to do if all the existing items
are smaller than your goal item.
A couple of notes on the selection of the testpos...
testpos = lowerbound + ((upperbound-lowerbound) / 2);
First off, this will never overflow, unlike the more obvious ((lowerbound +
upperbound)/2). It also works with pointers as well as integer
indexes.
Second, the division is assumed to round down. Rounding down for non-negatives
is OK (all you can be sure of in C) as the difference is always non-negative
anyway.
This is one aspect that may need care if you use non-half-open
ranges, though - make sure the test position is inside the search range, and not just outside (on one of the already-found best-so-far positions).
Finally, in a binary tree search, the moving of bounds is implicit and the
choice of testpos is built into the structure of the tree (which may be
unbalanced), yet the same principles apply for what the search is doing. In this
case, we choose our child node to shrink the implicit ranges. For first match
cases, either we've found a new smaller best match (go to the lower child in hopes of finding an even smaller and better one) or we've overshot (go to the higher child in hopes of recovering). Again, the four main cases can be handled by switching the comparison operator.
BTW - there are more possible operators to use for that template parameter. Consider an array sorted by year then month. Maybe you want to find the first item for a particular year. To do this, write a comparison function that compares the year and ignores the month - the goal compares as equal if the year is equal, but the goal value may be a different type to the key that doesn't even have a month value to compare. I think of this as a "partial key comparison", and plug that into your binary search template and you get what I think of as a "partial key search".
EDIT The paragraph below used to say "31 Dec 1999 to be equal to 1 Feb 2000". That wouldn't work unless the whole range in-between was also considered equal. The point is that all three parts of the begin- and end-of-range dates differ, so you're not deal with a "partial" key, but the keys considered equivalent for the search must form a contiguous block in the container, which will normally imply a contiguous block in the ordered set of possible keys.
It's not strictly just "partial" keys, either. Your custom comparison might consider 31 Dec 1999 to be equal to 1 Jan 2000, yet all other dates different. The point is the custom comparison must agree with the original key about the ordering, but it might not be so picky about considering all different values different - it can treat a range of keys as an "equivalence class".
An extra note about bounds that I really should have included before, but I may not have thought about it this way at the time.
One way of thinking about bounds is that they aren't item indexes at all. A bound is the boundary line between two items, so you can number the boundary lines as easily as you can number the items...
| | | | | | | | |
| +-+ | +-+ | +-+ | +-+ | +-+ | +-+ | +-+ | +-+ |
| |0| | |1| | |2| | |3| | |4| | |5| | |6| | |7| |
| +-+ | +-+ | +-+ | +-+ | +-+ | +-+ | +-+ | +-+ |
| | | | | | | | |
0 1 2 3 4 5 6 7 8
Obviously the numbering of bounds is related to the numbering of the items. As long as you number your bounds left-to-right and the same way you number your items (in this case starting from zero) the result is effectively the same as the common half-open convention.
It would be possible to select a middle bound to bisect the range precisely into two, but that's not what a binary search does. For binary search, you select an item to test - not a bound. That item will be tested in this iteration and must never be tested again, so it's excluded from both subranges.
| | | | | | | | |
| +-+ | +-+ | +-+ | +-+ | +-+ | +-+ | +-+ | +-+ |
| |0| | |1| | |2| | |3| | |4| | |5| | |6| | |7| |
| +-+ | +-+ | +-+ | +-+ | +-+ | +-+ | +-+ | +-+ |
| | | | | | | | |
0 1 2 3 4 5 6 7 8
^
|<-------------------|------------->|
|
|<--------------->| | |<--------->|
low range i hi range
So the testpos and testpos+1 in the algorithm are the two cases of translating the item index into the bound index. Of course if the two bounds are equal, there's no items in that range to choose so the loop cannot continue, and the only possible result is that one bound value.
The ranges shown above are just the ranges still to be searched - the gap we intend to close between the proven-lower and proven-higher ranges.
In this model, the binary search is searching for the boundary between two ordered kinds of values - those classed as "lower" and those classed as "higher". The predicate test classifies one item. There is no "equal" class - equal-to-key values are part of the higher class (for x[i] >= key) or the lower class (for x[i] > key).