Related
Requirement: Must be done in-place.
For example:
Given matrix
1, 2, 3
4, 5, 6
7, 8, 9
Should replace by the average of its sum of 3*3 neighbor cells and its own:
(1+2+4+5)/4, (2+1+3+4+5+6)/6 , (3+2+6+5)/4
(1+2+5+4+7+8)/6, (1+2+3+4+5+6+7+8+9)/9, (2+3+5+6+8+9)/6
(4+5+7+8)/4, (4+5+6+7+8+9)/6, (5+6+8+9)/4
which is:
All floating number convert to int
3, 3.5(3), 4 3, 3, 4
4.5(4), 5, 5.5(5) => 4, 5, 5
6, 6.5(6), 7 6, 6, 7
I tried to just iterate over the matrix and update each cell, but I found this will affect the future calculation:
Say I update the original 1 to 3, but when I when I tried to update the original 2, the original 1 becomes 3 now.
Copying the original matrix for calculating average is a workaround but it's a bad idea, Could we achieve that without using that much space?
In most cases, you should just create a copy of the original matrix and use that for calculating the averages. Unless creating a copy of the matrix would use more memory than you have available, the overhead should be negligible.
If you have a really large matrix, you could use a "rolling" backup (in lack of a better term). Let's say you update the cells row-by-row and you are currently in row n. You don't need a backup of row n-2, as those cells are not relevant any more, and neither of row n+1, because those are still the original values. So you can just keep a backup of the previous and the current row. Whenever you advance to the next row, discard the backup of the previous row, move the backup of the current row to previous, and create a backup of the new current row.
Some pseudo-code (not taking any edge-cases into account):
previous = [] # or whatever works for the first row
for i in len(matrix):
current = copy(matrix[i])
for k in len(matrix[i]):
matrix[i][k] = previous[k-1] + ... + current[k] + ... matrix[i+1][k+1] / 9
previous = current
(You might also keep a backup of the next row, just so you can use only the backup rows for all the values instead of having to differentiate.)
You must have some kind of cache for the result data so you can keep reference to the original data. I don't think there is a way around it.
If the data set is large, you could optimize by using a smaller data buffer (like looking through a keyhole) and 'scrolling' the input matrix as you update it. In your case, you could use a buffer as small as 3x3.
It is a compromise between speed and space though. The smaller your buffer, the worse the performance will be.
To visualize the problem, starting from the top-left (0,0) of the dataset:
(result values are rounded down for simplicity)
First step: update first 4 cells (prime the buffer)
// Data Set // Data Viewport // Result Set
01,02,03,04,05 01,02,03 04,04,??
06,07,08,09,10 06,07,08 06,07,??
11,12,13,14,15 11,12,13 ??,??,??
16,17,18,19,20
21,22,23,24,25
then for each iteration..
( new values indicated with [xx] )
++ update first column in Data Set from Result Set
// Data Set // Data Viewport // Result Set
[04],02,03,04,05 01,02,03 04,04,??
[06],07,08,09,10 06,07,08 06,07,??
11 ,12,13,14,15 11,12,13 ??,??,??
16 ,17,18,19,20
21 ,22,23,24,25
++ shift Data Viewport and Result Set right 1 column
// Data Set // Data Viewport // Result Set
[04],02,03,04,05 02,03,04 04,[03],??
[06],07,08,09,10 07,08,09 07,[08],??
11 ,12,13,14,15 12,13,14 ??, ?? ,??
16 ,17,18,19,20
21 ,22,23,24,25
++ update middle column of Result Set
// Data Set // Data Viewport // Result Set
[04],02,03,04,05 02,03,04 04,[05],??
[06],07,08,09,10 07,08,09 07,[08],??
11 ,12,13,14,15 12,13,14 ??, ?? ,??
16 ,17,18,19,20
21 ,22,23,24,25
At the following iteration, the data state would be:
// Data Set // Data Viewport // Result Set
04,[04],03,04,05 03,04,05 05,[06],??
06,[07],08,09,10 08,09,10 08,[09],??
11, 12 ,13,14,15 13,14,15 ??, ?? ,??
16, 17 ,18,19,20
21, 22 ,23,24,25
.. etc
Don't forget to handle the other edge cases.
*The Data Viewport representation is just for visualization. In code, the actual viewport would be the result buffer.
I have an app that accepts integers at a variable rate every .25 to 2 seconds.
I'd like to output the data in a smoothed format for 3, 5 or 7 seconds depending on user input.
If the data always came in at the same rate, let's say every .25 seconds, then this would be easy. The variable rate is what confuses me.
Data might come in like this:
Time - Data
0.25 - 100
0.50 - 102
1.00 - 110
1.25 - 108
2.25 - 107
2.50 - 102
ect...
I'd like to display a 3 second rolling average every .25 seconds on my display.
The simplest form of doing this is to put each item into an array with a time stamp.
array.push([0.25, 100])
array.push([0.50, 102])
array.push([1.00, 110])
array.push([1.25, 108])
ect...
Then every .25 seconds I would read through the array, back to front, until I got to a time that was less than now() - rollingAverageTime. I would sum that and display it. I would then .Shift() the beginning of the array.
That seems not very efficient though. I was wondering if someone had a better way to do this.
Why don't you save the timestamp of the starting value and then accumulate the values and the number of samples until you get a timestamp that is >= startingTime + rollingAverageTime and then divide the accumulator by the number of samples taken?
EDIT:
If you want to preserve the number of samples, you can do this way:
Take the accumulator, and for each input value sum it and store the value and the timestamp in a shift register; at every cycle, you have to compare the latest sample's timestamp with the oldest timestamp in the shift register plus the smoothing time; if it's equal or more, subtract the oldest saved value from the accumulator, delete that entry from the shift register and output the accumulator, divided by the smoothing time. If you iterate you obtain a rolling average with (i think) the least amount of computation for each cycle:
a sum (to increment the accumulator)
a sum and a subtraction (to compare the timestamp)
a subtraction (from the accumulator)
a division (to calculate the average, done in a smart way can be a shift right)
For a total of about 4 algebric sums and a division (or shift)
EDIT:
For taking into account the time from the last sample as a weighting factor, you can divide the value for the ratio between this time and the averaging time, and you obtain an already weighted average, without having to divide the accumulator.
I added this part because it doesn't add computational load, so you can implement quite easy if you want to.
The answer from clabacchio has the basics right, but perhaps you need a bit more sophisticated answer.
Calculating the average:
0.25 - 100
0.50 - 102
1.00 - 110
In the above subset of the data what is the answer you want? You could use the mean of these numbers or you could do it in a weighted fashion. You could convert the data into:
0.50 - 0.25 = 0.25 ---- (100+102)/2 = 101
1.00 - 0.50 = 0.50 ---- (102+110)/2 = 106
Then you can take the weighted average of these values, weight being the time difference, and value being the average value.
The final answer = (0.25*101 + 0.5*106)/(0.25+0.5) = whatever the value is.
Now coming to "moving" averages:
You can either use previous k values or previous k seconds worth of data. In both cases you can keep two sums: weighted sum and sum of weights.
So... the worst case scenario is 4 readings per second over 7 seconds = 28 values in your array to process. That will be done in nanoseconds anyway, so not worth optimizing IMHO.
I'm trying to make a randomizer that will use the Monte Carlo Hit or Miss Simulation.
I have a Key-Value pair that represents the ID and the probability value:
ID - Value
2 - 0.37
1 - 0.35
4 - 0.14
3 - 0.12
When you add all of those values, you will get a total of 1.0.
You can imagine those values as the total area of a "slice" on the "wheel" (EG: ID 2 occupies 37% of the wheel, while ID 3 only occupies 12% of the wheel). When converted to "range" it will look like this:
ID - Value - Range
2 - 0.37 - 0 to 37
1 - 0.35 - 37 to 72
4 - 0.14 - 72 to 86
3 - 0.12- 86 to 100
Now, I am using Random.NextDouble() to generate a random value that is between 0.0 and 1.0. That random value will be considered as the "spin" on the wheel. Say, the randomizer returns 0.35, then ID 2 will be selected.
What is the best way to implement this given that I have an array of doubles?
The simplest solutions are often the best, if your range is 0 - 100 by design (or another manageebly small number), you can allocate an int[] and use the table of ranges you created to fill in the ID at the corresponding index, your "throw" will then look like:
int randomID = rangesToIDs[random.nextInt(rangesToIDs.length)];
Btw, it is not necessary to sort the ID's on range size, as the randoms are assumed to be distributed uniformly it does not matter where in the lookup table a range is placed. It only matters that the number of entries is proportional to the chance to throw an ID.
Let's assume your initial data is represented as array D[n], where D[i] = (id, p) and sum(D[i].p for i=0..n-1) == 1.
Build a second array P[n] such that P[i] = (q, id): P[i] = (sum(D[j].p for j in 0..i), D[j].id) -- i.e., convert individual probablity of each slice i into cumulative probability of all slices preceding i (inclusive). Note that, by definition, this array P is ordered by field q (i.e. by cumulative probability).
Now you can use binary search to find the slice chosen by the random number r (0 <= r <= 1):
find highest i such that P[i].q <= r; then P[i].id is your slice.
It is possible to speed up the lookup further by hashing the probability range with a fixed grid. I can write more details on this if anybody is interested.
As jk wrote sorted dictionary of should be fine.
let's say you got dictionary like this:
0.37 2
0.72 1
0.86 4
1.00 3
You roll xx = 0.66..
Iterate through dictionary starting from lowest number (that's 0.37)
if xx < dict[i].key
return dict[i].value
Or another solution which comes to my mind is List of custom objects containing lower and upper bound and value. You iterate then through list and check if rolled number is in range of up and low bounds.
a sorted map/dictionary with the 'Value' as the key and the 'ID' as the value would allow you to quickly find the upper bound of the range you are in and then look up the ID for that range
assuming your dictionary allows it, a binary search would be better to find the upper bound than interating throught the entire dictionary
boundaries = [37, 72, 86, 100]
num = 100 * random
for i in boundaries:
if num < i then return i
How would you implement a random number generator that, given an interval, (randomly) generates all numbers in that interval, without any repetition?
It should consume as little time and memory as possible.
Example in a just-invented C#-ruby-ish pseudocode:
interval = new Interval(0,9)
rg = new RandomGenerator(interval);
count = interval.Count // equals 10
count.times.do{
print rg.GetNext() + " "
}
This should output something like :
1 4 3 2 7 5 0 9 8 6
Fill an array with the interval, and then shuffle it.
The standard way to shuffle an array of N elements is to pick a random number between 0 and N-1 (say R), and swap item[R] with item[N]. Then subtract one from N, and repeat until you reach N =1.
This has come up before. Try using a linear feedback shift register.
One suggestion, but it's memory intensive:
The generator builds a list of all numbers in the interval, then shuffles it.
A very efficient way to shuffle an array of numbers where each index is unique comes from image processing and is used when applying techniques like pixel-dissolve.
Basically you start with an ordered 2D array and then shift columns and rows. Those permutations are by the way easy to implement, you can even have one exact method that will yield the resulting value at x,y after n permutations.
The basic technique, described on a 3x3 grid:
1) Start with an ordered list, each number may exist only once
0 1 2
3 4 5
6 7 8
2) Pick a row/column you want to shuffle, advance it one step. In this case, i am shifting the second row one to the right.
0 1 2
5 3 4
6 7 8
3) Pick a row/column you want to shuffle... I suffle the second column one down.
0 7 2
5 1 4
6 3 8
4) Pick ... For instance, first row, one to the left.
2 0 7
5 1 4
6 3 8
You can repeat those steps as often as you want. You can always do this kind of transformation also on a 1D array. So your result would be now [2, 0, 7, 5, 1, 4, 6, 3, 8].
An occasionally useful alternative to the shuffle approach is to use a subscriptable set container. At each step, choose a random number 0 <= n < count. Extract the nth item from the set.
The main problem is that typical containers can't handle this efficiently. I have used it with bit-vectors, but it only works well if the largest possible member is reasonably small, due to the linear scanning of the bitvector needed to find the nth set bit.
99% of the time, the best approach is to shuffle as others have suggested.
EDIT
I missed the fact that a simple array is a good "set" data structure - don't ask me why, I've used it before. The "trick" is that you don't care whether the items in the array are sorted or not. At each step, you choose one randomly and extract it. To fill the empty slot (without having to shift an average half of your items one step down) you just move the current end item into the empty slot in constant time, then reduce the size of the array by one.
For example...
class remaining_items_queue
{
private:
std::vector<int> m_Items;
public:
...
bool Extract (int &p_Item); // return false if items already exhausted
};
bool remaining_items_queue::Extract (int &p_Item)
{
if (m_Items.size () == 0) return false;
int l_Random = Random_Num (m_Items.size ());
// Random_Num written to give 0 <= result < parameter
p_Item = m_Items [l_Random];
m_Items [l_Random] = m_Items.back ();
m_Items.pop_back ();
}
The trick is to get a random number generator that gives (with a reasonably even distribution) numbers in the range 0 to n-1 where n is potentially different each time. Most standard random generators give a fixed range. Although the following DOESN'T give an even distribution, it is often good enough...
int Random_Num (int p)
{
return (std::rand () % p);
}
std::rand returns random values in the range 0 <= x < RAND_MAX, where RAND_MAX is implementation defined.
Take all numbers in the interval, put them to list/array
Shuffle the list/array
Loop over the list/array
One way is to generate an ordered list (0-9) in your example.
Then use the random function to select an item from the list. Remove the item from the original list and add it to the tail of new one.
The process is finished when the original list is empty.
Output the new list.
You can use a linear congruential generator with parameters chosen randomly but so that it generates the full period. You need to be careful, because the quality of the random numbers may be bad, depending on the parameters.
I'm looking for an algorithm which can take two sets of integers (both positive and negative) and find subsets within each that have the same sum.
The problem is similar to the subset sum problem except that I'm looking for subsets on both sides.
Here's an example:
List A {4, 5, 9, 10, 1}
List B {21, 7, -4, 180}
So the only match here is:
{10, 1, 4, 9} <=> {21, 7, -4}
Does anyone know if there are existing algorithms for this kinda problems?
So far, the only solution I have is a brute force approach which tries every combination but it performs in Exponential time and I've had to put a hard limit on the number of elements to consider to avoid it from taking too long.
The only other solution I can think of is to run a factorial on both lists and look for equalities there but that is still not very efficient and takes exponentially longer as the lists get bigger.
What others have said is true:
This problem is NP-complete. An easy reduction is to usual subset-sum. You can show this by noting that a subset of A sums to a subset of B (not both empty) only if a non-empty subset of A union (-B) sums to zero.
This problem is only weakly NP-complete, in that it's polynomial in the size of the numbers involved, but is conjectured to be exponential in their logarithms. This means that the problem is easier than the moniker "NP-complete" might suggest.
You should use dynamic programming.
So what am I contributing to this discussion? Well, code (in Perl):
#a = qw(4 5 9 10 1);
#b = qw(21 7 -4 180);
%a = sums( #a );
%b = sums( #b );
for $m ( keys %a ) {
next unless exists $b{$m};
next if $m == 0 and (#{$a{0}} == 0 or #{$b{0}} == 0);
print "sum(#{$a{$m}}) = sum(#{$b{$m}})\n";
}
sub sums {
my( #a ) = #_;
my( $a, %a, %b );
%a = ( 0 => [] );
while( #a ) {
%b = %a;
$a = shift #a;
for my $m ( keys %a ) {
$b{$m+$a} = [#{$a{$m}},$a];
}
%a = %b;
}
return %a;
}
It prints
sum(4 5 9 10) = sum(21 7)
sum(4 9 10 1) = sum(21 7 -4)
so, notably, there is more than one solution that works in your original problem!
Edit: User itzy correctly pointed out that this solution was wrong, and worse, in multiple ways!! I'm very sorry about that and I've hopefully addressed these concerns in the new code above. Nonetheless, there is still one problem, namely that for any particular subset-sum, it only prints one of the possible solutions. Unlike the previous problems, which were straight-up errors, I would classify this as an intentional limitation. Best of luck and beware of bugs!
Like the subset sum problem, this problem is weakly NP-complete, so it has a solution that runs in time polynomial(M), where M is the sum of all numbers appearing in the problem instance. You can achieve that with dynamic programming. For each set you can generate all possible sums by filling a 2-dimensional binary table, where "true" at (k,m) means that a subset sum m can be achieved by picking some elements from the first k elements of the set.
You fill it iteratively - you set (k,m) to "true" if (k-1,m) is set to "true" (obviously, if you can get m from k-1 elements, you can get it from k elements by not picking the k-th) or if (k-1,m-d) is set to "true" where d is the value of k-th element in the set (the case where you pick the k-th element).
Filling the table gets you all the possible sums in the last column (the one representing the whole set). Do this for both sets and find common sums. You can backtrack the actual subsets representing the solutions by reversing the process which you used to fill the tables.
Thanks a lot for all the quick responses!
The dynamic programming solution is not really different to the exhaustive approch we have right now and I guess if we need the optimal solution we do need to consider every possible combination but the time it takes to generate this exhaustive list of sums are too long..
Did a quick test and the time it takes to generate all possible sums for x number of elements very quickly go over 1 min:
11 elements took - 0.015625 seconds
12 elements took - 0.015625 seconds
13 elements took - 0.046875 seconds
14 elements took - 0.109375 seconds
15 elements took - 0.171875 seconds
16 elements took - 0.359375 seconds
17 elements took - 0.765625 seconds
18 elements took - 1.609375 seconds
19 elements took - 3.40625 seconds
20 elements took - 7.15625 seconds
21 elements took - 14.96875 seconds
22 elements took - 31.40625 seconds
23 elements took - 65.875 seconds
24 elements took - 135.953125 seconds
25 elements took - 282.015625 seconds
26 elements took - 586.140625 seconds
27 elements took - 1250.421875 seconds
28 elements took - 2552.53125 seconds
29 elements took - 5264.34375 seconds
which for the business problem we're trying to solve is not really acceptable.. I'm gonna go back to the drawing board and see whether we do indeed need to know all the solutions or can we just do with one (the smallest/largest subset, e.g.) instead and hopefully that can help simply the problem and make my algorithm perform to expectaion.
Thanks all the same!
subset sum is Np-complete and you can polynomially reduce your problem to it, so your problem is NP-complete too.