1D Memoization in Recursive solution of Longest Increasing Subsequence - algorithm

Calculating LIS (Longest Increasing Subsequence) in an array is a very famous Dynamic Programming problem. However in every tutorial they first show the recursive solution without using the concepts of DP and then solve it by applying Bottom-Up DP (Iterative Solution).
My question is:
How will we use Memoization in Recursive Solution itself.
Not just Memoization but memoization using 1D array.
I did some research but could not find anything of relevance. Although there are 2 places where recursive memoization has been asked 1 & 2 but the solutions over there, are using 2D Map / Array for memoization.
Anyway Memoizing the solution with 1D array, is giving wrong output.
Here is what I did:
int lis(int idx, int prev)
{
if(idx >= N)
return 0;
if(dp[idx])
return dp[idx];
int best_ans = lis(idx+1, prev);
int cur_ans = 0;
if(arr[idx] > prev)
{
cur_ans = 1 + lis(idx+1, arr[idx]);
}
int ans = max(best_ans, cur_ans);
dp[idx] = ans;
return ans;
}
int main()
{
// Scan N
// Scan arr
ans = lis(0, -1));
print ans;
}
Although I know the reason why this solution is giving Wrong output as:
There can be more than one solution for the giving index based on what was the previous value.
But I still want to know how it can be done using 1D array.
I am curious to know the solution because I have read that every DP Top-Down solution can be reframed into Bottom-Up and vice-versa.
It would be highly helpful if someone could provide some insight for the same.
Thanks in advance.

This can't be done because the problem fundamentally needs a 2D data structure to solve.
The bottom-up approach can cheat by producing one row at a time of the data structure. Viewed over time, it produces a 2D data structure, but at any given time you only see one dimension of it.
The top-down approach has to build the entire 2D data structure.
This is a fundamental tradeoff in DP. It is usually easier to write down the top-down approach. But the bottom-up approach only has to have part of the overall data structure at any time, and therefore has significantly lower memory requirements.

def LongestIncreasingSubsequenceMemo(nums, i, cache):
if cache[i] > 0:
return cache[i]
result = 1
for j in range(i):
if nums[i] > nums[j]:
result = max(result, 1 + LongestIncreasingSubsequenceMemo(nums, j, cache))
cache[i] = result
return result
def main():
nums = [1,2,3,4,5]
if not nums:
return 0
n = len(nums)
cache = [0 for i in range(n)]
result = 1
for i in range(n):
result = max(result, LongestIncreasingSubsequenceMemo(nums, i, cache))
return result
if __name__ == "__main__":
print(main())
In the above solution, we are taking a one-dimensional array and updating it for each element in the array.

This can be done and there is no requirement for a 2d array. Because we need to find the max LIS ending at every index So if we calculate LIS for the element at arr[0], instead of calculating again and again we calculate once and store it in DP[1].
If we calculated LIS for {arr[0],arr[1]} then we store the result in DP[2] and so on until the DP[n]. See the code below to understand this fully.
Memoization of the recursive code
above code got accepted on gfg as well

Related

Big-O & Runing Time of this algorthm , how can convert this to an iterative algorthm

What is the runing time of this algorthm in Big-O and how i convert this to iterative algorthm?
public static int RecursiveMaxOfArray(int[] array) {
int array1[] = new int[array.length/2];
int array2[] = new int[array.length - (array.length/2)];
for (int index = 0; index < array.length/2 ; index++) {
array1[index] = array[index];
}
for (int index = array.length/2; index < array.length; index++) {
array2[index - array.length/2] = array[index] ;
}
if (array.length > 1) {
if(RecursiveMaxOfArray(array1) > RecursiveMaxOfArray(array2)) {
return RecursiveMaxOfArray(array1) ;
}
else {
return RecursiveMaxOfArray(array2) ;
}
}
return array[0] ;
}
At each stage, an array of size N is divided into equal halves. The function is then recursively called three times on an array of size N/2. Why three instead of the four which are written? Because the if statement only enters one of its clauses. Therefore the recurrence relation is T(N) = 3T(N/2) + O(N), which (using the Master theorem) gives O(N^[log2(3)]) = O(n^1.58).
However, you don't need to call it for the third time; just cache the return result of each recursive call in a local variable. The coefficient 3 in the recurrence relation becomes 2; I'll leave it to you to apply the Master theorem on the new recurrence.
There's another answer that accurately describes your algorithm's runtime complexity, how to determine it, and how to improve it, so I won't focus on that. Instead, let's look at the other part of your question:
how [do] i convert this to [an] iterative algorithm?
Well, there's a straightforward solution to that which you hopefully could have gotten yourself - loop over the list and track the smallest value you've seen so far.
However, I'm guessing your question is better phrased as this:
How do I convert a recursive algorithm into an iterative algorithm?
There are plenty of questions and answers on this, not just here on StackOverflow, so I suggest you do some more research on this subject. These blog posts on converting recursion to iteration may be an excellent place to start if this is your approach to take, though I can't vouch for them because I haven't read them. I just googled "convert recursion to iteration," picked the first result, then found this page which links to all four of the blog post.

Alternative approach for Rod Cutting Algorithm ( Recursive )

In the book Introduction To Algorithms , the naive approach to solving rod cutting problem can be described by the following recurrence:
Let q be the maximum price that can be obtained from a rod of length n.
Let array price[1..n] store the given prices . price[i] is the given price for a rod of length i.
rodCut(int n)
{
initialize q as q=INT_MIN
for i=1 to n
q=max(q,price[i]+rodCut(n-i))
return q
}
What if I solve it using the below approach:
rodCutv2(int n)
{
if(n==0)
return 0
initialize q = price[n]
for i = 1 to n/2
q = max(q, rodCutv2(i) + rodCutv2(n-i))
return q
}
Is this approach correct? If yes, why do we generally use the first one? Why is it better?
NOTE:
I am just concerned with the approach to solving this problem . I know that this problem exhibits optimal substructure and overlapping subproblems and can be solved efficiently using dynamic programming.
The problem with the second version is it's not making use of the price array. There is no base case of the recursion so it'll never stop. Even if you add a condition to return price[i] when n == 1 it'll always return the result of cutting the rod into pieces of size 1.
your 2nd approach is absolutely correct and its time complexity is also same as the 1st one.
In Dynamic Programming also, we can make tabulation on same approach.Here is my solution for recursion :
int rodCut (int price[],int n){
if(n<=0) return 0;
int ans = price[n-1];
for(int i=1; i<=n/2 ; ++i){
ans=max(ans, (rodCut(price , i) + rodCut(price , n-i)));
}
return ans;
}
And, Solution for Dynamic Programming :
int rodCut(int *price,int n){
int ans[n+1];
ans[0]=0; // if length of rod is zero
for(int i=1;i<=n;++i){
int max_value=price[i-1];
for(int j=1;j<=i/2;++j){
max_value=max(max_value,ans[j]+ans[i-j]);
}
ans[i]=max_value;
}
return ans[n];
}
Your algorithm looks almost correct - you need to be a bit careful when n is odd.
However, it's also exponential in time complexity - you make two recursive calls in each call to rodCutv2. The first algorithm uses memoisation (the price array), so avoids computing the same thing multiple times, and so is faster (it's polynomial-time).
Edit: Actually, the first algorithm isn't correct! It never stores values in prices, but I suspect that's just a typo and not intentional.

Coin change algorithm and pseudocode: Need clarification

I'm trying to understand the coin change problem solution, but am having some difficulty.
At the Algorithmist, there is a pseudocode solution for the dynamic programming solution, shown below:
n = goal number
S = [S1, S2, S3 ... Sm]
function sequence(n, m)
//initialize base cases
for i = 0 to n
for j = 0 to m
table[i][j] = table[i-S[j]][j] + table[i][j-1]
This is a pretty standard O(n^2) algorithm that avoids recalculating the same answer multiple times by using a 2-D array.
My issue is two-fold:
How to define the base cases and incorporate them in table[][] as initial values
How to extract out the different sequences from the table
Regarding issue 1, there are three base cases with this algorithm:
if n==0, return 1
if n < 0, return 0
if n >= 1 && m <= 0, return 0
How to incorporate them into table[][], I am not sure. Finally, I have no idea how to extract out the solution set from the array.
We can implement a dynamic programming algorithm in at least two different approaches. One is the top-down approach using memoization, the other is the bottom-up iterative approach.
For a beginner to dynamic programming, I would always recommend using the top-down approach first since this will help them understand the recurrence relationships in dynamic programming.
So in order to solve the coin changing problem, you've already understood what the recurrence relationship says:
table[i][j] = table[i-S[j]][j] + table[i][j-1]
Such a recurrence relationship is good but is not that well-defined since it doesn't have any boundary conditions. Therefore, we need to define boundary conditions in order to ensure the recurrence relationship could successfully terminate without going into an infinite loop.
So what will happen when we try to go down the recursive tree?
If we need to calculate table[i][j], which means the number of approaches to change i using coins from type 0 to j, there are several corner cases we need to handle:
1) What if j == 0?
If j == 0 we will try to solve the sub-problem table(i,j-1), which is not a valid sub-problem. Therefore, one boundary condition is:
if(j==0) {
if(i==0) table[i][j] = 1;
else table[i][j] = 0;
}
2) What if i - S[j] < 0?
We also need to handle this boundary case and we know in such a condition we should either not try to solve this sub-problem or initialize table(i-S[j],j) = 0 for all of these cases.
So in all, if we are going to implement this dynamic programming from a top-down memoization approach, we can do something like this:
int f(int i, int j) {
if(calc[i][j]) return table[i][j];
calc[i][j] = true;
if(j==0) {
if(i==0) return table[i][j]=1;
else return table[i][j]=0;
}
if(i>=S[j])
return table[i][j]=table[i-S[j][j]+table[i][j-1];
else
return table[i][j]=table[i][j-1];
}
In practice, it's also possible that we use the value of table arrays to help track whether this sub-problem has been calculated before (e.g. we can initialize a value of -1 means this sub-problem hasn't been calculated).
Hope the answer is clear. :)

Algorithm required for this task?

Inputs: n (int) and n values (float) that represent exchange rates
(different between them) with a random value between 4 and 5.
Output: compute the maximum number of values that can be used (in the
same order) to represent an ascending then descending curve?
e.x. The eight values
4.5 4.6 4.3 4.0 4.8 4.4 4.7 4.1
should output
5 (4.5 4.6 4.8 4.4 4.1)
My approach
If I try successive ifs, I get a random array that respects the curve condition, but not the longest.
I have not tried backtracking because I am not that familiar with it, but something tells me I have to compute all the solutions with it then pick the longest.
And lastly: brute force, but since it is an assignment for algorithm design; I may as well not hand it in. :)
Is there a simpler/more efficient/faster method?
Here's my try based on Daniel Lemire's algorithm. It seems it doesn't take into account the positions 0, i and n. I'm sure the ifs are the problem, how can I fix them?
for(int i = 0; i<n-1; i++){
int countp=0; // count ascending
int countn=0; // count descending
for(int j=0;j<=i;j++){
if(currency[j]<currency[j+1]){
countp++;
System.out.print(j+" ");
}
}
System.out.print("|| ");
for(int j=i;j<n-1;j++){
if(currency[j]>currency[j+1]){
countn++;
System.out.print(j+" ");
}
}
System.out.println();
if(countn+countp>maxcount) maxcount=countn+countp;
}
Firstly, you want to be able to compute the longest monotonic subsequence from one point to another. (Whether it is increasing or decreasing does not affect the problem much.) To do this, you may use dynamic programming. For example, to solve the problem given indexes 0 to i, you start by solving the problem from 0 to 0 (trivial!), then from 0 to 1, then from 0 to 2, and so on, each time recording (in an array) your best solution.
For example, here is some code in python to compute the longest non-decreasing sequence going from index 0 to index i. We use an array (bbest) to store the solution from 0 to j for all j's from 0 to i: that is, the length of the longest non-decreasing subsequence from 0 to j. (The strategy used is dynamic programming.)
def countasc(array,i):
mmin = array[0] # must start with mmin
mmax= array[i] # must end with mmax
bbest=[1] # going from 0 to 0 the best we can do is length 1
for j in range(1,i+1): # j goes from 1 to i
if(array[j]>mmax):
bbest.append(0) # can't be used
continue
best = 0 # store best result
for k in range(j-1,-1,-1): # count backward from j-1 to 0
if(array[k]>array[j]) :
continue # can't be used
if(bbest[k]+1>best):
best = bbest[k]+1
bbest.append(best)
return bbest[-1] # return last value of array bbest
or equivalently in Java (provided by request):
int countasc(float[] array,int i) {
float mmin = array[0];
float mmax = array[i];
ArrayList<Integer> bbest= new ArrayList<Integer>();
bbest.add(1);
for (int j = 1; j<=i;++j) {
if(array[j]>mmax){
bbest.add(0);
continue;
}
int best = 0;
for(int k = j-1; k>=0;--k) {
if(array[k]>array[j])
continue;
if(bbest.get(k).intValue()+1>best)
best = bbest.get(k).intValue()+1;
}
bbest.add(best);
}
return bbest.get(bbest.size()-1);
}
You can write the same type of function to find the longest non-increasing sequence from i to n-1 (left as an exercise).
Note that countasc runs in linear time.
Now, we can solve the actual problem:
Start with S, an empty array
For i an index that goes from 0 to n-1 :
compute the length of the longest increasing subsequence from 0 to i (see function countasc above)
compute the length of the longest decreasing subsequence from n-1 to i
add these two numbers, add the sum to S
return the max of S
It has quadratic complexity. I am sure you can improve this solution. There is a lot of redundancy in this approach. For example, for speed, you should probably not repeatedly call countasc with an uninitialized array bbest: it can be computed once. Possibly you can bring down the complexity to O(n log n) with some more work.
A first step is to understand how to solve the related longest increasing subsequence problem. For this problem, there is a simple algorithm that is O(n^2) though the optimal algorithm is O(n log n). Understanding these algorithms should put you on the right track to a solution.

Efficient list intersection algorithm

Given two lists (not necessarily sorted), what is the most efficient non-recursive algorithm to find the set intersection of those lists?
I don't believe I have access to hashing algorithms.
You could put all elements of the first list into a hash set. Then, iterate the second one and, for each of its elements, check the hash to see if it exists in the first list. If so, output it as an element of the intersection.
You might want to take a look at Bloom filters. They are bit vectors that give a probabilistic answer whether an element is a member of a set. Set intersection can be implemented with a simple bitwise AND operation. If you have a large number of null intersections, the Bloom filter can help you eliminate those quickly. You'll still have to resort to one of the other algorithms mentioned here to compute the actual intersection, however.
http://en.wikipedia.org/wiki/Bloom_filter
without hashing, I suppose you have two options:
The naive way is going to be compare each element to every other element. O(n^2)
Another way would be to sort the lists first, then iterate over them: O(n lg n) * 2 + 2 * O(n)
From the eviews features list it seems that it supports complex merges and joins (if this is 'join' as in DB terminology, it will compute an intersection). Now dig through your documentation :-)
Additionally, eviews has their own user forum - why not ask there_
with set 1 build a binary search tree with O(log n) and iterate set2 and search the BST m X O(log n) so total O(log n) + O(m)+O(log n) ==> O(log n)(m+1)
in C++ the following can be tried using STL map
vector<int> set_intersection(vector<int> s1, vector<int> s2){
vector<int> ret;
map<int, bool> store;
for(int i=0; i < s1.size(); i++){
store[s1[i]] = true;
}
for(int i=0; i < s2.size(); i++){
if(store[s2[i]] == true) ret.push_back(s2[i]);
}
return ret;
}
Here is another possible solution I came up with takes O(nlogn) in time complexity and without any extra storage. You can check it out here https://gist.github.com/4455373
Here is how it works: Assuming that the sets do not contain any repetition, merge all the sets into one and sort it. Then loop through the merged set and on each iteration create a subset between the current index i and i+n where n is the number of sets available in the universe. What we look for as we loop is a repeating sequence of size n equal to the number of sets in the universe.
If that subset at i is equal to that subset at n this means that the element at i is repeated n times which is equal to the total number of sets. And since there are no repetitions in any set that means each of the sets contain that value so we add it to the intersection. Then we shift the index by i + whats remaining between it and n because definitely none of those indexes are going to form a repeating sequence.
First, sort both lists using quicksort : O(n*log(n). Then, compare the lists by browsing the lowest values first, and add the common values. For example, in lua) :
function findIntersection(l1, l2)
i, j = 1,1
intersect = {}
while i < #l1 and j < #l2 do
if l1[i] == l2[i] then
i, j = i + 1, j + 1
table.insert(intersect, l1[i])
else if l1[i] > l2[j] then
l1, l2 = l2, l1
i, j = j, i
else
i = i + 1
end
end
return intersect
end
which is O(max(n, m)) where n and m are the sizes of the lists.
EDIT: quicksort is recursive, as said in the comments, but it looks like there are non-recursive implementations
Using skip pointers and SSE instructions can improve list intersection efficiency.
Why not implement your own simple hash table or hash set? It's worth it to avoid nlogn intersection if your lists are large as you say.
Since you know a bit about your data beforehand, you should be able to choose a good hash function.
I second the "sets" idea. In JavaScript, you could use the first list to populate an object, using the list elements as names. Then you use the list elements from the second list and see if those properties exist.
If there is a support for sets (as you call them in the title) as built-in usually there is a intersection method.
Anyway, as someone said you could do it easily (I will not post code, someone already did so) if you have the lists sorted. If you can't use recursion there is no problem. There are quick sort recursion-less implementations.
In PHP, something like
function intersect($X) { // X is an array of arrays; returns intersection of all the arrays
$counts = Array(); $result = Array();
foreach ($X AS $x) {
foreach ($x AS $y) { $counts[$y]++; }
}
foreach ($counts AS $x => $count) {
if ($count == count($X)) { $result[] = $x; }
}
return $result;
}
From the definition of Big-Oh notation:
T(N) = O(f(N)) if there are positive constants c and n 0 such that
T(N) ≤ cf(N) when N ≥ n 0.
Which in practice means that if the two lists are relatively small in size say something less 100 elements in each two for loops works just fine. Loop the first list and look for similar object in the second.
In my case it works just fine because I won't have more than 10 - 20 max elements in my lists.
However, a good solution is the sort the first O(n log n), sort the second also O(n log n) and merge them, another O(n log n) roughly speeking O(3 n log n), say that the two lists are the same size.
Time: O(n) Space: O(1) Solution for identifying points of intersection.
For example, the two given nodes will detect the point of intersection by swapping pointers every time they reach the end. Video Explanation Here.
public ListNode getIntersectionNode(ListNode headA, ListNode headB) {
ListNode pA = headA;
ListNode pB = headB;
while (pA != pB) {
pA = pA == null ? headB : pA.next;
pB = pB == null ? headA : pB.next;
}
return pA;
}
Thanks.
Edit
My interpretation of intersection is finding the point of intersection.
For example:
For the given lists A and B, A and B will "meet/intersect" at point c1, and the algo above will return c1. As OP stated that OP has NO access to Hashmaps or some sort, I believe OP is saying that the algo should have O(1) space complexity.
I got this idea from Leetcode some time ago, if interested: Intersection of Two Linked Lists.

Resources