Hello StackOverflow community!
I had this question in my mind from so many days and finally have decided to get it sorted out. So, given a algorithm or say a function which implements some non-standard algorithm in your daily coding activity, how do you go about analyzing the rum time complexity?
Ok let me be more specific. Suppose you are solving this problem,
Given a NxN matrix consisting of positive integers, find the longest increasing sequence in it. You may only traverse in up, down, left or right directions but not diagonally.
Eg: If the matrix is
[ [9,9,4],
[6,6,8],
[2,1,1] ].
the algorithm must return 4
(The sequence being 1->2->6->9)
So yeah, looks like I have to use DFS. I get this part. I have done my Algorithms course back in Uni and can work my way around such questions. So, I come up with this solution say,
class Solution
{
public int longestIncreasingPathStarting(int[][] matrix, int i, int j)
{
int localMax = 1;
int[][] offsets = {{0,1}, {0,-1}, {1,0}, {-1,0}};
for (int[] offset: offsets)
{
int x = i + offset[0];
int y = j + offset[1];
if (x < 0 || x >= matrix.length || y < 0 || y >= matrix[i].length || matrix[x][y] <= matrix[i][j])
continue;
localMax = Math.max(localMax, 1+longestIncreasingPathStarting(matrix, x, y));
}
return localMax;
}
public int longestIncreasingPath(int[][] matrix)
{
if (matrix.length == 0)
return 0;
int maxLen = 0;
for (int i = 0; i < matrix.length; ++i)
{
for (int j = 0; j < matrix[i].length; ++j)
{
maxLen = Math.max(maxLen, longestIncreasingPathStarting(matrix, i, j));
}
}
return maxLen;
}
}
Inefficient, I know, but I wrote it this way on purpose! Anyways my question is, how do you go about analyzing the run time of longestIncreasingPath(matrix) function?
I can understand the analysis they teach us in a Algos course, you know the standard MergeSort, QuickSort analysis etc. but unfortunately and I hate to say this, that did not prepare me to apply it in my day-day coding job. I want to do it now, and hence would like to start it by analyzing such functions.
Can someone help me out here and describe the steps one would take to analyze the runtime of the above function? That would greatly help me. Thanks in advance, Cheers!
For day to day work eye-balling things usually works well.
In this case you will try to go in every direction recursively. So a really bad example comes to mind like: [[1,2,3], [2,3,4], [3,4,5]] so that you have two options from most cells. I happen to know that this will be O((2*n) ! / (n!*n!)) steps, but another good guess would be O(2^N). Now that you have an example where you know or can compute more easily the complexity, the overall complexity has to be at least that.
Usually, it doesn't really matter which one it is exactly since for both O(N!) and O(2^N) the run-time grows very fast and should only work fast for up to around 10-20 maybe a bit more if you are willing to wait. You would not run this algorithm for N ~= 1000, you would need something polynomial. So an rough estimate that you have a exponential solution would be enough to make a decision.
So in general to get an idea of the complexity, try to relate your solution to other algorithms where you know the complexity already or figure out a worst case scenario for the algorithm where it's easier to judge the complexity. Even if you are slightly off it might still help you make a decision.
If you need to compare algorithms of more similar complexity (ie. O(NlogN) vs O(N^2) for N~=100) you should implement both and benchmark since the constant factor might be the leading contributor to the run-time.
Related
I was watching a lecture about program efficiency where the professor said:
"If I measure running time, it will certainly vary as the algorithm change. (...) But one of the problems is that it will also vary as a function of the implementation (...) If I use a loop that's got a couple of more steps inside of it in one algorithm than another, it's going to change the time."
I am having a hard time wrapping my head around the implementation's influence.
So my question is: Why can't we consider those extra loop steps inside of one algorithm, when compared to the other, simply something that is necessary for it to run and that is also a part of the algorithm's efficiency? Or did I completely miss the point here?
Thank you!
They are pointing out the difference between "algorithm" and "specific code written in a programming language". "Algorithm" is somewhat of a vague term and "algorithms" are often described in pseudo-code that can be either very detailed, or not detailed at all.
For instance, consider the following algorithm, to test whether a number n is prime or not:
If any number d between 2 and the square root of n divides n:
Return False (n is not prime)
Else:
Return True (n is prime)
How exactly do you loop over the numbers between 2 and the square root? You could do:
// IMPLEMENTATION A
bool is_prime(int n)
{
int s = sqrt(n);
for (int d = 2; d <= s; d++)
{
if (n % d == 0)
return false;
}
return true;
}
or:
// IMPLEMENTATION B
bool is_prime(int n)
{
for (int d = 2; d * d <= n; d++)
{
if (n % d == 0)
return false;
}
return true;
}
Those two codes both implement the algorithm that I described. However, they might not have exactly the same runtime, since the former requires computing sqrt(n) once, but the latter requires computing d * d at every iteration in the loop.
Computer scientists want to be able to discuss the complexity of the algorithm that I described above in pseudo-code. And they don't want someone to give the boring answer "Sorry, the complexity of that algorithm is impossible to calculate, because we don't know if it's implementation A or implementation B".
To find all prime numbers from 1 to N.
I know we usually approach this problem using Sieve of Eratosthenes, I had an alternate approach in mind using gcd that I wanted your views on.
My approach->
Keep a maintaining a variable if all prime numbers are processed till any iteration. If gcd of this var, number i ==1. That means the nos. are co-prime so i must be prime.
For ex: gcd(210,11) == 1, so 11 is prime.
{210=2*3*5*7}
Pseudocode:
Init num_list={contains numbers 2 to N} [since 0 and 1 arent prime nos.]
curr_gcd = 2, gcd_val=1
For i=3;i<=N;i++
gcd_val=__gcd(curr_gcd,i)
if gcd_val == 1 //(prime)
curr_gcd = curr_gcd * i
else //(composite so remove from list)
numList.remove(i)
Alternatively, we can also have a list and push the prime numbers into that list.
SC = O(N)
TC = O(N log(N)) [TC to calculate gcd using euclid's method => O(log(max(a,b)))]
Does this seem right or I am calculating the TC incorrectly here. Please post your views on this.
TIA!
Looks like the time complexity of my approach is closer to O(log^2(n)) as pointed out by many in the comments.
Also, the curr_gcd var would become quite large as N is increased and would definitely overflow int and long size limits.
Thanks to everyone who responded!
Maybe your method is theoretically right,but evidently, it's not excellent.
It's efficiency is worse than SoE, the range of data that it needs is too large. So maybe it seems elegant to look but hard to use.
In my views, "To find all prime numbers from 1 to N" is already a well-known problem and that means it's solution is well considered.
At first, maybe we use brute-force to deal with it like this.
int primes[N],cnt;//store all prime numbers
bool st[N];//st[i]:whether i is rejected
void get_primes(int n){
for(int i=2;i<=n;i++){
if(st[i]) continue;
primes[cnt++]=i;
for(int j=i+i;j<=n;j+=i){
st[j]=true;
}
}
}
it's a O(n^2) time algorithm.Too slow to endure.
Go ahead. We have SoE, which use O(nlognlogn) time.
But we have a better algorithm called "liner sieve", which only use O(n) time, just as it's name. I implement it with C language like this.
int primes[N],cnt;
bool st[N];
void get_primes(int n){
for(int i=2;i<=n;i++){
if(!st[i]) primes[cnt++]=i;
for(int j=0;primes[j]*i<=n;j++){
st[primes[j]*i]=true;
if(i%primes[j]==0) break;
}
}
}
this O(n) algorithm is used by me to slove this kind of algorithm problems that appear in major IT companies and many kinds of OJ.
What is the runing time of this algorthm in Big-O and how i convert this to iterative algorthm?
public static int RecursiveMaxOfArray(int[] array) {
int array1[] = new int[array.length/2];
int array2[] = new int[array.length - (array.length/2)];
for (int index = 0; index < array.length/2 ; index++) {
array1[index] = array[index];
}
for (int index = array.length/2; index < array.length; index++) {
array2[index - array.length/2] = array[index] ;
}
if (array.length > 1) {
if(RecursiveMaxOfArray(array1) > RecursiveMaxOfArray(array2)) {
return RecursiveMaxOfArray(array1) ;
}
else {
return RecursiveMaxOfArray(array2) ;
}
}
return array[0] ;
}
At each stage, an array of size N is divided into equal halves. The function is then recursively called three times on an array of size N/2. Why three instead of the four which are written? Because the if statement only enters one of its clauses. Therefore the recurrence relation is T(N) = 3T(N/2) + O(N), which (using the Master theorem) gives O(N^[log2(3)]) = O(n^1.58).
However, you don't need to call it for the third time; just cache the return result of each recursive call in a local variable. The coefficient 3 in the recurrence relation becomes 2; I'll leave it to you to apply the Master theorem on the new recurrence.
There's another answer that accurately describes your algorithm's runtime complexity, how to determine it, and how to improve it, so I won't focus on that. Instead, let's look at the other part of your question:
how [do] i convert this to [an] iterative algorithm?
Well, there's a straightforward solution to that which you hopefully could have gotten yourself - loop over the list and track the smallest value you've seen so far.
However, I'm guessing your question is better phrased as this:
How do I convert a recursive algorithm into an iterative algorithm?
There are plenty of questions and answers on this, not just here on StackOverflow, so I suggest you do some more research on this subject. These blog posts on converting recursion to iteration may be an excellent place to start if this is your approach to take, though I can't vouch for them because I haven't read them. I just googled "convert recursion to iteration," picked the first result, then found this page which links to all four of the blog post.
In the book Introduction To Algorithms , the naive approach to solving rod cutting problem can be described by the following recurrence:
Let q be the maximum price that can be obtained from a rod of length n.
Let array price[1..n] store the given prices . price[i] is the given price for a rod of length i.
rodCut(int n)
{
initialize q as q=INT_MIN
for i=1 to n
q=max(q,price[i]+rodCut(n-i))
return q
}
What if I solve it using the below approach:
rodCutv2(int n)
{
if(n==0)
return 0
initialize q = price[n]
for i = 1 to n/2
q = max(q, rodCutv2(i) + rodCutv2(n-i))
return q
}
Is this approach correct? If yes, why do we generally use the first one? Why is it better?
NOTE:
I am just concerned with the approach to solving this problem . I know that this problem exhibits optimal substructure and overlapping subproblems and can be solved efficiently using dynamic programming.
The problem with the second version is it's not making use of the price array. There is no base case of the recursion so it'll never stop. Even if you add a condition to return price[i] when n == 1 it'll always return the result of cutting the rod into pieces of size 1.
your 2nd approach is absolutely correct and its time complexity is also same as the 1st one.
In Dynamic Programming also, we can make tabulation on same approach.Here is my solution for recursion :
int rodCut (int price[],int n){
if(n<=0) return 0;
int ans = price[n-1];
for(int i=1; i<=n/2 ; ++i){
ans=max(ans, (rodCut(price , i) + rodCut(price , n-i)));
}
return ans;
}
And, Solution for Dynamic Programming :
int rodCut(int *price,int n){
int ans[n+1];
ans[0]=0; // if length of rod is zero
for(int i=1;i<=n;++i){
int max_value=price[i-1];
for(int j=1;j<=i/2;++j){
max_value=max(max_value,ans[j]+ans[i-j]);
}
ans[i]=max_value;
}
return ans[n];
}
Your algorithm looks almost correct - you need to be a bit careful when n is odd.
However, it's also exponential in time complexity - you make two recursive calls in each call to rodCutv2. The first algorithm uses memoisation (the price array), so avoids computing the same thing multiple times, and so is faster (it's polynomial-time).
Edit: Actually, the first algorithm isn't correct! It never stores values in prices, but I suspect that's just a typo and not intentional.
{1,3,5} denomination coins; Sum = 11.
find minimum number of coins which can be used to make the sum
(We can use any number of coins of each denomination)
I searched for Run Time complexity of this Coin change problem particularly using dynamic programming method. But was not able to find explanation anywhere.
How to calculate the complexity of non dynamic solution and then change it for the dynamic one? (not the greedy one)
Edit:
Here is an implementation for which analysis was asked.
public int findCoinChange(int[] coins, int sum,int count) {
int ret = 0, maxRet = -1;
if(sum ==0)maxRet = count;
else if(sum < 0)maxRet = -1;
else{
for(int i:coins){
ret = findCoinChange(coins, sum - i,count+1);
if(maxRet< 0)maxRet = ret;
else if(ret >=0 && ret < maxRet){
maxRet = ret;
}
}
}
if(maxRet < 0)return -1;
else return maxRet;
}
Looks like Combinatorial explosion to me. However I am not sure how to deduce a run time complexity for this.
The dynamic programming solution to this problem is clearly O(k * n) (nested loops, blah blah blah) where k is the number of coins and n is the amount of money that change is being made for.
I don't know what you mean by non-dynamic programming solution. Sorry, you're going to have specify what algorithm you mean. The greedy algorithm fails in some cases, so you shouldn't be referring to that. Do you mean the linear programming solution? That's a terrible approach to this problem because we don't know what the complexity is, and it's possible to make it run arbitrarily slowly.
I also don't know what you mean by "change it for the dynamic one."