I am trying to compute the run-time analysis of this Insertion Sort algorithm:
1) n = length[A]
2) count = 0
3) for (i=1; i<=n; i++)
4) for (j=1; j<=i; j++)
5) if A[j] <= 100
6) for (k=j; k<=j+2*i; k++)
7) A[j] = A[j]-1
8) count = count+1
9) return (count)
I have watched some videoes on youtube like: https://www.youtube.com/watch?v=tmKUHLs21PU
I have also read by book and I cannot find anything online that is similair to this (because of the 3 nested for loops and and if statement).
Now I am pretty good up until about like 5.
I understand that the runtime for line 3 is n, and for 4 it is Σ j =1 to n (tj)
after that I am completely lost, I know that there are to 'Σ's involved with the if statement and 3rd for loop. Can somebody please explain in detail what to do next and why it is like that. Thank you.
This sounds a lot like a homework problem, and it wouldn't be doing you any favors to just give you all the answers, but here are some principles that can hopefully help you figure out the rest on your own.
Line 4 will happen once the first time through the outer loop, twice the second time, and so forth up to n times on the nth time through the loop.
1 + 2 + ... + n
If we rearrange these to put the first and last addend together, then the second and the second-to-last, we see a pattern:
1 + 2 + ... (n-1) + n
= (n + 1) + (n - 1 + 2) + ... + (n - n/2 + n/2 + 1)
= (n + 1) + (n + 1) + ... + (n + 1)
= (n + 1) * n/2
= n²/2 + n/2
In terms of asymptotic complexity, the constant 1/2 and the n are outweighed by the n², so the big-O of line 4 is n².
Line 5 will have to be evaluated as many times as line 4 runs, regardless of what it evaluates to, so that'll be n². But how many times the lines inside it are run will depend on the values in the array. This is where you start running into best-case and worst-case complexity.
In the best case, the value in the array will always be greater than 100, so the complexity of the entire algorithm is equal to the complexity of line 5.
In the worst case, the value in A[j] will always be less than or equal to 100, so the for loop on line 6 will be evaluated, increasing the complexity of the overall algorithm.
I'll leave you to figure out how the remaining lines will affect the overall complexity.
And by the way, this doesn't look like an insertion sort to me. It's not comparing array values to each other and swapping their positions in an array. It's comparing array values to a constant (100) and reducing their values based on their position in the array.
Related
I have the following code. What time complexity does it have?
I have tried to write a recurrence relation for it but I can't understand when will the algorithm add 1 to n or divide n by 4.
void T(int n) {
for (i = 0; i < n; i++);
if (n == 1 || n == 0)
return;
else if (n%2 == 1)
T(n + 1);
else if (n%2 == 0)
T(n / 4);
}
You can view it like this: you always divide by four only if you have odd you add 1 to n before division. So, you should count how many times 1 was added. If there no increments then you have log4n recursive calls. Let's assume that you always have to add 1 before division. Then can rewrite it like this:
void T(int n) {
for (i = 0; i < n; i++);
if (n == 1 || n == 0)
return;
else if (n%2 == 0)
T(n / 4 + 1);
}
But n/4 + 1 < n/2, and in case of recursive call T(n/2), running time is O(log(n,4)), but base of logarithm doesn't impact running time in big-O notation because it's just like constant factor. So running time is O(log(n)).
EDIT:
As ALB pointed in a comment, there is cycle of length n. So, with accordance with master theorem running time is Theta(n). You can see it in another way as sum of n * (1 + 1/2 + 1/4 + 1/8 + ...) = 2 * n.
Interesting question. Be aware that even though your for loop is doing nothing, since it is not an optimized solution (see Dukeling's comment), it will be considered in your time complexity as if computing time was taken to iterate through it.
First part
The first section is definitely O(n).
Second part
Let's suppose for the sake of simplicity that half the time n will be odd and the other half time it will be even. Hence, the recursion is looping (n+1) half the time and (n/4) the other half.
Conclusion
For each time T(n) is called, the recursion will implicitly loop n times. Hence, The first half of the time, we will have a complexity of n * (n+1) = n^2 + n. The other half of the time, we will deal with a n * (n/4) = (1/4)n^2.
For Big O notation, we care more about the upper bound than its precise behavior. Hence, your algorithm would be bound by O(n^2).
I came across this question in a review for my class and I am having a difficult time understanding the professors justification and explanation of the solution. The question:
The following code computes 2^n for given n. Determine total number of lines executed. Justify your answer.
Power2(int n)
1) if(n=0)
2) return 1
3) else
4) k=Power2(n/2)
5) k=k*k;
6) if(k is even)
7) return k
8) else
9) return 2*k
The justification
I don't understand from the "hence" part on. If someone could break these steps down for me a little more and describe how these steps are equivalent, it would be a great help.
t(2^k) = //this is because 2^k is even so
[t(2^(k-1))] + 1 //you apply the first rule for t(n)
t(2^(k-1)) + 1 = //note that 2^(k-1) is also even
[t(2^(k-2)) + 1] + 1 //so you apply the same t(n) rule
each 2^(i = 0...k) will be even (except 2^0), so that you can apply the first t(n) rule for all steps, coming to the end of the recursion after k-1 steps.
then, you note that if you want to evaluate t(2^k), you must do about k operations, and k = lg(2^k), so Power2 is a logarithmic-time function,
The line 7) is wrong it should read 7) if (n is even).
Regarding the Hence part, they take
n = 2k
and show there are k iterations. Yet, from above equation we get
k = log2(n)
thus having k iterations is having log2(n) iterations.
The algorithm is O(log(n))
def mystery(L):
sum1 = 0
sum2 = 0
bound = 1
while bound <= len(L):
i = 0
while i < bound:
j = 0
while j < len(L):
if L[j] > L[i]:
sum1 = sum1 + L[j]
j = j + 2
j = 1
while j < len(L):
sum2 = sum2 + L[j]
j = j*2
i = i + 1
bound = bound * 2
return sum1 + sum2
I am having trouble finding the complexity of this function. I got to the i loop and don't know what to do.
It's a bit tricky to sort out how many times the middle level while loop runs. The outer loop increases bound by a factor of two on each pass (up to len(L)), which means the i loop will run O(bound) times per pass for O(log(N)) passes (where N is len(L)). The tricky part is how to add up the bound values, since they're changing on each pass.
I think the easiest way to figure out the sum is to start with the largest bound, just before the loop quits. First, lets assume that N (aka len(L)) is a power of 2. Then the last bound value will be exactly equal to N. The next smaller one (used on the next to last iteration) will be N/2 and the next after that will be N/4. Their sum will be:
N + N/2 + N/4 + N/8 + ... + 1
If we factor out N from each term, we'll get:
N*(1 + 1/2 + 1/4 + 1/8 + ... + 1/N)
You should recognize the sum in the parentheses, it's a simple geometric series (the sum of the powers of 1/2), which comes up pretty often in mathematics and analysis. If the sum went on forever, it would add up exactly to 2. Since we're quitting a bit early, it will be less than two by an amount equal to the last term (1/N). When we multiply the N term in again, we get the whole thing as being run 2*N - 1 times, so the loop is O(N)
The same Big-O bound works when N is not exactly a power of 2, since the values we added up in the analysis above will each serve as the upper bound for one of the actual bound values we will see in the loop.
So, the i loop runs O(N) times.
The code looks like:
for (int i = 1; i < N; i++) {
if (a[i] < a[i-1]) {
swap(i, i-1);
i = 0;
}
}
After trying out a few things i figure the worst case is when the input array is in descending order. Then looks like the compares will be maximum and hence we will consider only compares. Then it seems it would be a sum of sums, i.e sum of ... {1+2+3+...+(n-1)}+{1+2+3+...+(n-2)}+{1+2+3+...+(n-3)}+ .... + 1 if so what would be O(n) ?
If I am not on the right path can someone point out what O(n) would be and how can it be derived? cheers!
For starters, the summation
(1 + 2 + 3 + ... + n) + (1 + 2 + 3 + ... + n - 1) + ... + 1
is not actually O(n). Instead, it's O(n3). You can see this because the sum 1 + 2 + ... + n = O(n2, and there are n copies of each of them. You can more properly show that this summation is Θ(n3) by looking at the first n / 2 of these terms. Each of those terms is at least 1 + 2 + 3 + ... + n / 2 = Θ(n2), so there are n / 2 copies of something that's Θ(n2), giving a tight bound of Θ(n3).
We can upper-bound the total runtime of this algorithm at O(n3) by noting that every swap decreases the number of inversions in the array by one (an inversion is a pair of elements out of place). There can be at most O(n2) inversions in an array and a sorted array has no inversions in it (do you see why?), so there are at most O(n2) passes over the array and each takes at most O(n) work. That collectively gives a bound of O(n3).
Therefore, the Θ(n3) worst-case runtime you've identified is asymptotically tight, so the algorithm runs in time O(n3) and has worst-case runtime Θ(n3).
Hope this helps!
It does one iteration of the list per swap. The maximum number of swaps necessary is O(n * n) for a reversed list. Doing each iteration is O(n).
Therefore the algorithm is O(n * n * n).
This is one half of the infamous Bubble Sort, which has a O(N^2). This partial sort has O(N) because the For loop goes from 1 to N. After one iteration, you will end up with the largest element at the end of the list and the rest of the list in some changed order. To be a proper Bubble Sort, it needs another loop inside this one to iterate j from 1 to N-i and do the same thing. The If goes inside the inner loop.
Now you have two loops, one inside the other, and they both go from 1 to N (sort of). You will have N * N or N^2 iterations. Thus O(N^2) for the Bubble Sort.
Now you have take your next step as a programmer: Finish writing the Bubble Sort and make it work correctly. Try it with different lengths of list a and see how long it takes. Then never use it again. ;-)
The following algorithm returns the previous larger element of an array. It is from page 11 of these notes.
// Input: An array of numeric values a[1..n]
// Returns: An array p[1..n] where p[i] contains the index of the previous
// larger element to a[i], or 0 if no such element exists.
previousLarger(a[1..n])
for (i = 1 to n)
j = i-1;
while (j > 0 and a[j] <= a[i]) j--;
p[i] = j;
return p
My homework question is: Given input sequence {a1,...,an} is a random permutation of the set {1,...,n}, what is the expected running time?
I think this requires some sort of probabilistic analysis, but I need some hints since I have only done worst-case analysis in the past. I'm trying to find a formula for the cost of the j-loop for a given i (1 + the number of times we do operation j--), then sum that formula up from 1 to n.
What does "expected" mean? I don't really know how to interpret this.
Building on #Heuster's answer:
1) You know that the answer is between O(n) and O(n^2). This is just to check the final result.
2) The expected number of steps for element i would indeed be:
sum_{k=1}^i 1 / (k+1)
= O(log i)
3) You have to sum all those number over i. This gives you:
sum_{i=1}^n O(log i)
= O(n log n)
What I did is not rigorous at all but you can prove derive it. O(n log n) is between O(n) and O(n^2) so it seems a good candidate :)
For and arbitrary index i, what is the chance that a[i-1] > a[i] (in other words, the inner while loop will take one step)? That one is easy: all elements in a are different, so P(a[i-1] > a[i]) = P(a[i] > a[i-1]) = 1/2.
Now, look at the case that the inner while loop would need to take two steps. That is, a[i-2] > a[i] > a[i-1]. This is exactly one of the 6 permutations of 3 elements, so the chance is 1 / 3! = 1 / 6.
Let's generalize this and assume that the inner while loop would need to take k steps. We consider the sublist a[i-k], a[i-k+1], ..., a[i]. We know that a[i-k] is the maximum element of this sublist and a[i] the second largest (otherwise, the inner loop would have stopped sooner). The order of the elements in between is irrelevant. The chance that we take k steps is thus 1 / (k + 1) * 1 / k = 1 / (k * (k + 1)). Note that this indeed degeneralizes to 1/2 for k = 1 and 1/6 for k = 2.
The chance that no element before a[i] is larger is simply 1 / i (a[i] is the maximum element in that sublist). In that case, the inner loop would need i steps.
The expected number of steps for element i would be (sum of probability times value):
Sum[{k, 1, i} k * 1 / ((k * (k + 1))] + i / i
= Sum[{k, 1, i} 1 / (k + 1)] + 1
= H_{i+1}
where H_{i} is the ith harmonic number, which is the discrete variant of log i. That is, the number of steps for element i is Θ(i).
What is remaining now is sum over all i to find the expected running time. With the exact value (H_{i+1}) this doesn't lead to a nice expression, see Wolfram Alpha.
The standard way to proceed, however, is to continue with the approximated log i. Clearly, log 0 + log 1 + ... + log n-1 is less than n log n. Now, consider the last half of the sum:
log n/2 + log n/2+1 + ... + log n-1 > n/2 log n/2
= n/2 (log n - log 2)
= Ω(n log n)
Therefore, the expected running time is Θ(n log n).