Here's the code:
y = 0
for j=0 to n:
for k=0 to (j*n):
y+=2
My logic is that the inner for loop will have this summation given the known solution of sum of i from 0 to n which n(n+1)/2:
(j*n)(j*n + 1)/2 #in this case, j*n is what we're summing to
Then, this inner loop would be looped from j=0 to n, which by that logic allows me to sum that from 0 to n:
( (n(n+1)/2) * n)((n(n+1)/2) * n + 1) / 2
Where I subbed j for (n(n+1)/2). After doing the multiplications I end up with
O(n^6)
I can't tell if my logic is sound or if I'm missing something because that number seems big. Thanks.
We can make a back of the envelope calculation.
j is ranging from 0 to n. So, the highest number for j is n. That is the absolute worst case for the inner loop.
So, the absolute worst case for the inner loop is if j == n, in which case the loop has j * n == n * n == n² iterations.
Meaning, the inner loop will in the absolute worst case have n² iterations. The outer loop, in turn, has n iterations, which means that our over-estimated, absolute worst-case upper bound is O(n³). It can't be worse than that. In fact, we have over-estimated by assuming that j * n == n², so we know it must definitely be less than n³.
Now, we can try to find an even more exact bound. In fact, we can actually find an exact number of iterations, we don't even need Bachmann-Landau notation.
Under the assumption that the loop bounds are exclusive, the statement in the inner loop will be executed (n³ - n²) / 2 times, and y will be n³ - n². (Says Wolfram Alpha.)
Related
Looking at the code below:
Algorithm sort
Declare A(1 to n)
n = length(A)
for i = 1 to n
for j = 1 to n-1 inclusive do
if A[i-1] > A[i] then
swap( A[i-1], A[i] )
end if
next j
next i
I would say that there are:
2 loops, both n, n*n = n^2 (n-1 truncated to n)
1 comparison, in the j loop, that will execute n^2 times
A swap that will execute n^2 times
There are also 2n additions with the loops, executing n^2 times, so 2n^2
The answers given in a mark scheme:
Evaluation of algorithm
Comparisons
The only comparison appears in the j loop.
Since this loop will iterate a total of n^2
times, it will execute
exactly n^2
Data swaps
There may be a swap operation carried out in the j loop.
Swap( A[i-1], A[i] ) Each of these will happen n^2 times.
Therefore there are 2n^2 operation carried out within the j loop
The i loop has one addition operation incrementing i which happens n
times
Adding these up we the number of addition operations which is 2n^2 +
n
As n gets very big then n^2 will dominate therefore it is O(n^2)
NOTE: Calculations might include assignment operations but these will not affect overall time so ignore
Marking overview:
1 mark for identifying i loop will execute n times.
1 mark for identifying j loop will execute 2n^2 times Isn't this meant to be n*n = n^2? For i and j
1 mark for correct number of calculations 2n^2 + n Why is this not
+2n?
1 mark for determining that the order will be dominated by n^2 as n
gets very big giving O(n^2) for the algorithm
Edit: As can be seen from the mark scheme, I am expected to count:
Loop numbers, but n-1 can be truncated to n
Comparisons e.g. if statements
Data swaps (counted as one statement, i.e. arr[i] = arr[i+1], temp = arr[i], etc. are considered one swap)
Calculations
Space - just n for array, etc.
Could someone kindly explain how these answers are derived?
Thank you!
Here's my take on the marking scheme, explicitly marking the operations they're counting. It seems they're counting assignments (but conveniently forgetting that it takes 2 or 3 assignments to do a swap). That explains why they count increment but not the [i-1] indexing.
Counting swaps
i loop runs n times
j loop runs n-1 times (~n^2-n)
swap (happens n^2 times) n^2
Counting additions (+=)
i loop runs n times
j loop runs n-1 times (~n^2)
increment j (happens n^2 times) n^2
increment i (happens n times) n
sum: 2n^2 + n
for i = 0 to n do
for j = n to 0 do
for k = 1 to j-i do
print (k)
I'm wondering about the lower-bound runtime of the above code. In the notes I am reading it explains the lower bound runtime to be
with the explanation;
To find the lower bound on the running time, consider the values of i, such that 0 <= i <= n/4 and values of j, such that 3n/4 <= j <= n. Note that for each of the n^2/16 different combinations of i and j, the innermost loop executes at least n/2 times.
Can someone explain where these numbers came from? They seem to be arbitrary to me.
There are n iterations of the first loop and for each of them n iterations of the second loop. In total these are n^2 iterations of the second loop.
Now if you only consider the lower quarter of possible values for i, then you have n^2/4 iterations of the inner loop left. If you also only consider the upper quarter of values for j then you have n^2/16 iterations of the inner loop left.
For each iteration of these constrained cases you have j-i >= 3n/4-n/4 = n/2 and therefore the most inner loop is iterated at least n/2 times for each of these n^2/16 iterations of the outer loops. Therefore the full number of iterations of the most inner loop is at least n^2/16*n/2.
Because we considered only specific iterations, the actual number of iterations is higher and this result is a lower bound. Therefore the algorithm is in Omega(n^3).
The values are insofar arbitrary that you could use many others. But these are some simple ones which make the argument j-i >= 3n/4-n/4 = n/2 possible. For example if you took only the lower half of the i iterations and the upper half of the j iterations, then you would have j-i >= n/2-n/2 = 0, leading to Omega(0), which is not interesting. If you took something like lower tenth and upper tenth it would still work, but the numbers wouldn't be as nice.
I can't really explain the ranges from your book, but if you attempt to proceed via the methodology below, I hope it would become more clear to find what is it that you are looking for.
The ideal way for the outer loop (index i) and the inner loop (index j) is as follows, since j - i >= 1 should be sustained, so that the innermost loop would execute (at least once in every case).
The performed decomposition was made because the range of j from i to 0 is ignored by the innermost loop.
for ( i = 0 ; i < n ; i ++ ) {
for ( j = n; j > i ; j -- ) {
for ( k = 1; k <= j - i ; k ++ ) {
printf(k);
}
}
}
This algorithm's order of growth complexity T(n) is:
Hence, your algorithm does iterate more than the algorithm above (j goes from n to 0).
Using Sigma Notation, you can do this:
Where c represent the execution cost of print(k), and c' is the execution cost of the occurring iterations that don't involve the innermost loop.
Refreshing up on algorithm complexity, I was looking at this example:
int x = 0;
for ( int j = 1; j <= n; j++ )
for ( int k = 1; k < 3*j; k++ )
x = x + j;
I know this loops ends up being O(n^2). I'm believing inner loop is executed 3*n times( 3(1+2+...n) ), and the outer loop executes n times. So, O(3n*n) = O(3n^2) = O(n^2).
However, the source I'm looking at expands the execution of the inner loop to: 3(1+2+3+...+n) = 3n^2/2 + 3n/2. Can anyone explain the 3n^2/2 + 3n/2 execution times?
for each J you have to execute J * 3 iterations of internal loop, so you command x=x+j will be finally executed n * 3 * (1 + 2 + 3 ... + n) times, sum of Arithmetic progression is n*(n+1)/2, so you command will be executed:
3 * n * (n+1)/2 which is equals to (3*n^2)/2 + (3*n)/2
but big O is not how much iterations will be, it is about assymptotic measure, so in expression 3*n*(n+1)/2 needs to remove consts (set them all to 0 or 1), so we have 1*n*(n+0)/1 = n^2
Small update about big O calculation for this case: to make big O from the 3n(n+1)/2, for big O you can imagine than N is infinity, so:
infinity + 1 = infinity
3*infinity = infinity
infinity/2 = infinity
infinity*infinity = infinity^2
so you after this you have N^2
The sum of integers from 1 to m is m*(m+1)/2. In the given problem, j goes from 1 to n, and k goes from 1 to 3*j. So the inner loop on k is executed 3*(1+2+3+4+5+...+n) times, with each term in that series representing one value of j. That gives 3n(n+1)/2. If you expand that, you get 3n^2/2+3n/2. The whole thing is still O(n^2), though. You don't care if your execution time is going up both quadratically and linearly, since the linear gets swamped by the quadratic.
Big O notation gives an upper bound on the asymptotic running time of an algorithm. It does not take into account the lower order terms or the constant factors. Therefore O(10n2) and O(1000n2 + 4n + 56) is still O(n2).
What you are doing is try to count the number the number of operations in your algorithm. However Big O does not say anything about the exact number of operations. It simply provides you an upper bound on the worst case running time that may occur with an unfavorable input.
The exact precision of your algorithm can be found using Sigma notation like this:
It's been empirically verified.
I can probably figure out part b if you can help me do part a. I've been looking at this and similar problems all day, and I'm just having problems grasping what to do with nested loops. For the first loop there are n iterations, for the second there are n-1, and for the third there are n-1.. Am I thinking about this correctly?
Consider the following algorithm,
which takes as input a sequence of n integers a1, a2, ..., an
and produces as output a matrix M = {mij}
where mij is the minimum term
in the sequence of integers ai, a + 1, ..., aj for j >= i and mij = 0 otherwise.
initialize M so that mij = ai if j >= i and mij = 0
for i:=1 to n do
for j:=i+1 to n do
for k:=i+1 to j do
m[i][j] := min(m[i][j], a[k])
end
end
end
return M = {m[i][j]}
(a) Show that this algorithm uses Big-O(n^3) comparisons to compute the matrix M.
(b) Show that this algorithm uses Big-Omega(n^3) comparisons to compute the matrix M.
Using this face and part (a), conclude that the algorithm uses Big-theta(n^3) comparisons.
In part A, you need to find an upper bound for the number of min ops.
In order to do so, it is clear that the above algorithm has less min ops then the following:
for i=1 to n
for j=1 to n //bigger range then your algorithm
for k=1 to n //bigger range then your algorithm
(something with min)
The above has exactly n^3 min ops - thus in your algorithm, there are less then n^3 min ops.
From this we can conclude: #minOps <= 1 * n^3 (for each n > 10, where 10 is arbitrary).
By definition of Big-O, this means the algorithm is O(n^3)
You said you can figure B alone, so I'll let you try it :)
hint: the middle loop has more iterations then for j=i+1 to n/2
For each iteration of outer loop inner two nested loop would give n^2 complexity if i == n. Outer loop will run for i = 1 to n. So total complexity would be a series like: 1^2 + 2^2 + 3^2 + 4^2 + ... ... ... + n^2. This summation value is n(n+1)(2n+1)/6. Ignoring lower order terms of this summation term ultimately the order would be O(n^3)
I'm trying to figure out how to give a worst case time complexity. I'm not sure about my analysis. I have read nested for loops big O is n^2; is this correct for a for loop with a while loop inside?
// A is an array of real numbers.
// The size of A is n. i,j are of type int, key is
// of type real.
Procedure IS(A)
for j = 2 to length[A]
{
key = A[ j ]
i = j-1
while i>0 and A[i]>key
{
A[i+1] = A[i]
i=i-1
}
A[i+1] = key
}
so far I have:
j=2 (+1 op)
i>0 (+n ops)
A[i] > key (+n ops)
so T(n) = 2n+1?
But I'm not sure if I have to go inside of the while and for loops to analyze a worse case time complexity...
Now I have to prove that it is tightly bound, that is Big theta.
I've read that nested for loops have Big O of n^2. Is this also true for Big Theta? If not how would I go about finding Big Theta?!
**C1= C sub 1, C2= C sub 2, and no= n naught all are elements of positive real numbers
To find the T(n) I looked at the values of j and looked at how many times the while loop executed:
values of J: 2, 3, 4, ... n
Loop executes: 1, 2, 3, ... n
Analysis:
Take the summation of the while loop executions and recognize that it is (n(n+1))/2
I will assign this as my T(n) and prove it is tightly bounded by n^2.
That is n(n+1)/2= θ(n^2)
Scratch Work:
Find C1, C2, no є R+ such that 0 ≤ C1(n^2) ≤ (n(n+1))/2 ≤ C2(n^2)for all n ≥ no
To make 0 ≤ C1(n) true, C1, no, can be any positive reals
To make C1(n^2) ≤ (n(n+1))/2, C1 must be ≤ 1
To make (n(n+1))/2 ≤ C2(n^2), C2 must be ≥ 1
PF:
Find C1, C2, no є R+ such that 0 ≤ C1(n^2) ≤ (n(n+1))/2 ≤ C2(n^2) for all n ≥ no
Let C1= 1/2, C2= 1 and no = 1.
show that 0 ≤ C1(n^2) is true
C1(n^2)= n^2/2
n^2/2≥ no^2/2
⇒no^2/2≥ 0
1/2 > 0
Therefore C1(n^2) ≥ 0 is proven true!
show that C1(n^2) ≤ (n(n+1))/2 is true
C1(n^2) ≤ (n(n+1))/2
n^2/2 ≤ (n(n+1))/2
n^2 ≤ n(n+1)
n^2 ≤ n^2+n
0 ≤ n
This we know is true since n ≥ no = 1
Therefore C1(n^2) ≤ (n(n+1))/2 is proven true!
Show that (n(n+1))/2 ≤ C2(n^2) is true
(n(n+1))/2 ≤ C2(n^2)
(n+1)/2 ≤ C2(n)
n+1 ≤ 2 C2(n)
n+1 ≤ 2(n)
1 ≤ 2n-n
1 ≤ n(2-1) = n
1≤ n
Also, we know this to be true since n ≥ no = 1
Hence by 1, 2 and 3, θ(n^2 )= (n(n+1))/2 is true since
0 ≤ C1(n^2) ≤ (n(n+1))/2 ≤ C2(n^2) for all n ≥ no
Tell me what you thing guys... I'm trying to understand this material and would like y'alls input!
You seem to be implementing the insertion sort algorithm, which Wikipedia claims is O(N2).
Generally, you break down components based off your variable N rather than your constant C when dealing with Big-O. In your case, all you need to do is look at the loops.
Your two loops are (worse cases):
for j=2 to length[A]
i=j-1
while i > 0
/*action*/
i=i-1
The outer loop is O(N), because it directly relates to the number of elements.
Notice how your inner loop depends on the progress of the outer loop. That means that (ignoring off-by-one issues) the inner and outer loops are related as follows:
j's inner
value loops
----- -----
2 1
3 2
4 3
N N-1
----- -----
total (N-1)*N/2
So the total number of times that /*action*/ is encountered is (N2 - N)/2, which is O(N2).
Looking at the number of nested loops isn't the best way to go about getting a solution. It's better to look at the "work" that's being done in the code, under a heavy load N. For example,
for(int i = 0; i < a.size(); i++)
{
for(int j = 0; j < a.size(); j++)
{
// Do stuff
i++;
}
}
is O(N).
A function f is in Big-Theta of g if it is both in Big-Oh of g and Big-Omega of g. The worst case happens when the data A is monotonically decreasing function. Then, for every iteration of the outer loop, the while loop executes. If each statement contributed a time value of 1, then the total time would be 5*(1 + 2 + ... + n - 2) = 5*(n - 2)*(n - 1) / 2. This gives a quadratic dependence on the data. However, if the data A is a monotonically increasing sequence, the condition A[i] > key will always fail. Thus the outer loop executes in constant time, N - 3 times. The best case of f then has a linear dependence on the data. For the average case, we take the next number in A and find its place in the sorting that has previously occurred. On average, this number will be in the middle of this range, which implies the inner while loop will run half as often as in the worst case, giving a quadratic dependence on the data.
Big O (basically) about how many times the elements in your loop will be looked at in order to complete a task.
For example, a O(n) algorithm will iterate through every element just once.
A O(1) algorithm will not have to iterate through every element at all, it will know exactly where in the array to look because it has an index. An example of this is an array or hash table.
The reason a loop inside of a loop is O(n^2) is because every element in the loop has to be iterated over itself ^2 times. Changing the type of the loop has nothing to do with it since it's about # of iterations essentially.
There are approaches to algorithms that will allow you to reduce the number of iterations you need. An example of these are "divide & conquer" algorithms like Quicksort, which if I recall correctly is O(nlog(n)).
It's tough to come up with a better alternative to your example without knowing more specifically what you're trying to accomplish.