I'am taking a course in complexity theory,and so it's need some mathematical background which i have a problem,.
so while i'am trying to do some practicing i stuck in the bellow example
1) for (i = 1; i < n; i++) {
2) SmallPos = i;
3) Smallest = Array[SmallPos];
4) for (j = i+1; j <= n; j++)
5) if (Array[j] < Smallest) {
6) SmallPos = j;
7) Smallest = Array[SmallPos]
}
8) Array[SmallPos] = Array[i];
9) Array[i] = Smallest;
}
Thus, the total computing time is:
T(n) = (n) + 4(n-1) + n(n+1)/2 – 1 + 3[n(n-1) / 2]
= n + 4n - 4 + (n^2 + n)/2 – 1 + (3n^2 - 3n) / 2
= 5n - 5 + (4n2 - 2n) / 2
= 5n - 5 + 2n^2 - n
= 2n^2 + 4n - 5
= O(n^2)
and what i don't understand or confused about line 4 analyzed to n(n+1)/2 – 1,
and line 5 3[n(n-1) / 2].
i knew that the sum of positive series is =n(first+last)/2 ,but when i tried to calculate it as i understand it it gives me different result.
i calculate for line no 4 so it shoulb be =n((n-1)+2)/2 according to n(first+last)/2 ,but here it's n(n+1)/2 – 1.
and same for 3[n(n-1) / 2].....i don't understand this too
also here's what is written in the analysis it could help if anyone can explain to me,
Statement 1 is executed n times (n - 1 + 1); statements 2, 3, 8, and 9 (each representing O(1) time) are executed n - 1 times each, once on each pass through the outer loop. On the first pass through this loop with i = 1, statement 4 is executed n times; statement 5 is executed n - 1 times, and assuming a worst case where the elements of the array are in descending order, statements 6 and 7 (each O(1) time) are executed n - 1 times.
On the second pass through the outer loop with i = 2, statement 4 is executed n - 1 times and statements 5, 6, and 7 are executed n - 2 times, etc. Thus, statement 4 is executed (n) + (n-1) +... + 2 times and statements 5, 6, and 7 are executed (n-1) + (n-2) + ... + 2 + 1 times. The first sum is equal to n(n+1)/2 - 1, and the second is equal to n(n-1)/2.
Thus, the total computing time is:
T(n) = (n) + 4(n-1) + n(n+1)/2 – 1 + 3[n(n-1) / 2]
= n + 4n - 4 + (n^2 + n)/2 – 1 + (3n^2 - 3n) / 2
= 5n - 5 + (4n2 - 2n) / 2
= 5n - 5 + 2n^2 - n
= 2n^2 + 4n - 5
= O(n^2)
here's the link for the file containing this example:
http://www.google.com.eg/url?sa=t&rct=j&q=Consider+the+sorting+algorithm+shown+below.++Find+the+number+of+instructions+executed+&source=web&cd=1&cad=rja&ved=0CB8QFjAA&url=http%3A%2F%2Fgcu.googlecode.com%2Ffiles%2FAnalysis%2520of%2520Algorithms%2520I.doc&ei=3H5wUNiOINDLswaO3ICYBQ&usg=AFQjCNEBqgrtQldfp6eqdfSY_EFKOe76yg
line 4: as the analysis says, it is executed n+(n-1)+...+2 times. This is a sum of (n-1) terms. In the formula you use, n(first+last)/2, n represents the number of terms. If you apply the formula to your sequence of n-1 terms, then it should be (n-1)((n)+(2))/2=(n²+n-2)/2=n(n+1)/2-1.
line 5: the same formula can be used. As the analysis says, you have to calculate (n-1)+...+1. This is a sum of n-1 terms, with the first and last being n-1 and 1. The sum is given by (n-1)(n-1+1)/2. The factor 3 is from the 3 lines (5,6,7) that are each being done (n-1)(n)/2 times
Related
I do not understand why there is no factor of √n in time complexity.
Here is the code
def sieve_of_erastothenes(max):
flags = [True] * (max+1)
num = 2
while num*num <= max:
if flags[num] is True: # if it is a prime number
cross_off(flags, num)
num += 1
return [i for i in range(2,len(flags)) if flags[i] is True]
def cross_off(flags, prime):
# no point in crossing anything before prime*prime
for num in range(prime*prime, len(flags), prime):
flags[num] = False
num += prime
The outer loop (num*num <= max) runs for √n times. If flag has been set, inner loop runs for n/p times. Summation of n/p where p is only prime gives n*loglogn
So time complexity should be O(√n * n * loglogn). But everywhere I read, it is given as O(nloglogn)
I think I have figured it out with the help of comments from Matt Timmermans and rici. I had 2 misunderstandings
The inner loop for a given prime runs n / prime times.
The summation n/2 + n/3 + n/5 + n/7 + n/11 .... is already taking into account the outer √n factor. The summation itself is because of the outer loop.
The failure case of if condition IS being counted, but it is being overpowered by the other factor. So we can write it as,
num => 2 3 4 5 6 7 8 9 10 11
sum => n/2 + n/3 + 1 + n/5 + 1 + n/7 + 1 + 1 + 1 + n/11...
sum => (n/2 + n/3 + n/5 ...) + (1 + 1 + 1 + 1....)
sum => (nloglogn) + (n) # Worst case. 1+1+1.. will always be less than n
O(nloglogn + n) = O(nloglogn)
I'm a self-taught computer science student.
Now I'm reading CLRS and I did the 2.2-2 exercise, it's about Selection Sort.
First array subscript is 1.
The pseudocode that I wrote is:
FOR i=1 to A.length - 1
FOR j=i+1 to A.length
IF A[j] < A[i]
new_index=j
IF new_index > i
tmp = A[i]
A[i] = A[new_index]
A[new_index] = A[i]
My reasoning is:
the first loop test is executed n times (1, 2, 3 ... n). When i becomes n the loop stops. So line 5 is executed (n-1) times and so on.
Then I think that the second loop test is executed (n^2 + n - 2)/2 times. When initial j = 2, it assumes: 2, 3, 4, 5 ... (n + 1), the loop test is executed n times, when j = 3 the loop test is executed (n - 1) times and so on. So when j = n the loop test is executed 2 times.
For this reason the second loop test is executed:
2 + 3 + 4 + 5 + ... + n = [(n-1)*(n+2)] / 2 = (n^2 + n - 2) / 2.
So the inner if of the second loop is executed: 1 + 2 + 3 + 4 + ... + (n-1) = [(n-1)*n] / 2.
Before this written I read a lot of hypothetical solutions but no one is equal to mine. So I'd like to know if my reasoning is wrong.
I hope that I written all details in good form.
The pseudocode is right and your analysis is along the right lines, but your solution has some mistakes in its reasoning.
Some Tips
Then I think that the second loop test is executed (n^2 + n - 2)/2 times
It's executed n(n-1)/2 times. See the solution below.
When initial j = 2, it assumes: 2, 3, 4, 5 ... (n + 1), the loop test is executed n times,
Remember the code is: FOR j=i+1 to A.length, so when the inner for loop starts at j = 2, it goes up to j = A.length, i.e. it goes up to j = n. In other words, it executes repeatedly for j = 2, 3, 4, ..., n and so executes a total of n - 1 times, not n times!
For this reason the second loop test is executed: 2 + 3 + 4 + 5 + ... + n = [(n-1)*(n+2)] / 2 = (n^2 + n - 2) / 2. So the inner if of the second loop is executed: 1 + 2 + 3 + 4 + ... + (n-1) = [(n-1)*n] / 2.
Suppose the body of the if statement of the second loop is always executed (the if condition is always true), then it should execute the same number of times as the second loop test. So I don't follow your reasoning here. And as for the summation, to find number of iterations, you need to add up the following:
j = 1: executes (n-1) times
j = 2: executes (n-2) times
j = 3: executes (n-3) times
...
j = n: executes 1 time
So you need to add up: (n-1) + (n-2) + (n-3) + ... + 1 = n(n-1)/2
Solution
The inner for loop executes exactly C(n, 2) = n(n-1)/2 times. The loop produces all pairs (i,j) such that 1 <= i < j <= n – which from combinatorics is n-choose-2, or C(n, 2).
The if statement body (new_index=j) executes at most n(n-1)/2 times – we don't know exactly how many because we don't know what the values in A are – so we don't know how many times A[j] < A[i]. It could execute zero times (consider the case when the array is sorted). But nevertheless, we find an upper bound by assuming the condition is always true. In this worst case, the if statement's body executes the same number of times as the inner for loop – which from the previous bullet point is n(n-1)/2
By similar reasoning, the other if statement body (the one performing the swap) executes at most n times.
So overall, the algorithm performs exactly n(n-1)/2 iterations of the inner for loop iterations having ϴ(1) work – and exactly n iterations of the if statement performing a swap in ϴ(1) time. This gives a time complexity of ϴ(n^2).
I've got the question about the time complexity algorithm and asymptotic growth.
The pseudo code of question is
1: function NAIVE(x,A)
2: answer = 0
3: n= length of A
4: for I from - to n do
5: aux = 1
6. for j from 1 to I do
7: aux = aux*x
8: answer = answer + aux * A[I]
9. return answer
I have to find upperbound with O-notation and lowerbound witn Ω-notation.
I got the time complexity f(n) = 5n^2 + 4n + 8 and g(n) = n^2.
My question is I'm not too sure about the running time of line number 6 to 8.
I got constant 2 and time n+1 for line number 4 and constant 1 and time 1 for line 5.
I'm stuck on after that. I tried it and I got constant 2 and time n^2 + 1 for line 6, because it runs in the for loop (for loop and for loop) so I thought its n^2+1. Is that correct?
And for line number 8 its has 3 constants and run time is n^2. is this correct? I'm not too sure about the line number 8. This is how I got f(n) = 5n^2 + 4n + 8!
Please help me out to complete this question!
I wonder my work is right or not!
Thank you
Let's go through this step by step.
The complexity of line 7 T7 = 1.
The complexity of the surrounding for will be T6(I) = I * T7 = I.
The complexity of line 5 T5 = 1.
The complexity of line 8 T8 = 1.
The complexity of the surrounding for (assuming that - stands for 0) is
T4(n) = Sum{I from 0 to n} (T5 + T6(I) + T8)
= Sum{I from 0 to n}(2 + I)
= Sum{I from 0 to n}(2) + Sum{I from 0 to n}(I)
= (n + 1) * 2 + (n+1)/2 * (0 + n)
= 2n + 2 + n^2/2 + n/2
= 1/2 n^2 + 5/2 n + 2
The complexity of the remaining lines is T2 = T3 = T9 = 1.
The complexity of the entire algorithm is
T(n) = T2 + T3 + T4(n) + T9
= 1 + 1 + 1/2 n^2 + 5/2 n + 2 + 1
= 1/2 n^2 + 5/2 n + 5
This runtime is in the complexity classes O(n^2) and Ω(n^2).
Here's the pseudocode:
Baz(A) {
big = −∞
for i = 1 to length(A)
for j = 1 to length(A) - i + 1
sum = 0
for k = j to j + i - 1
sum = sum + A(k)
if sum > big
big = sum
return big
So line 3 will be O(n) (n being the length of the array, A)
I'm not sure what line 4 would be...I know it decreases by 1 each time it is run, because i will increase.
and I can't get line 6 without getting line 4...
All help is appreciated, thanks in advance.
Let us first understand how first two for loops work
for i = 1 to length(A)
for j = 1 to length(A) - i + 1
First for loop will run from 1 to n(length of Array A) and the second for loop will depend on value of i. SO when i = 1 second for loop will run for n times..When i increments to 2 your second for loop will run for (n-1) time ..so it will go on till 1.
So your second for loop will run as follows:
n + (n - 1) + (n - 2) + (n - 3) + .... + 1 times...
You can use following formula: sum(1 to n) = N * (N + 1) / 2 which gives (N^2 + N)/2 So we have Big oh for these two loops as
O(n^2) (Big Oh of n square )
Now let us consider third loop also...
Your third for loop looks like this
for k = j to j + i - 1
But this actually means,
for k = 0 to i - 1 (you are just shifting the range of values by adding/subtracting j but number of times the loop should run will not change, as difference remains same)
So your third loop will run from 0 to 1(value of i) for first n iterations of second loop then it will run from 0 to 2(value of i) for first (n - 1) iterations of second loop and so on..
So you get:
n + 2(n-1) + 3(n-2) + 4(n-3).....
= n + 2n - 2 + 3n - 6 + 4n - 12 + ....
= n(1 + 2 + 3 + 4....) - (addition of some numbers but this can not be greater than n^2)
= `N(N(N+1)/2)`
= O(N^3)
So your time complexity will be N^3 (Big Oh of n cube)
Hope this helps!
Methodically, you can follow the steps using Sigma Notation:
Baz(A):
big = −∞
for i = 1 to length(A)
for j = 1 to length(A) - i + 1
sum = 0
for k = j to j + i - 1
sum = sum + A(k)
if sum > big
big = sum
return big
For Big-O, you need to look for the worst scenario
Also the easiest way to find the Big-O is to look into most important parts of the algorithm, it can be loops or recursion
So we have this part of the algorithm consisting of loops
for i = 1 to length(A)
for j = 1 to length(A) - i + 1
for k = j to j + i - 1
sum = sum + A(k)
We have,
SUM { SUM { i } for j = 1 to n-i+1 } for i = 1 to n
= 1/6 n (n+1) (n+2)
= (1/6 n^2 + 1/6 n) (n + 2)
= 1/6 n^3 + 2/6 2 n^2 + 1/6 n^2 + 2/6 n
= 1/6 n^3 + 3/6 2 n^2 + 2/6 n
= 1/6 n^3 + 1/2 2 n^2 + 1/3 n
T(n) ~ O(n^3)
Hobbyist here, I'm reading CLS and I'm having trouble with the analysis stuff. It's mostly that I'm not very good at figuring out summations from different kinds of loops and stuff. Anyway, I wrote up an explanation and some example code to show you what I mean. I think it's right, but I'm getting mixed messages from a lot of different sources. First off, here's some example code in C++, you'll notice that it contains a reference to an integer n and several incrementations of it. Those were put there to count how many times each statement took as a way to check it.
int *intSelSort(int a[], int size, int &n){
int i, k, in;
for(int j = 0; ++n && j < size -1; j++){
k = a[j];
n++;
for(int i = j+1; ++n && i < size; i++)
if(++n && a[i] < k){
k = a[i];
in = i;
}
a[in] = a[j];
a[j] = k;
n+=2;
}
return a;
}
This example, when used with an array of length 5 leaves n with a value of 41. The counts are all removed save for the one in the innermost for loop's condition, the count is 14. That's important for later.
Anyway, CLS does things a little differently, starting with the fact that it uses pseudocode and array indices beginning at 1. I wrote some to mirror their work
//[1][2][3][4][5]
//Highest j is 4, i is 5
int k, index; //O(1)
n = A.Size //O(1)
for j = 1, j < n, j++ do //O(N)
k = A[j] //O(N - 1)
for i = j + 1, i <= n, i++ do //O((n^2 + n - 2)/2)
if(A[i] < k) //O((n^2 - n)/2)
k = A[i] //O(c)
index = i //O(c)
A[index] = A[j] //O(N - 1)
A[j] = k //O(N - 1)
(If n = 5, first it makes 5 comparisons, counting the extra)
(Then 4, 3, 2, 1)
2, 3, 4, 5, 6 (n - 0) comparisons
3, 4, 5, 6 (n - 1) comparisons
4, 5, 6 (n - 2) comparisons
5, 6 (n - 3) comparisons
[j = 5, i = 6], exits for loop, no comparison
(n - 0) + (n - 1) + (n - 2) + (n - 3) = 4n - 6 comparisons, 14.
Or more generally,
sum(n - N)_(0 -> N = n - 2) =
sum(n)(0 -> N = n - 2) - sum(N)(0 -> N = n - 2) =
n(n - 2 + 1) - ((n - 1)(n - 2))/2 = (n^2 + n - 2)/2
If n = 5, this yields 14. Exactly the number of steps in the test.
For the if in that for loop,
sum(n - N - 1)_(0 - > N = n - 2)
= (n^2 + n - 2)/2 - sum(1)_(0 -> N = n - 2)
= (n^2 + n - 2)/2 - (n - 1) = (n^2 - n)/2, or in this case, 10 compares
Testing the inner most if statement alone also yielded 10 compares.
C is dependent on whether the if statement is ever true.
Case one: The list is already sorted, the if statement will never be
true, so the statements in there will never be called.
Case two: The list is reverse sorted, the if statements are called
floor(N/2) times?
5 4 3 2 1
1 4 3 2 5
1 2 3 4 5
9 8 7 6 5 4 3 2 1
1 8 7 6 5 4 3 2 9
1 2 7 6 5 4 3 8 9
1 2 3 6 5 4 7 8 9
1 2 3 4 5 6 7 8 9
Are there other cases in which the if statement is true more times?
Is there a way to prove that the if statement's sub statements are
of time complexity n or less, so I can ignore them? Are they O(n)?
Did I make any drastic errors here? I've seen websites with different
summation formulas for this, so I'm not sure.
I think your analysis is correct.
To your question at the end:
For sequence n, 1, 2, ..., n-1 the if statement is n-1 times true (and n-1 times two elements are swaped.)
for the reversed order it is not floor(N/2). It is
N-1 + N-3 + N-5 + ... + 0 = Σi=0,..⌊N/2⌋ N-1-2⋅i ≈ ((N-1) ⋅ N)/2
and ⌊N/2⌋ times two elements are swaped.
But it is not importend how often the if statement is true, due to the O(1) complexity of the actions in the body of the if statement. Your analysis is very precise. This is not necessary, if you arte using the big-O notation. Let me simplify your example:
int k, index; // O(1)
n = A.Size // O(1)
for j = 1, j < n, j++ do // O(N) - runs n-1 times
k = A[j] // O(1) - takes only 1 step but is executet O(N) times.
for i = j + 1, i <= n, i++ do // O(N) - the first run takes n-1 steps -+
if(A[i] < k) // O(1) - this can be evaluated in 1 step |- together 1 to 3 steps. So the whole if block is in O(1).
k = A[i] // O(1) - this is at worst 1 more step -+
index = i // O(1) - this also
A[index] = A[j] // O(1) - one step executet O(N) times
A[j] = k // O(1) - one step executet O(N) times
Notice: This are not overall complexitys, but only the complexity of each item of the code by it self.
Now you only have to multiply the nested complexitys and you get an overall complexity of
O(N) ⋅ O(N) ⋅ O(1) = O(N2).
If you are interested in a more precise analysis you can reduce the complexity of the nested for loop to O(N-i) and sum this up for i from 1 to N-1.
O(N-1) + O(N-2) + ... + O(N-N+1) = O((N-1)⋅N/2) = O(N2)
Notice also tht the big-O notations and the complexity of algorithms can not directly translated into execution time or number of steps an algorithm exactly will do.