Given the following pseudo code
D(A) // A is an array of numbers
U_Size = 1
for i=2 to length(A)
U=?
for j=1 to U_Size
if A[j]=A[i]
then U = FALSE
j = U_size
if U = TRUE
then U_Size = U_Size + 1
A[U_Size]=A[i]
return U_Size
What will be the best opton to replace the "?" in line 4? ( U=? )
and what exactly does this program do - Answered
How should I determine the run-time & space complexity of this program
MY ANSWER: In line 4, I initialized U -> U = TRUE and figured that the program arranges all of the different elements of the array in the beginning of the arry and returns the amount of different elements
The Question remained unanswered is: How should I determine the run-time & space complexity of this program (2)
If you are not familiar with the Big O notation, I would suggest to read about it because in computer science we use that to represent the time and space complexity.
Let the length of input array is n.
The time complexity of this code will be O(n2) in the worst-case and the worst-case occurs for an array of all distinct numbers. For an input array of all distinct numbers, the if condition if A[j] = A[i] is always false, so the inner looop of j loops from 1 to U_size for every i and U_size is increased by 1 everytime.If it is still not clear then you can understand it using math for an array of all distinct numbers.
For i = 2, and U_size = 1, the inner loop of j runs from 1 to 1 i.e 1 time.
For i = 3, and U_size = 2, the inner loop of j runs from 1 to 2 i.e 2 times.
For i = 4, and U_size = 3, the inner loop of j runs from 1 to 3 i.e 3 times.
For i = 5, and U_size = 4, the inner loop of j runs from 1 to 4 i.e 4 times.
.
.
.
.
.
For i = n (length of array A), and U_size = n-1, the inner loop of j runs from 1 to n-1 i.e n-1 times.
So, if you sum up the running times for all the iterations of i,
1 + 2 + 3 + 4 + ... + n-1, you get n*(n-1)/2 which is ~ O(n2).
Now, space complexity. You have used the array A itself for assigning after the j loop. If you do not take into account the input array for reporting space, then you get O(1) as space complexity.
If you store a separate array for assigning the element i.e A[U_Size]=A[i] or you consider the input array itself for space, then you would say space complexity as O(n).
Related
This question already has answers here:
Big O, how do you calculate/approximate it?
(24 answers)
Closed 5 years ago.
I'm studying algorithm's complexity and I'm still not able to determine the complexity of some algorithms ... Ok I'm able to figure out basic O(N) and O(N^2) loops but I'm having some difficult in routines like this one:
// What is time complexity of fun()?
int fun(int n)
{
int count = 0;
for (int i = n; i > 0; i /= 2)
for (int j = 0; j < i; j++)
count += 1;
return count;
}
Ok I know that some guys can calculate this with the eyes closed but I would love to to see a "step" by "step" how to if possible.
My first attempt to solve this would be to "simulate" an input and put the values in some sort of table, like below:
for n = 100
Step i
1 100
2 50
3 25
4 12
5 6
6 3
7 1
Ok at this point I'm assuming that this loop is O(logn), but unfortunately as I said no one solve this problem "step" by "step" so in the end I have no clue at all of what was done ....
In case of the inner loop I can build some sort of table like below:
for n = 100
Step i j
1 100 0..99
2 50 0..49
3 25 0..24
4 12 0..11
5 6 0..5
6 3 0..2
7 1 0..0
I can see that both loops are decreasing and I suppose a formula can be derived based on data above ...
Could someone clarify this problem? (The Answer is O(n))
Another simple way to probably look at it is:
Your outer loop initializes i (can be considered step/iterator) at n and divides i by 2 after every iteration. Hence, it executes the i/2 statement log2(n) times. So, a way to think about it is, your outer loop run log2(n) times. Whenever you divide a number by a base continuously till it reaches 0, you effectively do this division log number of times. Hence, outer loop is O(log-base-2 n)
Your inner loop iterates j (now the iterator or the step) from 0 to i every iteration of outer loop. i takes the maximum value of n, hence the longest run that your inner loop will have will be from 0 to n. Thus, it is O(n).
Now, your program runs like this:
Run 1: i = n, j = 0->n
Run 2: i = n/2, j = 0->n/2
Run 3: i = n/4, j = 0->n/4
.
.
.
Run x: i = n/(2^(x-1)), j = 0->[n/(2^(x-1))]
Now, runnning time always "multiplies" for nested loops, so
O(log-base-2 n)*O(n) gives O(n) for your entire code
Lets break this analysis up into a few steps.
First, start with the inner for loop. It is straightforward to see that this takes exactly i steps.
Next, think about which different values i will assume over the course of the algorithm. To start, consider the case where n is some power of 2. In this case, i starts at n, then n/2, then n/4, etc., until it reaches 1, and finally 0 and terminates. Because the inner loop takes i steps each time, then the total number of steps of fun(n) in this case is exactly n + n/2 + n/4 + ... + 1 = 2n - 1.
Lastly, convince yourself this generalizes to non-powers of 2. Given an input n, find smallest power of 2 greater than n and call it m. Clearly, n < m < 2n, so fun(n) takes less than 2m - 1 steps which is less than 4n - 1. Thus fun(n) is O(n).
I am trying to calculate the number of steps executed for the following nested loop specially for asymptotic growth. Based on the number of steps I will derive the Big O for this algorithm.
def get_multiples(list):
multiple = []
for o in list:
for i in list:
multiple.append(o*i)
return multiple
The way I have calculated is as follows (list consists of large number of elements = "n"):
Assignment statement (no. of steps = 1):
multiple = []
Nested Loops:
for o in list:
for i in list:
multiple.append(o*i)
In the outer loop the variable o is assigned n times. Each time the outer loop executes, first the variable i is assigned n times, then the variables are multiplied n times and finally the list is appended n times. Therefore the no. of steps = n*(n+n+n) = 3n2
Return statement (No. of steps = 1):
return multiple
Therefore the total no. of steps = 3n2 + 2
However the correct answer is 3n2 + n +2. Apparently the execution of the outer loop takes additional n steps which is not required for the inner loop.
Can somebody explain to me what did I miss ?
It does not make a difference to complexity since it will still be O(n2)
I think that the correct way to calculate the nested loop is as follows:
The number o is assigned n times.
the number i is assigned n2 times, o*i is calculated n2 times, the append function is called n2 times.
therefore n + n2 + n2 + n2 = 3n2 + n
add it to the rest and you get 3n2 + n + 2
def get_multiples(list):
multiple = [] // 1 STEP
for o in list: // Executed n times, so n STEPS
for i in list: // Executed n times for each value of o, so n*n STEPS
multiple.append(o*i) // 1 STEP to multiply and 1 STEP to append, repeated for each pair of (o, i), so 2*n*n STEPS
return multiple // 1 STEP
Adding the above: 1 + n + n2 + 2n2 + 1 = 3n2 + n + 2
Here is the code:
int Outcome = 0;
for (int i = 0; i < N; i++)
for (int j = i+2; j = 0; j--)
Outcome += i*j;
Here's my analysis. Since the first line is an assignment statement, this takes exactly one time unit, O(1). The breakdown for line 2 is : 1 + N + N = 2N + 2. With line 3,
since the loop’s content is a single operation, the loop and its block perform i+1 operations. This is also a nested for loop. Finally, line 4 takes exactly one time unit to execute. Therefore, the big-Oh notation for this code in terms of N is O(N2).
To be exact: As you say, line 4 is 1 operation. For a specific i, you execute the inner loop i+3 times. Therefore, your total number of operations is
sum(0 <= i <= N-1 : i+3)
= 3N + sum(0 <= i <= N-1 : i)
= 3N + N(N-1) / 2
= N^2/2 + 5N/2
= O(N^2)
Your intuition is correct about the final efficiency class, but it is possible to be more rigorous. The first thing is that you usually just pick the most expensive basic operation to count for your analysis. In this case it would likely be the multiplication in the innermost loop, which is executed once per iteration. So how many times is it called? On the first iteration of the outermost loop, the inner loop will iterate twice. On the second outer iteration, it will be three times, and similarly up to N+2 (I'm assuming the inner loop condition is meant to be j >= 0). So that leaves us with the following summation:
sum(2, 3, 4, 5, 6 ..., N+2)
= sum(1, 2, 3, 4 ..., N+2) - 1
= (N+2)(N+3)/2 - 1
Which is in O(N²) (and actually since you have this specific result that will always be the same you can say it's in ϴ(N²)).
What is the big O value for the following algorithm? Why is it that value?
algorithm A (val array <ptr to int>)
1 n = 0
2 loop ( n < array size )
1 min = n;
2 m = n;
3 loop ( m < array size)
1 if (array[m] < array[min])
1 min = m;
4 swap(array[min],array[n]);
3 n = n + 1
I answered O(n^2) am I correct? As to how I arrived to this conclusion, the inner loops executes the n times where n = the array size and the outer loop executes n times where n is the array size n*n = n^2
That is so-called Selection sort, and indeed it has O(n2) complexity.
Yes! you are correct!
This is selection sort algorithm.
Its Θ(n^2) to be more precise.
Edit : Why is it that value?
You take the first element. Compare it with all the other elements to find minimum in the array and place it in the first place. Iterations : n.
You take the second element. Compare it with rest of the array and find minimum in that part (second minimum in whole array) and place it in the second place. Iterations : n-1.
Continuing in this way for last element, Iterations : 1.
Total = n+n-1+ ... +1 = n(n+1)/2. That is O(n^2).
So I was looking at this code from a textbook:
for (int i=0; i<N; i++)
for(int j=i+1; j<N; j++)
The author stated that the inner for-loop iterates for exactly N*(N-1)/2 times but gives no basis for how he arrived to such an equation. I understand N*(N-1) but why divide by 2? I ran the code myself and sure enough when N is 10, the inner loop iterates 45 times (10*9/2).
I messed around with the code myself and tried the following (assigned only i to j):
for (int i=0; i<N; i++)
for(int j=i; j<N; j++)
With N = 10, this results in 55. So I'm having trouble understanding the underlying math here. Sure I could just plug in all the values and bruteforce my way through the problem, but I feel there is something essential and very simple I'm missing. How would you come up with an equation for describing the for loop I just constructed? Is there a way to do it without relying on the outputs? Would really appreciate any help thanks!
Think about what happens each time the outer loop iterates. The first time, i == 0, so the inner loop starts at 1 and runs to N-1, which is N-1 iterations in total. The next time through the outer loop, i has incremented to 1, so the inner loop starts at 2 and runs up to N-1, for a total of N-2 iterations. And that pattern continues: the third time through the outer loop, you get N-3 iterations, the fourth time through, N-4, etc. When you get to the last iteration of the outer loop, i == N-1, so the inner loop starts with j = N and stops immediately. So that's zero iterations.
The total number of iterations is the sum of all these numbers:
(N-1) + (N-2) + (N-3) + ... + 1 + 0
To look at it another way, this is just the sum of the positive integers from 1 to N-1. The result of this sum is called the (N-1)th triangular number, and Wikipedia explains how you can find that the formula for the n'th triangular number is n(n+1)/2. But here you have the (N-1)th triangular number, so if you set n=N-1, you get
(N-1)(N-1+1)/2 = N(N-1)/2
You're looking at nested loops where the outer one runs N times and the inner one (N-1). You're in effect adding up the sum of 1 + 2 + 3 + ....
The N * (N+1) / 2 is a "classic" formula in mathematics. Young Carl Gauss, later a famous mathematician, was given in-class busywork: Adding up the numbers from 1 to 100. The teacher expected to keep the kids busy for an hour but Carl came up with the answer almost immediately: 5050. He explained: 1 + 100; 2 + 99; 3 + 98; 4 + 97; and so on up to 50 + 51. That's 50 sums of 101 each. You could also see that as (100 / 2) * (100 + 1); that's where the /2 comes from.
As for why it's (N-1) instead of the (N+1) I mentioned... that could have to do with starting from 1 rather than 0, that would drop one iteration from the inner loop, I think.
Look at how many times the inner (j) loop runs for each value of i. When N = 10, the outer (i) loop runs 10 times, and the j loop should run 0, 1, 2, 3, 4, 5, 6, 7, 8, and 9 times. Now you just add up those numbers to see how many times the inner loop runs. You can sum the numbers from 0 to N-1 with the formula N(N-1)/2. This is a very slight modification of a well-known formula for adding the numbers from 1 to N.
For a visual aid, you can see why 1 + 2 + 3 + ... + n = n * (n+1) / 2
If you count the iterations of the inner loop, you get:
1 2 3 4 5 6 7 8 9 10
To get the total for an arbitrary number of iterations, you can "wrap" the numbers around like this:
0 1 2 3 4
9 8 7 6 5
Now, if we add each of those columns, the all add to 9 (N-1), and there are 5 (N/2) columns. It's pretty obvious that for any even N, we'd still get N/2 columns that each added up to (N-1). As such, when the total number of iterations is even, the total number of iterations is always (N/2)(N-1), which (thanks to the commutative property) we can rewrite as N(N-1)/2.
If we did the same for an odd number of iterations, we'd have one "odd" column that couldn't be paired. In this case, we can ignore the '0' since we know it won't affect the overall sum in any case. For example, let's consider N=9 instead of N=10. For that, we get:
1 2 3 4
8 7 6 5
This gives us (N-1)/2 columns (9-1=8, 8/2=4) that each add up to N, so the sum will be N*(N-1)/2. Even though we've arrived at it slightly differently, this is an exact match for the formula above for when N is even. Again, it seems pretty obvious that this would remain true regardless of the number of columns we used (i.e., total number of iterations).
For any N (odd or even), the sum of the numbers from 0 through N-1 is N*(N-1)/2.