My homework involves Big O analysis and I think I've got the hang of it, but I'm not 100% sure. Would any of you lovely people mind taking a look and telling me if I'm on the right track?
The assignment is below. For questions 1 and 3, my analysis and answers are on the right, after the // marks. For question 2, my analysis and answers are below the algorithm type.
Thanks in advance for your help! :-)
1.For each of the following program fragments, give a Big-Oh analysis of the running time in terms of N:
(a) // Fragment (a)
for ( int i = 0, Sum = 0; i < N; i++ ) // N operations
for( int j = 0; j < N; j++ ) // N operations
Sum++; // Total: N^2 operations => O(N^2)
(b) // Fragment (b)
for( int i = 0, Sum = 0; i < N * N; i++ ) // N^2 operations
for( int j = 0; j < N; j ++ ) // N operations
Sum++; // Total: N^3 operations => O(N^3)
(c) // Fragment (c)
for( int i = 0, Sum = 0; i < N; i++ ) // N operations
for( int j = 0; j < i; j ++ ) // N-1 operations
Sum++; // Total: N(N-1) = N^2 – N operations => O(N^2)
(d) // Fragment (d)
for( int i = 0, Sum = 0; i < N; i++ ) // N operations
for( int j = 0; j < N * N; j++ ) // N^2 operations
for( int k = 0; k < j; k++ ) // N^2 operations
Sum++; // Total: N^5 operations => O(N^5)
2. An algorithm takes 0.5 milliseconds for input size 100. How long will it take for input size 500 if the running time is:
a. Linear
0.5 *5 = 2.5 milliseconds
b. O( N log N)
O (N log N) – treat the first N as a constant, so O (N log N) = O (log N)
Input size 100 = (log 100) + 1 = 2 + 1 = 3 operations
Input size 500 = (log 500) + 1= 2.7 + 1 = 3.7 ≈ 4 operations
Input size 100 runs in 0.5 milliseconds, so input size 500 takes 0.5 * (4/3) ≈ 0.67 milliseconds
c. Quadratic
Input size 100 in quadratic runs 100^2 operations = 10,000 operations
Input size 500 in quadratic runs 500^2 operations = 250,000 operations = 25 times as many
Input size of 100 runs in 0.5 milliseconds, so input size of 500 takes 25 * 0.5 = 12.5 milliseconds
d. Cubic
Input size 100 in quadratic runs 100^3 operations = 1,000,000 operations
Input size 500 in quadratic runs 500^3 operations = 125,000,000 operations = 125 times as many
Input size of 100 runs in 0.5 milliseconds, so input size of 500 takes 125 * 0.5 = 62.5 milliseconds
3. Find the Big-O for the following:
(a) f(x) = 2x^3 + x^2 log x // O(x^3)
(b) f(x) = (x^4 – 34x^3 + x^2 -20) // O(x^4)
(c) f(x) = x^3 – 1/log(x) // O(x^3)
4. Order the following functions by growth rate: (1 is slowest growth rate; 11 is fastest growth rate)
__6_ (a) N
__5_ (b) √N
__7_ (c) N^1.5
__9_ (d) N^2
__4_ (e) N log N
__2_ (f) 2/N
_11_ (g) 2^N
__3_ (h) 37
_10_ (i) N^3
__1_ (j) 1/ N^2
__8_ (k) N^2 /log N
* My logic in putting (j) and (f) as the slowest is that as N grows, 1/N^2 and 2/N decrease, so their growth rates are negative and therefore slower than the rest which have positive growth rates (or a 0 growth rate in the case of 37 (h)). Is that correct?
I looked at your questions 1 to 3 and its looks alright.
Follow these rules and check for yourself:
1) Multiplicative constants can be omitted,
Example 50n^2 simplifies to n^2
2) n^a dominates n^b if a>b
Example n^3 dominates n^2, so n^3 + n^2 + n ,simplifies to n3
3) Any exponential dominates any polynomial
Example 3^n dominates n^5
Example 2^n dominates n^2+5n+100
4) Any polynomial dominates any logarithm
Example n dominates (log n)3
As for Question 4 use below as a guide (from least to greatest):
Log2 n < n < n log2 n < n^2 < n^3 < 2^n
the answer for the (b) of the time calculation is wrong. you can not assume one of the n as constant.So nlogn becomes 1log1 which means log1 is 0. so 0.
so that answer is 100 log100 operations comparision with 500log500 ...
coming to the least to greatest.
b is 4 and a is 5. c,e,k are competition for the position 6 and 7 and 8.
1,2,3 positions given by you are correct.9,10,11 are correct.
i will check some analysis over 6,7,8 and let you know..
if you need any clarrification over my answer you can comment on that ..
#op Can you please tell me why you considered O(nlgn) = O(lg n)? As far as I understand your analysis for Q2 part b is actually the analysis of O(lg n) algorithms, to analyze nlgn algos you need to factor in that n in the left.
(a) Correct
(b) Correct
(c) Correctish. 0 + 1 + 2 + ... + (n - 1) = n(n - 1) / 2 = 0.5n^2 - 0.5n = O(n^2)
(d) Correct (there is a 1/2 in there like for (c), but the complexity is still O(N^5))
a. Correct
b. let K be the duration of one step.
K * (100 log 100) = 0.5, so K = 7.5 * 10^-4
K * (500 log 500) = 7.5 * 1-^-4 * 500 log 500 = 3.3737ms
Alternatively, (500 log 500) / (100 log 100) = 6.7474
When n = 500 it will be 6.7474 times slower, 6.7474 * 0.5ms = 3.3737ms
c. Correct
d. Correct
(a) Correct
(b) Correct
(c) Correct
__5_ (a) N
__4_ (b) √N
__7_ (c) N^1.5
__9_ (d) N^2
__6_ (e) N log N
__2_ (f) 2/N
_11_ (g) 2^N
__3_ (h) 37
_10_ (i) N^3
__1_ (j) 1 / N^2
__8_ (k) N^2 /log N
I agree with the positioning of (f) and (j). However, you should be aware that they don't occur out there 'in the wild' because every algorithm has at least one step, and therefore cannot beat O(1). See Are there any O(1/n) algorithms? for a more detailed explanation.
Related
This is a test I failed because I thought this complexity would be O(n), but it appears i'm wrong and it's O(n^2). Why not O(n)?
First, notice that the question does not ask what is the time complexity of a function calculating f(n), but rather the complexity of the function f(n) itself. you can think about f(n) as being the time complexity of some other algorithm if you are more comfortable talking about time complexity.
This is indeed O(n^2), when the sequence a_i is bounded by a constant and each a_i is at least 1.
By the assumption, for all i, a_i <= c for some constant c.
Hence, a_1*1+...+a_n*n <= c * (1 + 2 + ... + n). Now we need to show that 1 + 2 +... + n = O(n^2) to complete the proof.
1 + 2 + ... + n <= n + n + ... + n = n * n = n ^ 2
and
1 + 2 + ... + n >= n / 2 + (n / 2 + 1) + ... + n >= (n / 2) * (n / 2) = n^2/4
So the complexity is actually Theta(n^2).
Note that if a_i was not constant, e.g., a_i = i then the result is not correct.
in that case, f(n) = 1^2 + 2^2 + ... + n^2 and you can show easily (using the same method as before) that f(n) = Omega(n^3), which means it's not O(n^2).
Preface, not super great with complexity-theory but I'll take a stab.
I think what is confusing is that its not a time complexity problem, but rather the functions complexity.
So for easy part i just goes up to n ie. 1,2,3 ...n , then for ai all entries must be above 0 meaning that a could be something like this 2,5,1... for n times. If you multiply them together n*n = O(n2).
The best case would be if a is 1,1,1 which drop the complexity down to O(n) but the average case will be n so you get squared.
Unless it's mentioned that a[i] is O(n), it's definitely O(n)
Here an another try to achieve O(n*n) if sum should be returned as result.
int sum = 0;
for(int i = 0; i<=n; i++){
for(int j = 0; j<=n; j++){
if(i == j){
sum += A[i] * j;
}
}
return sum;
This is the question
I am taking a data structures class, i am kind of confused on how to go with this type of problem. Any leads will be helpful, thanks in advance.
Let me explain one of these and see if you can make an attempt at the rest.
f(n) is O( g(n) ) "function f of n" is ("big O") **Order** g(n)
if for some n (maybe not f(0) or f(1) or... but eventually for some n)
and for some **constant** (1, 2, 50, 80 million, something)
f(n) <= c * g(n)
So if we say some function f is "O(log n)" than means that starting at
some n that we pass into f(), and some number c then
f(n) <= c * log(n)
Lets take a really simple example:
function f ( n ) {
answer = 0;
for (i = 0; i < n; ++i) { // loops n times
answer += n+3; // 2 ops: add, add
answer /= (7*n); // 2 ops: mult, div
answer ^= 2; // 1 op: xor
} // 2 + 2 + 1 = 5
return answer;
}
So, we might say 'c' is 5, and g(n) is n (we clearly loop n times).
f (n) <= 5 * g(n)
f (n) <= 5 * n
f () is O(n)
basically what this is saying is the constant factors don't matter at all when n gets big enough. It makes almost no difference if f(n) is (5n) or (7n) or (27n) when we might compare it to other functions which could be (87log(n)) or (0.01n²).
\ n 10 1000 100000
f(n) \-----------------------------
7n | 70 7000 700000 O(n) grows linearly with prob size
87logn | ~200 ~600 ~1000 O(log n) grows slowly [good!]
.01n² | 10 10000 100000000 O(n²) grows fast [bad!]
Consider the following randomized search algorithm on a sorted array a of length n (in increasing order). x can be any element of the array.
size_t randomized_search(value_t a[], size_t n, value_t x)
size_t l = 0;
size_t r = n - 1;
while (true) {
size_t j = rand_between(l, r);
if (a[j] == x) return j;
if (a[j] < x) l = j + 1;
if (a[j] > x) r = j - 1;
}
}
What is the expectation value of the Big Theta complexity (bounded both below and above) of this function when x is selected randomly from a?
Although this seems to be log(n), I carried out an experiment with instruction count, and found out that the result grows a little faster than log(n) (according to my data, even (log(n))^1.1 better fit the result).
Someone told me that this algorithm has an exact big theta complexity (so obviously log(n)^1.1 is not the answer). So, could you please give the time complexity along with your approach to prove it? Thanks.
Update: the data from my experiment
log(n) fit result by mathematica:
log(n)^1.1 fit result:
If you're willing to switch to counting three-way compares, I can tell you the exact complexity.
Suppose that the key is at position i, and I want to know the expected number of compares with position j. I claim that position j is examined if and only if it's the first position between i and j inclusive to be examined. Since the pivot element is selected uniformly at random each time, this happens with probability 1/(|i - j| + 1).
The total complexity is the expectation over i <- {1, ..., n} of sum_{j=1}^n 1/(|i - j| + 1), which is
sum_{i=1}^n 1/n sum_{j=1}^n 1/(|i - j| + 1)
= 1/n sum_{i=1}^n (sum_{j=1}^i 1/(i - j + 1) + sum_{j=i+1}^n 1/(j - i + 1))
= 1/n sum_{i=1}^n (H(i) + H(n + 1 - i) - 1)
= 1/n sum_{i=1}^n H(i) + 1/n sum_{i=1}^n H(n + 1 - i) - 1
= 1/n sum_{i=1}^n H(i) + 1/n sum_{k=1}^n H(k) - 1 (k = n + 1 - i)
= 2 H(n + 1) - 3 + 2 H(n + 1)/n - 2/n
= 2 H(n + 1) - 3 + O(log n / n)
= 2 log n + O(1)
= Theta(log n).
(log means natural log here.) Note the -3 in the low order terms. This makes it look like the number of compares is growing faster than logarithmic at the beginning, but the asymptotic behavior dictates that it levels off. Try excluding small n and refitting your curves.
Assuming rand_between to implement sampling from a uniform probability distribution in constant time, the expected running time of this algorithm is Θ(lg n). Informal sketch of a proof: the expected value of rand_between(l, r) is (l+r)/2, the midpoint between them. So each iteration is expected to skip half of the array (assuming the size is a power of two), just like a single iteration of binary search would.
More formally, borrowing from an analysis of quickselect, observe that when you pick a random midpoint, half of the time it will be between ¼n and ¾n. Neither the left nor the right subarray has more than ¾n elements. The other half of the time, neither has more than n elements (obviously). That leads to a recurrence relation
T(n) = ½T(¾n) + ½T(n) + f(n)
where f(n) is the amount of work in each iteration. Subtracting ½T(n) from both sides, then doubling both sides, we have
½T(n) = ½T(¾n) + f(n)
T(n) = T(¾n) + 2f(n)
Now, since 2f(n) = Θ(1) = Θ(n ᶜ log⁰ n) where c = log(1) = 0, it follows by the master theorem that T(n) = Θ(n⁰ lg n) = Θ(lg n).
I have a short program here:
Given any n:
i = 0;
while (i < n) {
k = 2;
while (k < n) {
sum += a[j] * b[k]
k = k * k;
}
i++;
}
The asymptotic running time of this is O(n log log n). Why is this the case? I get that the entire program will at least run n times. But I'm not sure how to find log log n. The inner loop is depending on k * k, so it's obviously going to be less than n. And it would just be n log n if it was k / 2 each time. But how would you figure out the answer to be log log n?
For mathematical proof, inner loop can be written as:
T(n) = T(sqrt(n)) + 1
w.l.o.g assume 2 ^ 2 ^ (t-1)<= n <= 2 ^ (2 ^ t)=>
we know 2^2^t = 2^2^(t-1) * 2^2^(t-1)
T(2^2^t) = T(2^2^(t-1)) + 1=T(2^2^(t-2)) + 2 =....= T(2^2^0) + t =>
T(2^2^(t-1)) <= T(n) <= T(2^2^t) = T(2^2^0) + log log 2^2^t = O(1) + loglogn
==> O(1) + (loglogn) - 1 <= T(n) <= O(1) + loglog(n) => T(n) = Teta(loglogn).
and then total time is O(n loglogn).
Why inner loop is T(n)=T(sqrt(n)) +1?
first see when inner loop breaks, when k>n, means before that k was at least sqrt(n), or in two level before it was at most sqrt(n), so running time will be T(sqrt(n)) + 2 ≥ T(n) ≥ T(sqrt(n)) + 1.
Time Complexity of a loop is O(log log n) if the loop variables is reduced / increased exponentially by a constant amount. If the loop variable is divided / multiplied by a constant amount then complexity is O(Logn).
Eg: in your case value of k is as follow. Let i in parenthesis denote the number of times the loop has been executed.
2 (0) , 2^2 (1), 2^4 (2), 2^8 (3), 2^16(4), 2^32 (5) , 2^ 64 (6) ...... till n (k) is reached.
The value of k here will be O(log log n) which is the number of times the loop has executed.
For the sake of assumption lets assume that n is 2^64. Now log (2^64) = 64 and log 64 = log (2^6) = 6. Hence your program ran 6 times when n is 2^64.
I think if the codes are like this, it should be n*log n;
i = 0;
while (i < n) {
k = 2;
while (k < n) {
sum += a[j] * b[k]
k *= c;// c is a constant bigger than 1 and less than k;
}
i++;
}
Okay, So let's break this down first -
Given any n:
i = 0;
while (i < n) {
k = 2;
while (k < n) {
sum += a[j] * b[k]
k = k * k;
}
i++;
}
while( i<n ) will run for n+1 times but we'll round it off to n times.
now here comes the fun part, k<n will not run for n times instead it will run for log log n times because here instead of incrementing k by 1,in each loop we are incrementing it by squaring it. now this means it'll take only log log n time for the loop. you'll understand this when you learn design and analysis of algorithm
Now we combine all the time complexity and we get n.log log n time here I hope you get it now.
public void foo(int n, int m) {
int i = m;
while (i > 100) {
i = i / 3;
}
for (int k = i ; k >= 0; k--) {
for (int j = 1; j < n; j *= 2) {
System.out.print(k + "\t" + j);
}
System.out.println();
}
}
I figured the complexity would be O(logn).
That is as a product of the inner loop, the outer loop -- will never be executed more than 100 times, so it can be omitted.
What I'm not sure about is the while clause, should it be incorporated into the Big-O complexity? For very large i values it could make an impact, or arithmetic operations, doesn't matter on what scale, count as basic operations and can be omitted?
The while loop is O(log m) because you keep dividing m by 3 until it is below or equal to 100.
Since 100 is a constant in your case, it can be ignored, yes.
The inner loop is O(log n) as you said, because you multiply j by 2 until it exceeds n.
Therefore the total complexity is O(log n + log m).
or arithmetic operations, doesn't matter on what scale, count as basic operations and can be omitted?
Arithmetic operations can usually be omitted, yes. However, it also depends on the language. This looks like Java and it looks like you're using primitive types. In this case it's ok to consider arithmetic operations O(1), yes. But if you use big integers for example, that's not really ok anymore, as addition and multiplication are no longer O(1).
The complexity is O(log m + log n).
The while loop executes log3(m) times - a constant (log3(100)). The outer for loop executes a constant number of times (around 100), and the inner loop executes log2(n) times.
The while loop divides the value of m by a factor of 3, therefore the number of such operations will be log(base 3) m
For the for loops you could think of the number of operations as 2 summations -
summation (k = 0 to i) [ summation (j = 0 to lg n) (1)]
summation (k = 0 to i) [lg n + 1]
(lg n + 1) ( i + 1) will be total number of operations, of which the log term dominates.
That's why the complexity is O(log (base3) m + lg n)
Here the lg refers to log to base 2