Can any one tell me what is the worst time complexity of below code?
Is it linear or bigger?
void fun(int[] nums){
{
int min = min(nums);
int max = max(nums);
for(int i= min; i<=max;i++){
print(i); //constant complexity for print
}
}
int min(int[] nums);//return min in nums in linear time
int max(int[] nums);//return max in nums in linear time
where
0 <= nums.length <= 10^4 and -10^9 <= nums[i] <= 10^9
Can I say that time complexity of this code is O(Max(nums[i]) - Min(nums[i])) and can I say, this is linear time complexity?
As the complexity is linear with respect to the range R = max - min of the data, I would call it a pseudo-linear complexity. O(N + R).
This is detailed in this Wikipedia entry: Pseudo-polynomial time
As mentioned in the introduction of this article:
In computational complexity theory, a numeric algorithm runs in pseudo-polynomial time if its running time is a polynomial in the numeric value of the input (the largest integer present in the input)—but not necessarily in the length of the input (the number of bits required to represent it), which is the case for polynomial time algorithms.
Generally, when analysing the complexity of a given algorithm, we don't make any specific assumption about the inherent range limitation of a particular targeted language, except of course if this is especially mentionned in the problem.
If the range of numbers is constant (ie -10^9 <= nums[i] <= 10^9) then
for(int i= min; i<=max;i++){
print(i); //constant complexity for print
}
is in O(1), ie constant because you know, it iterates at most 2 * 10^9 numbers, regardless of how many numbers there are in the nums[] array. Thus it does not depend on the size of the input array.
Consider the following input arrays
nums = [-10^9, 10^9]; //size 2
nums = [-10^9, -10^9 + 1, -10^9 + 2, ..., 10^9 - 2, 10^9 - 1, 10^9] //size 2 * 10^9 + 1
for both min and max will have the same values -10^9 and 10^9 respectively. Thus your loop will iterate all numbers from -10^9 to 10^9. And even if there were 10^100000 numbers in the orginal array, the for loop will at most iterate from -10^9 to 10^9.
And you say min() and max() are in O(n), thus your overall algorithm would also be in O(n). But if you then take into account that the given maximum length (10^4) of the array is by magnitudes smaller then the limit of your numbers, you can even neglect calling min and max
And as for your comment
For ex. array =[1,200,2,6,4,100]. In this case we can find min and max in linear time(O(n) where n is length of array). Now, my for loop complexity is O(200) or O(n^3) which is much more than length of array. Can I still say its linear complexity
The size of the array and the values in the array are completely independent of each other. Thus you cannot express the complexity of the for loop in terms of n (as explained above). If you really want to take into account also the range of the numbers, you have to express it somehow like this O(n + r) where n is the size of the array, and r is the range of the numbers.
Related
Given an array nums
Count no. of pairs (two elements) where bitwise AND is greater than K
Brute force
for i in range(0,n):
for j in range(i+1,n):
if a[i]&a[j] > k:
res += 1
Better version:
preprocess to remove all elements ≤k
and then brute force
But i was wondering, what would be the limit in complexity here?
Can we do better with a trie, hashmap approach like two-sum?
( I did not find this problem on Leetcode so I thought of asking here )
Let size_of_input_array = N. Let the input array be of B-bit numbers
Here is an easy to understand and implement solution.
Eliminate all values <= k.
The above image shows 5 10-bit numbers.
Step 1: Adjacency Graph
Store a list of set bits. In our example, 7th bit is set for numbers at index 0,1,2,3 in the input array.
Step 2: The challenge is to avoid counting the same pairs again.
To solve this challenge we take help of union-find data structure as shown in the code below.
//unordered_map<int, vector<int>> adjacency_graph;
//adjacency_graph has been filled up in step 1
vector<int> parent;
for(int i = 0; i < input_array.size(); i++)
parent.push_back(i);
int result = 0;
for(int i = 0; i < adjacency_graph.size(); i++){ // loop 1
auto v = adjacency_graph[i];
if(v.size() > 1){
int different_parents = 1;
for (int j = 1; j < v.size(); j++) { // loop 2
int x = find(parent, v[j]);
int y = find(parent, v[j - 1]);
if (x != y) {
different_parents++;
union(parent, x, y);
}
}
result += (different_parents * (different_parents - 1)) / 2;
}
}
return result;
In the above code, find and union are from union-find data structure.
Time Complexity:
Step 1:
Build Adjacency Graph: O(BN)
Step 2:
Loop 1: O(B)
Loop 2: O(N * Inverse of Ackermann’s function which is an extremely slow-growing function)
Overall Time Complexity
= O(BN)
Space Complexity
Overall space complexity = O(BN)
First, prune everything <= k. Also Sort the value list.
Going from the most significant bit to the least significant we are going to keep track of the set of numbers we are working with (initially all ,s=0, e=n).
Let p be the first position that contains a 1 in the current set at the current position.
If the bit in k is 0, then everything that would yield a 1 world definetly be good and we need to investigate the ones that get a 0. We have (end - p) * (end-p-1) /2 pairs in the current range and (end-p) * <total 1s in this position larger or equal to end> combinations with larger previously good numbers, that we can add to the solution. To continue we update end = p. We want to count 1s in all the numbers above, because we only counted them before in pairs with each other, not with the numbers this low in the set.
If the bit in k is 1, then we can't count any wins yet, but we need to eliminate everything below p, so we update start = p.
You can stop once you went through all the bits or start==end.
Details:
Since at each step we eliminate either everything that has a 0 or everything that has a 1, then everything between start and end will have the same bit-prefix. since the values are sorted we can do a binary search to find p.
For <total 1s in this position larger than p>. We already have the values sorted. So we can compute partial sums and store for every position in the sorted list the number of 1s in every bit position for all numbers above it.
Complexity:
We got bit-by-bit so L (the bit length of the numbers), we do a binary search (logN), and lookup and updates O(1), so this is O(L logN).
We have to sort O(NlogN).
We have to compute partial bit-wise sums O(L*N).
Total O(L logN + NlogN + L*N).
Since N>>L, L logN is subsummed by NlogN. Since L>>logN (probably, as in you have 32 bit numbers but you don't have 4Billion of them), then NlogN is subsummed by L*N. So complexity is O(L * N). Since we also need to keep the partial sums around the memory complexity is also O(L * N).
There is implementation of this algorithm of finding prime numbers upto n in O(n*log(log(n)) time complexity. How can we achieve it in O(n) time complexity?
You can perform the Sieve of Eratosthenes to determine which numbers are prime in the range [2, n] in O(n) time as follows:
For each number x in the interval [2, n], we compute the minimum prime factor of x. For implementation purposes, this can easily be done by keeping an array --- say MPF[] --- in which MPF[x] represents the minimum prime factor of x. Initially, you should set MPF[x] equal to zero for every integer x. As the algorithm progresses, this table will get filled.
Now we use a for-loop and iterate from i = 2 upto i = n (inclusive). If we encounter a number for which MPF[i] equals 0, then we conclude immediately that i is prime since it doesn't have a least prime factor. At this point, we mark i as prime by inserting it into a list, and we set MPF[i] equal to i. Conversely, if MPF[i] does not equal 0, then we know that i is composite with minimum prime factor equal to MPF[i].
During each iteration, after we've checked MPF[i], we do the following: compute the number y_j = i * p_j for each prime number p_j less than or equal to MPF[i], and set MPF[y_j] equal to p_j.
This might seem counterintuitive --- why is the runtime O(n) if we have two nested loops? The key idea is that every value is set exactly one, so the runtime is O(n). This website gives a C++ implementation, which I've provided below:
const int N = 10000000;
int lp[N+1];
vector<int> pr;
for (int i=2; i<=N; ++i) {
if (lp[i] == 0) {
lp[i] = i;
pr.push_back (i);
}
for (int j=0; j<(int)pr.size() && pr[j]<=lp[i] && i*pr[j]<=N; ++j)
lp[i * pr[j]] = pr[j];
}
The array lp[] in the implementation above is the same thing as MPF[] that I described in my explanation. Also, pr stores the list of prime numbers.
Well, if the algorithm is O(n*log(n)) you generally can’t do better without changing the algorithm.
The complexity is O(n*log(n)). But you can trade between time and resources: By making sure you have O(log(n)) computing nodes running in parallel, it would be possible to do it in O(n).
Hope I didn’t do your homework...
I made up my own interview-style problem, and have a question on the big O of my solution. I will state the problem and my solution below, but first let me say that the obvious solution involves a nested loop and is O(n2). I believe I found a O(n) solution, but then I realized it depends not only on the size of the input, but the largest value of the input. It seems like my running time of O(n) is only a technicality, and that it could easily run in O(n2) time or worse in real life.
The problem is:
For each item in a given array of positive integers, print all the other items in the array that are multiples of the current item.
Example Input:
[2 9 6 8 3]
Example Output:
2: 6 8
9:
6:
8:
3: 9 6
My solution (in C#):
private static void PrintAllDivisibleBy(int[] arr)
{
Dictionary<int, bool> dic = new Dictionary<int, bool>();
if (arr == null || arr.Length < 2)
return;
int max = arr[0];
for(int i=0; i<arr.Length; i++)
{
if (arr[i] > max)
max = arr[i];
dic[arr[i]] = true;
}
for(int i=0; i<arr.Length; i++)
{
Console.Write("{0}: ", arr[i]);
int multiplier = 2;
while(true)
{
int product = multiplier * arr[i];
if (dic.ContainsKey(product))
Console.Write("{0} ", product);
if (product >= max)
break;
multiplier++;
}
Console.WriteLine();
}
}
So, if 2 of the array items are 1 and n, where n is the array length, the inner while loop will run n times, making this equivalent to O(n2). But, since the performance is dependent on the size of the input values, not the length of the list, that makes it O(n), right?
Would you consider this a true O(n) solution? Is it only O(n) due to technicalities, but slower in real life?
Good question! The answer is that, no, n is not always the size of the input: You can't really talk about O(n) without defining what the n means, but often people use imprecise language and imply that n is "the most obvious thing that scales here". Technically we should usually say things like "This sort algorithm performs a number of comparisons that is O(n) in the number of elements in the list": being specific about both what n is, and what quantity we are measuring (comparisons).
If you have an algorithm that depends on the product of two different things (here, the length of the list and the largest element in it), the proper way to express that is in the form O(m*n), and then define what m and n are for your context. So, we could say that your algorithm performs O(m*n) multiplications, where m is the length of the list and n is the largest item in the list.
An algorithm is O(n) when you have to iterate over n elements and perform some constant time operation in each iteration. The inner while loop of your algorithm is not constant time as it depends on the hugeness of the biggest number in your array.
Your algorithm's best case run-time is O(n). This is the case when all the n numbers are same.
Your algorithm's worst case run-time is O(k*n), where k = the max value of int possible on your machine if you really insist to put an upper bound on k's value. For 32 bit int the max value is 2,147,483,647. You can argue that this k is a constant, but this constant is clearly
not fixed for every case of input array; and,
not negligible.
Would you consider this a true O(n) solution?
The runtime actually is O(nm) where m is the maximum element from arr. If the elements in your array are bounded by a constant you can consider the algorithm to be O(n)
Can you improve the runtime? Here's what else you can do. First notice that you can ensure that the elements are different. ( you compress the array in hashmap which stores how many times an element is found in the array). Then your runtime would be max/a[0]+max/a[1]+max/a[2]+...<= max+max/2+...max/max = O(max log (max)) (assuming your array arr is sorted). If you combine this with the obvious O(n^2) algorithm you'd get O(min(n^2, max*log(max)) algorithm.
Say you're given a large set of numbers (size n) and asked to compute the average of the data. You only have enough space and memory for c numbers at one time. What is the run-time complexity of this data?
To compute an average for the whole dataset, the complexity would be O(n). Consider the following algorithm:
set sum = 0;
for(i = 0; i < n; i++){ // Loop n times
add value of n to sum;
}
set average = sum / n;
Since we can disregard the two constant time operations, the main operation (adding value to sum) occurs n times.
In this particular example, you only have data for 'c' numbers at one time. For each individual group, you'll need a time complexity of O(c). However, this will not change your overall complexity, because ultimately you will be making n passes.
To provide a concrete example, consider the case n = 100 and c = 40, and your values are passed in an array. Your first loop would have 40 passes, the second another 40, and the third only twenty. Regardless, you have made 100 passes through the loop.
This assumes also that it is a constant time operation to get the second set of numbers.
It is O(n).
A basic (though not particularly stable) algorithm computes it iteratively as follows:
mean = 0
for n = 0,1,2,.. length(arr)-1
mean = (mean*n + arr[n])/(n+1)
A variant of this algorithm can be used to parse the data from the array in sets of c numbers, but it is still linear in n.
Ie, spelling out the seralization:
To spell it out, you can do this:
mean = 0
for m = 0, c, 2c, ..., arr_length -1
sub_arr = request_sub_arr_between(m,min(m+c-1, total_length(arr)-1))
for i = 0, 1, ..., length(sub_arr)
n = m + i
mean = (mean*n + sub_arr[i])/(n+1)
This is still O(n), as we are only doing a bounded number of things for each n. In fact, the algorithm given at the top of this answer is a variant of this with c=1. If sub_arr is not kept in local memory, but sub_arr[n] is read at each step, then we are only storing 3 numbers at any step.
It's all in the title. Suppose $X$ is an array of n floats. The empirical CDF is the function (of t):
Fn(t) = (1/n) sum{1{Xi <= t} : i=1,...,n}
This has to be computed for t_1<t_2,...,t_m (e.g. for m different, sorted, values of t). My question is what is the numerical complexity of computing this? I think O(nlog(n))+O(mlog(n)) [sort the array then perform m binary search, one for each value of t]
but I may be naive. Can anyone confirm?
Edit:
Sorry for the mess. While writing the question, I realized that I was imposing some constraints that are not in the original problem. I respond to Yves's question below.
The Xi are not sorted.
The t_j are sorted and equi-spaced.
m is smaller than n, but not by orders of magnitudes: typically m~n/4.
The given expression, a sum of N 0/1 terms, is clearly O(N).
UPDATE:
If the Xi are presorted, the function is trivially CDFi = CDF(Xi) = i/N, and the computation is in a way O(0)!
If the Xi are unsorted, you'll need to sort first in O(N.Log(N)), unless the range of the variable allows a faster sorting such as Counting sort.
If you only need to evaluate for a small number of Xis, let K, then you can consider using the naïve summation, as K.N can beat N.Log(N).
UPDATE: (second change by the OP)
Else, sort the Xi if necessary and sort the tj if necessary. Then a single linear pass will suffice. Total complexity will be one of:
O(n.Log(n) + m.Log(m))
O(n.Log(n) + m)
O(n + m.Log(m))
O(n + m).
If m < Log(n) and the Xi are unsorted, use the naïve formula. Complexity O(m.n).
Possibly there could be better options when m>n.
UPDATE: final specs: Xi unsorted, Tj sorted, m < n.
The solution I would choose is as follows:
1) Sort the Xi.
2) "Merge" the sorted Xi and Tj. This means, progress simultaneously in the X and T lists, keeping two running indexes; make sure to always increment the index that causes the shortest move; use CDF(Tj)=i/n. This is a linear process. (Very close to a merge in mergesort.)
Global complexity is O(n.Log(n)), the merging term O(n) being absorbed in the former.
UPDATE: uniform sampling.
When the Tj values are equi-spaced, let Tj = T0 + D.j, you can use an histogram approach.
Allocate an array of m+1 counters, initially 0. For every Xi, compute a bin index as Floor((Xi - T0) / D). Clamp negative values to 0 and values larger than m to m. Increment that bin. In the end, every bin will tell you how many X values are in range [Tj, Tj+1[.
Compute the prefix sum of the counters. They will now tell you how many X values are smaller than Xj+1, and CDF(j)=Counter[j]/n.
[Caution, this is an unchecked sketch, can be wrong in details.]
Total computation will take n bin incrementations followed by a prefix sum on m elements, i.e. O(n) operations.
# Input data
X= [0.125, 6, 3.25, 9, 1.4375, 6, 3.125, 7]
n= len(X)
# Sampling points (1 to 6)
T0= 1
DT= 1
m= 6
# Initialize the counters: O(m)
C= [0] * m
# Accumulate the histogram: O(n)
for x in X:
i= max(0, int((x - T0) / DT))
if i < m:
C[i]+= 1
# Compute the prefix sum: O(m)
S= 0
for i in range(m - 1):
C[i + 1]+= C[i]
# Reduce: O(m)
for i in range(m):
C[i]/= float(n)
# Display
print "T=", C
T= [0.25, 0.25, 0.5, 0.5, 0.5, 0.75]
A CDF Fn(t) is always a non-decreasing function in [0..1]. Therefore I assume your notation is saying to count the number of elements Xi <= t and return that count divided by n.
Thus if t is very large, you have n/n = 1. For very small, it's 0/n = 0 as we'd expect.
This is a poor definition of an empiracle CDF. See for example see Law, Averill M., Simulation & Modeling, 4th ed., p 301 for some more advanced ideas.
The simplest efficient way to compute your function (given that m, the number of Fn(t) values you need, is unknown) is first to sort the inputs Xi. This requires O(n log n) time, but needs to be done only once no matter how many t values you're processing.
Let's call the sorted values Yi. To find the count of Yi values <= t is the same as finding i such that Yi <= t < Yi+i. This can be done by binary search in O(log n) time for a given value of t. Divide by n and you have the Fn(t) value required. Of course you can repeat this m times to get the job done in O(m log n) time.
However you say your special case is m presorted values of t_j. You can find all the i values with a single pass over the Yi and simultaneously over the t_j, in the fashion of the merge operation in mergesort. With this you find all the answers in O(m + n) time.
Putting this together with the sorting cost, you have O(m + n + n log n) = O(m + n log n).
Note this is always faster than using the binary search lookup m times, O(n log n + m log n) = O((m + n) log n).
The only case you'd want to skip the presorting is when m < O(log n). This is because with no presorting, processing all the t_j needs O(mn) time - you must touch all n elements to count the number <= t_j. Consequently, if m < O(log n), then skipping the presort leads to less than O(n log n), i.e. asymptotically faster than the presort method.