I know there is something wrong with the following reasoning but I'm not sure what it is.
The FFT:
given two polynomials
A = a_0 + a_1 x + a_2 x^2 + ... + a_n x^n
and
B = b_0 + b_1 x + b_2 x^2 + ... + b_n x^n
you can compute the coefficients of the product
AB = \sum _k = 0 ^ 2n ( \sum _ j = 0 ^ k (a_j b_{k-j}))x^k
in O(n log n ) time.
So given two vectors (a_0, ..., a_n) and (b_0, ..., b_n) we can calculate
the vector v_i = \sum j = 0 ^ k ( a_j b_{k-j}) in O(n log n) time (by embedding the vectors in zeros.)
Given the above, we should be able to calculate the dot product of A =(a_0, ..., a_n) and B =(b_0, ..., b_n) which is A.B = \sum_j=0 ^ n a_j b_j in O(n log n) time by preprocessing one of the vectors say B to be B' = (b_n, b_{n-1}, ..., b_1, b_0) then computing the convolution as in 2. in O(n log n) time.
If the above reasoning is correct, then that means we can implement Matrix Multiplication of two nxn matrices in O(n^2 log n ) time by computing the dot product in O(n log n) time O(n) times.
However, the best run-time for matrix multiplication we know is about O(n^2.4) so this seems unlikely to be true, which probably means of of the steps 1,2, or 3 is incorrect.
There are n^2 entries in the product, not n and so this algorithm would be O(n^2 * n * log n) = O(n^3 log n).
And the best algorithm for computing dot products is O(n), not O(n log n). That's why the naive algorithm for matrix multiplication is O(n^3). (It's n^2 dot products that can be done in O(n) time).
I wonder why it is an achievement that in step 3 you can calculate the dot product in O(n log n) time as it is well known that dot product can be calculated in O(n) time? Also step 2 looks like linear time instead of O(n log n) step?
Also the conclusion about O(n^2 log n) does not follow logically. You can't take the dot product O(n) times resulting in the AB matrix---as far as I know you have to take O(n^2) dot products, leading to (per your analysis) O(n^3 log n) which is inferior to standard O(n^3). This is because of your strange O(n log n) dot product result.
Related
I am finding an algorithm for a problem where I have two sets A and B of points with n and m points. I have two algorithms for the sets with complexity O(n log n) and O(m) and I am now wondering whether the complexity for the both algorithms combined is O(n log n) or O(m).
Basically, I am wondering whether there is some relation between m and n which would result in O(m).
If m and n are truly independent of one another and neither quantity influences the other, then the runtime of running an O(n log n)-time algorithm and then an O(m)-time algorithm is will be O(n log n + m). Neither term dominates the other - if n gets huge compared to m then the n log n part dominates, and if m is huge relative to n then the m term dominates.
This gets more complicated if you know how m and n relate to one another in some way. Many graph algorithms, for example, use m to denote the number of edges and n to denote the number of nodes. In those cases, you can sometimes simplify these expressions, but sometimes cannot. For example, the cost of implementing Dijkstra’s algorithm with a Fibonacci heap is O(m + n log n), the same as what we have above.
Size of your input is x: = m + n.
Complexity of a combined (if both are performed at most a constant number of times in the combined algorithm) algorithm is:
O(n log n) + O(m) = O(x log x) + O(x) = O(x log x).
Yes if m ~ n^n, then O(logm) = O(nlogn).
There is a log formula:
log(b^c) = c*log(b)
EDIT:
For both the algos combined the Big O is always the one that is larger because we are concerned about the asymptotic upper bound.
So it will depend on value of n and m. Eg: While n^n < m, the complexity is Olog(m), after that it becomes O(nlog(n)).
For Big-O notation we are only concerned about the larger values, so if n^n >>>> m then it is O(nlog(n)), else if m >>>> n^n then it is O(logm)
The big-O hierarchy for any constants a, b > 0 is;
O(a) ⊂ O(log n) ⊂ O(n^b) ⊂ O(C^n).
I need some explanations, thanks.
Leading constants don't matter, so O(2n^2) = O(n^2) and O(100 n log n) = O (n log n).
If f and g are functions then O(f * g) = O(f) * O(g). Now, apparently you are ok accepting that O(log n) < O(n). Multiply both sides by O(n) and you get O(n) * O(log n) = O(n * log n) < O(n * n) = O(n^2).
To see that O((log n)^3) is less than O(n^a) for any positive a is a little trickier, but if you are willing to accept that O(log n) is less than O(n^a) for any positive a, then you can see it by taking the third root of O((log n)^3) and O(n^a). You get O(log n) on the one side, and O(n^(a/3)) on the other side, and the inequality you are looking for is easy to deduce from this.
you can think about that number that a function produces, it usually goes that the smaller the number, the faster the algorithm is. And if it is a larger number the function produces its slower.
log 10 < 10^b < C^n
An algorithm decomposes (divides) a problem of size n into b sub-problems each of size n/b where b is an integer. The cost of decomposition is n, and C(1)=1. Show, using repeated substitution, that for all values of 2≥b, the complexity of the algorithm is O(n lg n).
This is what I use for my initial equation C(n) = C(n/b) + n
and after k-steps of substituting I get C(n) = C(n/b^k) + n [summation(from i=0 to k-1) of (1/b)^i]
k = log(base b) n
I'm not sure I'm getting all of this right because when I finish doing this i don't get n lgn, anybody can help me figure out what to do?
I think your recurrence is wrong. Since there are b separate subproblems of size n/b, there should be a coefficient of b in front of the C(n / b) term. The recurrence should be
C(1) = 1
C(n) = b C(n/b) +O(n).
Using the Master Theorem, this solves to O(n log n). Another way to see this is that after expanding the recurrence k times, we get
C(n) = bk C(n / bk) + kn
This terminates when k = logb n. Plugging in that value of k and simplifying yields a value that is O(n log n).
Hope this helps!
I think it's interesting to solve this in holiday:
Given n integers, all of them within 1..n^3, find if there is a triple which matches pythagoras equation in O(n^2).
As you know pythagoras equation is a^2 + b^2 = c^2. for example 3^2 + 4^2 = 5^2.
As you know O(n^2 log n) is easy (with a little thinking), but will help to solve O(n^2). (space is not important).
Edit: As yi_H offered there is lookup table which can solve this problem easily, but for making it harder, space limit is O(n^2).
O(n2) time, O(n) space: square all array elements, sort, for each z in the array use the classic linear-time algorithm to determine whether there exist x, y in the array such that x + y = z.
All pythagorean triplets (a,b,c) have the following relation:
a = d * (2 * m * n)
b = d * (m^2 - n^2)
c = d * (m^2 + n^2)
where
d >= 1 and (m,n) = 1
(meaning: m and n have no comomn factor.
I guess one can find an algorithm to produce all triplets that are below n^3 using this info.
I guess O(n^2 log n) would be to sort the numbers, take any two pairs (O(n^2)) and see whether there is c in the number for which c^2 = a^2 + b^2. You can do the lookup for c with binary search, that's O(log(n)).
Now if space isn't an issue you can create a hash for all the values in O(n), then you can look up c in this hash with O(1), so it's going to be O(n^2). You can even create an lookup table for it since the numbers are between 1..n^2, so it's going to be a guaranteed O(1) lookup ;) You can also use a special lookup table which can do initialization, add and lookup in O(1).
Given the following algorithm on a dataset of size N:
Separate the data into M=(N/lg N)
blocks in O(N) time.
Partition the blocks in O(M lg M) time. *
What is the big-O? How do I evaluate (N/lg N) * lg (N/lg N) ?
If it is not O(N), is there an M small enough that the whole thing does become O(N)?
* The partition algorithm is the STL's stable_partition which, in this example, will do M tests and at most M lg M swaps. But, the items being swapped are blocks of size lg N. Does this push the practical time of step 2 back up to O(N lg N) if they must be swapped in place?
Not homework, just a working engineer poking around in comp-sci stuff.
You evaluate by doing a bit of math.
log(x/y) = log(x) - log(y)
->
log(N / log(N)) = log(N) - log(log(N))
So, plugging this back in and combining into a single fraction.
N(log(N) - log(log(N))) / log(N)
=
N - N(log(log(N)) / log(N))
<=, since log(log(N)) <= log(N) as N -> inf., it's like multiplying by <= 1
N
So, the whole thing is O(N).
You can pretty easily guess that it is O(N log N) by noticing that M = N / log N is, itself, O(N). I don't know of a quick way to figure out that it's O(N) without a bit of doubt on my part due to having to multiply in the log M.
It is O(N):
N / lgN * lg(N / lgN)=N / lgN * (lgN-lglgN)=N*(1-lglgN / lgN)<=N