Could you please explain to me how to properly calculate the O(n) of the Karatsuba Algorithm?
I have read that it is equal to O(n^log_2(3)) but I still do not understand how to derive the value.
First of all, I am a bit confused that the input length is represented as n instead of n + m, but I guess it's just to cut short.
I understand that an operation is being executed for every digit, that's why it is n. But could you explain to me how is log_2(3) calculated and why is it O(n^log_2(3)) and not O(n log_n(3))?
Related
So i came upon this question where:
we have to sort n numbers between 0 and n^3 and the answer of time complexity is O(n) and the author solved it this way:
first we convert the base of these numbers to n in O(n), therefore now we have numbers with maximum 3 digits ( because of n^3 )
now we use radix sort and therefore the time is O(n)
so i have three questions :
1. is this correct? and the best time possible?
2. how is it possible to convert the base of n numbers in O(n)? like O(1) for each number? because some previous topics in this website said its O(M(n) log(n))?!
3. and if this is true, then it means we can sort any n numbers from 0 to n^m in O(n) ?!
( I searched about converting the base of n numbers and some said its
O(logn) for each number and some said its O(n) for n numbers so I got confused about this too)
1) Yes, it's correct. It is the best complexity possible, because any sort would have to at least look at the numbers and that is O(n).
2) Yes, each number is converted to base-n in O(1). Simple ways to do this take O(m^2) in the number of digits, under the usual assumption that you can do arithmetic operations on numbers up to O(n) in O(1) time. m is constant so O(m^2) is O(1)... But really this step is just to say that the radix you use in the radix sort is in O(n). If you implemented this for real, you'd use the smallest power of 2 >= n so you wouldn't need these conversions.
3) Yes, if m is constant. The simplest way takes m passes in an LSB-first radix sort with a radix of around n. Each pass takes O(n) time, and the algorithm requires O(n) extra memory (measured in words that can hold n).
So the author is correct. In practice, though, this is usually approached from the other direction. If you're going to write a function that sorts machine integers, then at some large input size it's going to be faster if you switch to a radix sort. If W is the maximum integer size, then this tradeoff point will be when n >= 2^(W/m) for some constant m. This says the same thing as your constraint, but makes it clear that we're thinking about large-sized inputs only.
There is wrong assumption that radix sort is O(n), it is not.
As described on i.e. wiki:
if all n keys are distinct, then w has to be at least log n for a
random-access machine to be able to store them in memory, which gives
at best a time complexity O(n log n).
The answer is no, "author implementation" is (at best) n log n. Also converting these numbers can take probably more than O(n)
is this correct?
Yes it's correct. If n is used as the base, then it will take 3 radix sort passes, where 3 is a constant, and since time complexity ignores constant factors, it's O(n).
and the best time possible?
Not always. Depending on the maximum value of n, a larger base could be used so that the sort is done in 2 radix sort passes or 1 counting sort pass.
how is it possible to convert the base of n numbers in O(n)? like O(1) for each number?
O(1) just means a constant time complexity == fixed number of operations per number. It doesn't matter if the method chosen is not the fastest if only time complexity is being considered. For example, using a, b, c to represent most to least significant digits and x as the number, then using integer math: a = x/(n^2), b = (x-(a*n^2))/n, c = x%n (assumes x >= 0). (side note - if n is a constant, then an optimizing compiler may convert the divisions into a multiply and shift sequence).
and if this is true, then it means we can sort any n numbers from 0 to n^m in O(n) ?!
Only if m is considered a constant. Otherwise it's O(m n).
I was given a task where I have to find a straight line that goes through the most points in a given set (>5000).
I was able to solve the problem with connecting 2 points and checking every other point if it is collinear, but that is an O(N^3) algorithm.
I would like to know if there is a way to make my program run better than O(N^3).
You can do it in O(n^2) by using a hash.
for each pair of 2 points find the line equation defined by them O(n^2)
put these equations in a hash O(1) (average complexity) in order to count occurrences of each.
go through all hash equations and find the one with the most count O(n ^ 2). This is the line you are searching for.
Total time complexity of the alg: O(n^2) * O(1) + O(n^2) = O(n^2).
The tricky bit is that the same line can seemingly have 2 different equations due to floating point precision. You need to find a hash function that takes this into account.
Another way is to:
put the equation in a vector O(n^2)
sort the vector O(n^2 log(n^2) = O(n^2 log n)
and then finally find the longest sequence of equal equations O(n ^ 2).
This will have a final complexity of O(n^2 log n).
I'm currently taking an algorithm analysis course. One of the questions of a quiz was to write an algorithm with the runtime T(n) = 4T(3n/4) + n^2 where the algorithm does not have to do anything significant.
I couldn't find any similar examples so I'm unsure how to proceed.
To simplify how to think about this kind of problem, just use an array of n elements to represent a problem of size n.
Then, the running time T(n) represents the algorithm run on the array.
The running time 4T(3n/4) represents the algorithm run on 3 quarters of the array four times.
The running time n^2 represents some quadratic operation on the array (for example, a bubble sort).
silly_algo (arr[], n)
if n == 0 return
for i : 1..4
silly_algo(arr, 3*n/4)
bubblesort(arr, n)
What I think I understand.
In the worst case the quick-sort algorithm picks the largest or smallest key in list/array to be sorted for every recursive call(if recursive implementation). I understand that the size of n will determine both the number of recursive calls and the number of comparisons(which will decrease by 1 with every step of recursion). So we have n+(n-1)+(n-2)+...+2+1 number of primitive comparison in total.
What I don't understand.
The part I don't quite grasp is how this is O(n^2)? Like I know that it is at least O(n^2) as n+(n-1)+(n-2)+...+2+1 < n^2 but how do I know it isn't say O(n*logn)? Do I have to prove that result to safely confirm it is O(n^2) or is that immediately obvious in a way I can't see? I guess in general how do I know that I have found the lowest function that represents the big O time complexity.
n+(n-1)+(n-2)+...+1 = n(n+1)/2 (you can prove it by mathematical induction), which obviously is O(n^2).
https://en.wikipedia.org/wiki/1_%2B_2_%2B_3_%2B_4_%2B_%E2%8B%AF
I am currently trying to learn time complexity of algorithms, big-o notation and so on. However, some point confuses me a lot. I know that most of the time, the input size of an array or whatever we are dealing with determines the running time of the algorithm. Let's say I have an unsorted array with size N and I want to find the maximum element of this array without using any special algorithm. I just want to iterate over the array and find the maximum element. Since the size of my array is N, this process runs at O(N) or linear time. Let M is an integer that is the square root of N. So N can be written as the square of M that is M*M or M^2. So, I think there is nothing wrong if I want to replace N with M^2. I know that M^2 is also the size of my array so my big-o notation could be written as O(M^2). So, my new running time looks like running in quadratic time. Why does this happen?
You are correct, if it happens to be that you have some variable M such that M^2 ~= N is always true, you could say the algorithm runs in O(M^2).
But, note that now - the algorithm runs in quadratic related to M, and not quadratic time related to the input, it is still linear related to the size of the input.
The important thing here is defining linear/quadratic, etc.
More precisely, you have to detail linear/quadratic, etc. with respect to something (N or M for your example). The most natural choice is to study the complexity wrt. the size of the input (N for your example).
Another trap for big integers is that the size of n is log(n). So for instance if you loop over all smaller integers, your algorithm is not polynomial.