I understand the concept of the Big O notation and its something i've brought upon myself to learn recently.
But say for a given algorithm, which recursively calls itself until the job is complete, if there is an OR in my return statement, how will that effect the Big O notation?
Here's the algorithm thus far:
**Algorithm: orderedSort(a,b,c)**
given strings a,b,c determine whether c is an ordered shuffle of a and b
l := length of (a + b)
if (l = 0) then
return true
if (l not = length of c)
return false
else
d := substring of a position 1 to 1
e := substring of b position 1 to 1
f := substring of c position 1 to 1
if (d = f) then
return orderedSort(a-d, b, c-f)
if (e = f) then
return orderedSort(a, b-e, c-f)
if (d and e = f) then
return orderedSort(a-d, b, c-f) or orderedSort(a, b-e, c-f)
Does having the or make it n^2?
It's far worse than you think. If both halves of the "or" will need to be evaluated some % of the time, then you will end up with O(2^n) (not O(n^2)) recursive calls.
Let's say it takes both halves of the OR 10% of the time. On average you have to go down 10ish levels before you do both halves, so you have around:
1 call with length n
2 calls with length n-10
4 calls with length n-20
8 calls with length n-30
...
2^(n/10) calls with length 0
Also, it's worse than that again, because all those string manipulations (length(a+b), a-d, etc.) take O(n) time, not constant time.
EDIT: I should mention that O(2^n) is not actually correct. It's "exponential time", but O(2^(n/10)) or whatever is strictly less than O(2^n). A correct way to write it is 2^O(n)
EDIT:
A good solution for this problem would use dynamic programming.
Let OK(i,j) = true if the first i+j characters of c are an ordered shuffle of the first i characters of a and the first j characters of b.
OK(i,0) is easy to calculate for all i. Then you can calculate all the OK(i,j) from OK(i,j-1). When you've covered all the cases with i+j = length(c), then return true if any one of them is true.
Related
fun root(n) =
if n>0 then
let
val x = root(n div 4);
in
if (2*x+1)*(2*x+1) > n then 2*x
else 2*x+1
end
else 0;
fun isPrime(n,c) =
if c<=root(n) then
if n mod c = 0 then false
else isPrime(n,c+1)
else true;
The time complexity for the root(n) function here is O(log(n)): the number is getting divided by 4 at every step and the code in the function itself is O(1). The time complexity for the isPrime function is o(sqrt(n)) as it runs iteratively from 1 to sqrt(n). The issue I face now is what would be the order of both functions together? Would it just be O(sqrt(n)) or would it be O(sqrt(n)*log(n)) or something else altogether?
I'm new to big O notation in general, I have gone through multiple websites and youtube videos trying to understand the concept but I can't seem to calculate it with any confidence... If you guys could point me towards a few resources to help me practice calculating, it would be a great help.
root(n) is O(log₄(n)), yes.
isPrime(n,c) is O((√n - c) · log₄(n)):
You recompute root(n) in every step even though it never changes, causing the "... · log₄(n)".
You iterate c from some value up to root(n); while it is upwards bounded by root(n), it is not downards bounded: c could start at 0, or at an arbitrarily large negative number, or at a positive number less than or equal to √n, or at a number greater than √n. If you assume that c starts at 0, then isPrime(n,c) is O(√n · log₄(n)).
You probably want to prove this using either induction or by reference to the Master Theorem. You may want to simplify isPrime so that it does not take c as an argument in its outer signature, and so that it does not recompute root(n) unnecessarily on every iteration.
For example:
fun isPrime n =
let
val sq = root n
fun check c = c > sq orelse (n mod c <> 0 andalso check (c + 1))
in
check 2
end
This isPrime(n) is O(√n + log₄(n)), or just O(√n) if we omit lower-order terms.
First it computes root n once at O(log₄(n)).
Then it loops from 0 up to root n once at O(√n).
Note that neither of us have proven anything formally at this point.
(Edit: Changed check (n, 0) to check (n, 2), since duh.)
(Edit: Removed n as argument from check since it never varies.)
(Edit: As you point out, Aryan, looping from 2 to root n is indeed O(√n) even though computing root n takes only O(log₄(n))!)
I created the following pseudocode but I am not sure how to calculate it's complexity:
(Pseudocode)
MyFunction(Q, L)
if (Q = empty) return
M = empty queue
NM = empty queue
M.Enqueue(Q.Dequeue)
while (Q is not empty)
pt = Q.Dequeue()
if (pt.y > M.peek().y) M.Enqueue(pt)
else NM.Enqueue(pt)
L.add(M)
if (NM is not empty) MyFunction(NM, L)
return L;
MyFunction receives a set Q of n points and a list L in which we will save k subsets of Q (1<=k<=n). When we calculate the first subset we go through all the n points of Q and select the ones that belong to the first subset. For the second subset we go through all the n points of Q except those that are already in the first subset and so on.
So, every recursive call the number of points will be reduced by an integer x until the number of points is 0. This integer x can be different from one recursive call to the other (it can be any value between 1 and n (n being the current number of points))
What would be the complexity of my algorithm then?
I was thinking that my recurrence relation would be something like this:
T(0) = 1
T(n) = T(n-x) + an
Is this correct? and if so how can I solve it?
Without any information on the distribution of points in Q, we can not know how they will be dispatched to M or NM queues.
However, it is easy to calculate the worst-case complexity of your algorithm. To calculate this, we assume that at each recursive call, all points in Q will end up in NM except the one that is being added to M before entering the loop. With this assumption, x becomes 1 in your recurrence relation. And you end up having O(n^2).
I'm having trouble with understanding the following property of divide-and-conquer algorithms.
A recursive method that divides a problem of size N into two independent
(nonempty) parts that it solves recursively calls itself less than N times.
The proof is
A recursive function that divides a problem of size N into two independent
(nonempty) parts that it solves recursively calls itself less than N times.
If the parts are one of size k and one of size N-k, then the total number of
recursive calls that we use is T(n) = T(k) + T(n-k) + 1, for N>=1 with T(1) = 0.
The solution T(N) = N-1 is immediate by induction. If the sizes sum to a value
less than N, the proof that the number of calls is less than N-1 follows from
same inductive argument.
I perfectly understand the formal proof above. What I don't understand is how this property is connected to the examples that are usually used to demonstrate the divide-and-conquer idea, particularly to the finding the maximum problem:
static double max(double a[], int l, int r)
{
if (l == r) return a[l];
int m = (l+r)/2;
double u = max(a, l, m);
double v = max(a, m+1, r);
if (u > v) return u; else return v;
}
In this case when a consists of N=2 elements max(0,1) will call itself 2 more times, that is max(0,0) and max(1,1), which equals to N. If N=4, max(0,3) will call itself 2 times, and then each of the subsequent calls will also call max 2 times, so the total number of calls is 6 > N. What am I missing?
You're not missing anything. The theorem and its proof are wrong. The error is here:
T(n) = T(k) + T(n-k) + 1
The constant term of 1 should be 2, as the function makes one recursive call for each of the two pieces into which it divides the problem. The correct bound is 2N-1, rather than N. Hopefully, this error will be fixed in the next edition of your textbook, or at least in the errata.
I have a recursive function working on a list, the function contains a loop where itself is called, and ends up with another function g. Its structure is similar as follows, to simplify the issue, we can assume that l is always a list without duplicate elements.
let rec f l = function
| [] -> g ()
| _ ->
List.fold_left
(fun acc x ->
let lr = List.filter (fun a -> (a <> x)) l in
acc + (f lr))
1 l
I am not sure how to express the complexity of this function, with List.length l and the complexity of g.
I think it is proportional to the complexity of g and the factorial of List.length l, could anyone confirm?
Since you assume that the list l does not contain any duplicates, what this function does is compute all sublists that have one less element than the original list and call itself recursively on all of them. So, the number of times g is called when starting with a list of size n is g?(n) = n · g?(n-1) = n!
Now, let's consider everything else the function has to do. The amount of work at each step of the recursion includes :
For each element in the original list, constructing a new list of one less element. This is a total amount of work equal to n2
Once the result of the recursive call is known, add it to an accumulator. This is a total amount of work equal to n (this part can be ignored, since the filter is more costly).
So, since we know how many times each recursive step will be called (based on our previous analysis), the total amount of non-g related work is: t?(n) = n2 + n (n-1)2 + n (n-1) (n-2)2 + ... + n!
This formula looks like a pain, but in fact t?(n) / n! has a finite non-zero limit as n increases (it is the sum of the k+1 / k! with 0 < k < n) and so t?(n) = Θ(n!).
Okay. I don't mean to seem mustrustful. This really does look like a functional programming homework because it's not very practical code.
Let F(n) be the number of comparisons plus the number of additions for an input of length n. And let G be the run time of g. Since g doesn't take any operands, G is constant. We are just counting the number of times its called.
The fold will execute its function n times. Each execution will call filter to do n comparisons and remove exactly one element from its input each time, then recursively call f on this shortened list and do one addition. So the total cost is
F(n) = n * (n + F(n - 1) + 1) [ if n > 0 ]
= G [ otherwise ]
The first term expands to
F(n) = n * F(n - 1) + n^2 + n
This is O(n! + n^3 + n^2 + nG) = O(n! + nG) as you proposed.
I hope this is helpful.
My question is: given a list L of length n, and an integer i such that 0 <= i < n!, how can you write a function perm(L, n) to produce the ith permutation of L in O(n) time? What I mean by ith permutation is just the ith permutation in some implementation defined ordering that must have the properties:
For any i and any 2 lists A and B, perm(A, i) and perm(B, i) must both map the jth element of A and B to an element in the same position for both A and B.
For any inputs (A, i), (A, j) perm(A, i)==perm(A, j) if and only if i==j.
NOTE: this is not homework. In fact, I solved this 2 years ago, but I've completely forgotten how, and it's killing me. Also, here is a broken attempt I made at a solution:
def perm(s, i):
n = len(s)
perm = [0]*n
itCount = 0
for elem in s:
perm[i%n + itCount] = elem
i = i / n
n -= 1
itCount+=1
return perm
ALSO NOTE: the O(n) requirement is very important. Otherwise you could just generate the n! sized list of all permutations and just return its ith element.
def perm(sequence, index):
sequence = list(sequence)
result = []
for x in xrange(len(sequence)):
idx = index % len(sequence)
index /= len(sequence)
result.append( sequence[idx] )
# constant time non-order preserving removal
sequence[idx] = sequence[-1]
del sequence[-1]
return result
Based on the algorithm for shuffling, but we take the least significant part of the number each time to decide which element to take instead of a random number. Alternatively consider it like the problem of converting to some arbitrary base except that the base name shrinks for each additional digit.
Could you use factoradics? You can find an illustration via this MSDN article.
Update: I wrote an extension of the MSDN algorithm that finds i'th permutation of n things taken r at a time, even if n != r.
A computational minimalistic approach (written in C-style pseudocode):
function perm(list,i){
for(a=list.length;a;a--){
list.switch(a-1,i mod a);
i=i/a;
}
return list;
}
Note that implementations relying on removing elements from the original list tend to run in O(n^2) time, at best O(n*log(n)) given a special tree style list implementation designed for quickly inserting and removing list elements.
The above code rather than shrinking the original list and keeping it in order just moves an element from the end to the vacant location, still makes a perfect 1:1 mapping between index and permutation, just a slightly more scrambled one, but in pure O(n) time.
So, I think I finally solved it. Before I read any answers, I'll post my own here.
def perm(L, i):
n = len(L)
if (n == 1):
return L
else:
split = i%n
return [L[split]] + perm(L[:split] + L[split+1:], i/n)
There are n! permutations. The first character can be chosen from L in n ways. Each of those choices leave (n-1)! permutations among them. So this idea is enough for establishing an order. In general, you will figure out what part you are in, pick the appropriate element and then recurse / loop on the smaller L.
The argument that this works correctly is by induction on the length of the sequence. (sketch) For a length of 1, it is trivial. For a length of n, you use the above observation to split the problem into n parts, each with a question on an L' with length (n-1). By induction, all the L's are constructed correctly (and in linear time). Then it is clear we can use the IH to construct a solution for length n.