I am trying to implement the CYK algorithm. I had a look at the Wikipedia page for it, the pseudocode it provides has the following notation for arrays.
P[n, n, r]
I am used to seeing arrays written with like P[n], what does the above notation mean?
Related
Oh, sorry about my explanation. Actually, I'm learning algorithm with my textbook, and now I 'm looking KMP algorithm. And in textbook, there are two ways to get failure function value. One is most efficient one as you said, O(n), and another one is the most unefficient one as I said above O(n³). Plus, there is no code in my book for O(n³) idea. Instead, the textbook says, "we can check all poossible prefix-suffix pair. If there is pattern P[1,.i], there is possible pair of i-1, and the time complexity is proportional to length, so (i-1) + (i-2)...+1 = i*(i-1)/2. So for all i, O(n³) is trivial ?
So my question is this. I can't understand the explanation in my textbook. Can you explain it???
I am new to Data Structures and have been trying to grasp the concepts. I understand Big - O notation, and looking for examples relate to O(n log n). I searched the internet but haven't got a satisfied example or implementation - where in I can see complexity of O(n log n).
Can some one point me to a better example and implementation for this ?
Thanks in advance
A classic example of an O(nlogn) algorithm is this of Merge Sort. Here you would find a detailed calculation of it's complexity. Generally speaking, there are many divide and conquer algorithms that have this complexity.
There is a well known theorem in complexity theory called the Master theorem. Its particular case says that if the complexity T(n) of an algorithm satisfies the equation
T(n) = a T(n/a) + b*n (1)
then
T(n) = O (n log n) (2)
The equation (1) above can be interpreted as though the algorithm works by splitting the problem into a parts and applying itself to each part separately and then doing some work on the complete input. This algorithmic pattern is sometimes called Merge and Recombine.
Consider the following example in Python
def f(x):
if len(x) > 1:
x1 = [z for z in x[1:] if z <= x[0]]
x2 = [z for z in x[1:] if z > x[0]]
return f(x1) + [x[0]] + f(x2)
else:
return x
this function implements a recursive algorithm that splits the input list into two parts and applies itself to each part independently, then concatenates the results. If we are lucky and x and y parts are of the same length then the complexity of the algorithm can be computed with the formula (2) above with a = 2.
If you are familiar with sorting and the Python language, you would recognize here an algorithm that emulates Quicksort but without the complexity of performing the sorting inplace. A somewhat cleaner example is Merge sort mentioned in the answer given by Christos.
I would suggest looking at the Cormen's Introduction to Algorithms Page-151 (Heapsort) and 170 (Quicksort). These are explained in detail in this book. The key idea in every case is that you need to understand the basic operation that is being done which is the comparison (in this two cases) and then analyze using the feature of the algorithm itself. For quicksort the pivottal analysis and for heapsort the heapify part.
This book covers everything you need to know in order to analyze complexity.
I was trying to write some sorting algorithms in prolog and find their complexity when I started thinking whether or not they are going to have a different complexity just because they are written in a logical language.
Take for example quicksort. It has an average complexity of nlogn and it's code(not complete) goes like this:
quicksort([Head|Tail], SortedList) :-
split(Head, Tail, Left, Right),
quicksort(Left, SortedLeft),
quicksort(Right, SortedRight),
append(SortedLeft, [Head|SortedRight], SortedList).
split has n. quicksort logn. Which gives the (average) nlogn. But what about append? It also has linear complexity. So is it overall (n^2)logn?
Doesn't the fact that in prolog we can only access the elements of a list in a linear way hurt the complexity of our programs. In that sense isn't it better to use another language like C for example?
When determining the complexity of algorithms, you should take the same steps as in other languages, and keep in mind that Prolog programs have a procedural/imperative reading as well as a logical one. The main difference comes from the possibility of backtracking, which in this case isn't very relevant.
The complexity of this program, assuming none of the steps backtracks, is given by a recurrence
T(n) ≤ 2T(n/2) + splitT(n) + appendT(n)
You've already observed that append/3 takes linear time, so if split/4 takes linear time as well,
splitT(n) + appendT(n) = Θ(n)
and you get T(n) ≤ 2T(n/2) + Θ(n), which is the same recurrence as for imperative quicksort.
Doesn't the fact that in prolog we can only access the elements of a list in a linear way hurt the complexity of our programs. In that sense isn't it better to use another language like C for example?
As I've shown, this is unrelated to the complexity problem that you presented. Yes, for many problems idiomatic C programs are faster than idiomatic Prolog programs. For other problems, idiomatic Prolog programs are a few lines of code whereas the corresponding C programs would include half a Prolog interpreter.
What is relevant here is that in C, quicksort can be written with O(lg n) ''space'' complexity since you only need to represent the stack and can modify an array in-place. A Prolog list, by contrast, is immutable, so your program will build a new list and has linear space complexity. Whether that's good or bad depends on the use case.
Recently, I have read something about Ackermann function and Knuth's up-arrow notation. I know that notation is used to denote vary large number. However, I can't find any practical use - the notation is applied in certain algorithms or programs - of this notation. So could anyone know is there any real-world use of this notation?
Graham's number, one of the largest numbers ever used in serious mathematical proof, is an upper bound for a problem related to Ramsey theory. This use is not directly related to programming though.
One example is the disjoint set data structure which is used in algorithms to compute connected components of graphs.
The complexity of the amortized time per operation when using this data structure is based on the inverse of the Ackermann function.
See the paper "Efficiency of a Good But Not Linear Set Union Algorithm" by R.Tarjan for the proof.
Closed. This question does not meet Stack Overflow guidelines. It is not currently accepting answers.
This question does not appear to be about programming within the scope defined in the help center.
Closed 2 years ago.
Improve this question
I am trying to compare 2 algorithms. I thought I may try and write a proof for them. (My math sucks, so hence the question.)
Normally in our math lesson last year we would be given a question like <can't use symbols in here so left them out>.
Prove: (2r + 3) = n (n + 4)
Then I would do the needed 4 stages and get the answer at the end.
Where I am stuck is proving prims and Kruskals - how can I get these algorithms in to a form like the mathmatical one above, so I can proceed to prove?
Note: I am not asking people to answer it for me - just help me get it in to a form where I can have a go myself.
To prove the correctness of an algorithm, you typically have to show (a) that it terminates and (b) that its output satisfies the specification of what you're trying to do. These two proofs will be rather different from the algebraic proofs you mention in your question. The key concept you need is mathematical induction. (It's recursion for proofs.)
Let's take quicksort as an example.
To prove that quicksort always terminates, you would first show that it terminates for input of length 1. (This is trivially true.) Then show that if it terminates for input of length up to n, then it will terminate for input of length n+1. Thanks to induction, this is sufficient to prove that the algorithm terminates for all input.
To prove that quicksort is correct, you must convert the specification of comparison sorting to precise mathematical language. We want the output to be a permutation of the input such that if i ≤ j then ai ≤ aj. Proving that the output of quicksort is a permutation of the input is easy, since it starts with the input and just swaps elements. Proving the second property is a little trickier, but again you can use induction.
You don't give many details but there is a community of mathematicians (Mathematical Knowledge Management MKM) who have developed tools to support computer proofs of mathematics. See, for example:
http://imps.mcmaster.ca/
and the latest conference
http://www.orcca.on.ca/conferences/cicm09/mkm09/
Where i am stuck is proving prims and Kruskals - how can i get these algorithms in to a form like the mathmatical one above so i can proceed to prove
I don't think you can directly. Instead, prove that both generate a MST, then prove that any two MST are equal ( or equivalent, since you can have more than one MST for some graphs ). If both algorithms generate MSTs which are shown to be equivalent, then the algorithms are equivalent.
From my maths classes at Uni I (vaguely) remember proving Prims and Kruskals algorithms - and you don't attack it by writing it in a mathematical form. Instead, you take proven theories for Graphs and combine them e.g. http://en.wikipedia.org/wiki/Prim%27s_algorithm#Proof_of_correctness to build the proof.
If your looking to prove the complexity, then simply by the working of the algorithm it's O(n^2). There are some optimisations for the special case where the graph is sparse which can reduce this to O(nlogn).
Most of the times the proof depends on the problem you have in your hand. Simple argument can be suffice at times, at some other times you might need rigorous proof. I once used a corollary and proof of already proved theorem to justify my algorithm is right. But that is for a college project.
Maybe you want to try out a semi-automatic proof method. Just to go for something different ;) For example, if you have a Java specification of Prim's and Kruskal's algorithms, optimally building upon the same graph model, you can use the KeY Prover to prove the equivalence of the algorithm.
The crucial part is to formalize your proof obligation in Dynamic Logic (this is an extension of first-order logic with types and means of symbolic execution of Java programs). The formula to prove could match the following (sketchy) pattern:
\forall Graph g. \exists Tree t.
(<{KRUSKAL_CODE_HERE}>resultVar1=t) <-> (<{PRIM_CODE_HERE}>resultVar2=t)
This expresses that for all graphs, both algorithms terminate and the result is the same tree.
If you're lucky and your formula (and algorithm implementations) are right, then KeY can prove it automatically for you. If not, you might need to instantiate some quantified variables which makes it necessary to inspect the previous proof tree.
After having proven the thing with KeY, you can either be happy about having learned something or try to reconstruct a manual proof from the KeY proof - this can be a tedious task since KeY knows a lot of rules specific to Java which are not easy to comprehend. However, maybe you can do something like extracting an Herbrand disjunction from the terms that KeY used to instantiate existential quantifiers at the right-hand side of sequents in the proof.
Well, I think that KeY is an interesting tool and more people should get used to prove critical Java code using tools like that ;)