As it currently stands, this question is not a good fit for our Q&A format. We expect answers to be supported by facts, references, or expertise, but this question will likely solicit debate, arguments, polling, or extended discussion. If you feel that this question can be improved and possibly reopened, visit the help center for guidance.
Closed 10 years ago.
I have a question about the performance of IP and FP.
Let's say I have a function to compute nth Fibonacci number.
In imperative programming I have a choice to computing the nth Fibonacci number using iterative way, recursion, or dynamic programming. Of course iterative way and dynamic programming will perform better compared to recursion asymptotically.
In functional programming, assume there is no state involved, then I can only do it in recursion way.
In this case, doesn't that mean functional programming will always perform as equal or slower compared to imperative programming in terms of efficiency (asymptotically)?
How does real world functional programming deal with this issue?
There is no one recursive way to implement the Fibonacci numbers. You can easily write a recursive function that calculates the nth Fibonacci numbers in O(n) time - it would work the same way as the iterative version (i.e. it'd keep track of the last two numbers you've calculated), but using tail recursion instead of an imperative loop. Since many functional languages require implementations to perform tail call optimization, there won't even be constant overhead compared to the iterative version.
And of course there are even ways to calculate the nth Fibonacci number in sublinear time (using the closed form or matrix multiplication), which work just as well in functional languages as in imperative languages.
Regarding dynamic programming: It is perfectly possible to do dynamic programming in functional languages. Since dynamic programming algorithms don't change a field of the array once it has been written for the first time, there is really no contradiction to functional programming here. All you need is to be able to access the already constructed parts of the array while the array is being constructed. Lazy arrays as they exist in Haskell work well for this.
Related
Closed. This question needs to be more focused. It is not currently accepting answers.
Want to improve this question? Update the question so it focuses on one problem only by editing this post.
Closed 8 years ago.
Improve this question
In usual circumstances, sorting arrays of ~1000s of simple items like integer or floats is sufficiently fast that the small differences between implementations just doesn't matter.
But what if you need to sort N modest sized arrays that have been generated by some similar process or simply have have some relatedness?
I leave the specifics of what of the mysterious array generator and relationships of the generated arrays intentionally vague. It is up to any applicable algorithms to specify a large as possible domain where they will work when they will be most useful.
EDIT: Let's narrow this by letting the arrays be independent samples. There exists an unchanging probability distribution of arrays that will be generated. Implicitly then there's a stable probability distribution of elements in the arrays but it's conditonal -- the elements within an array might not be independent. It seems like it'd be extremely hard to make use of relationships between elements within the arrays but I could be wrong. We can narrow further if needed by letting the elements in the arrays be independent. In that case the problem is to effectively learn and use the probability distribution of elements in the arrays.
Here is a paper on a self improving sorting algorithm. I am pretty strong with algorithms and machine learning, but this paper is definitely not an easy read for me.
The abstract says this
We investigate ways in which an algorithm can improve
its expected performance by fine-tuning itself automatically with respect to an arbitrary, unknown input distribution. We give such self-improving algorithms for
sorting and clustering. The highlights of this work:
a sorting algorithm with optimal expected limiting running time ...
In all cases, the algorithm begins with a learning phase
during which it adjusts itself to the input distribution
(typically in a logarithmic number of rounds), followed
by a stationary regime in which the algorithm settles to
its optimized incarnation.
As it currently stands, this question is not a good fit for our Q&A format. We expect answers to be supported by facts, references, or expertise, but this question will likely solicit debate, arguments, polling, or extended discussion. If you feel that this question can be improved and possibly reopened, visit the help center for guidance.
Closed 9 years ago.
This is a question to people who are programmers for a living -
I just proved (using the Master theorem) that if we use quicksort and we pick the pivot to be the median of the the subarray we are partitioning (using the median of medians algorithm with Θ(n) worst case run time) then the worst case run time of quicksort is Θ(n lg n) - so basically this means that this version of quicksort is as good as it can get.
My question now is - does anyone implement quicksort like this in practice? Or is it just one of those nice theoretical things that are actually not good in real life?
PS - I don't need proofs of what I'm stating, I just want to know if this is being widely known/useful
This is known (see the wikipedia entry), but since in practice the worst case is relatively rare, the added overhead of an O(N) selection algorithm on the average case is generally considered unacceptable.
It really depends on where you're working.
So far, personally, I never actually implemented it - But I really think it varies, depending on the requirements of your workplace.
When you made partition around some pivot, you already have "quality" of the pivot (how evenly it divides array). If it's lower than some threshold, you can try some smarter ways to select pivot. This keeps time complexity O(n*log n) and keeps constants low, because complex selection is done rarely.
If I don't mistake C++ STL uses something like this, but I haven't any links - that's from a conversation on work.
update
C++ STL (at least the one in Visual Studio) seems to do a different thing:
Perform partition
Unconditionally sort the smaller part by recursion (since it cannot be bigger than half that's safe for O(n*log n))
Handle the larger part in the same loop (without recursive call)
If number of iterations exceeds approx. 1.5 log2(N), it switches to heap sort which is O(n*log n).
As it currently stands, this question is not a good fit for our Q&A format. We expect answers to be supported by facts, references, or expertise, but this question will likely solicit debate, arguments, polling, or extended discussion. If you feel that this question can be improved and possibly reopened, visit the help center for guidance.
Closed 11 years ago.
Our algorithm professor gave us a assignment that requires us to choose a rare sorting algorithm (e.g. Introsort, Gnomesort, etc.) and do some research about it.
Wikipedia sure has a plenty of information about this, but it is still not enough for me to do the research in depth.
So I would like to find a book that include discussions of those rare sorting algorithms, since most of the textbooks (like CLRS, the one I am using) only discuss about some basic sorting algorithms (e.g. Bubble Sort, Merge Sort, Insertion Sort.).
Is there a book or website that contains a good amount of those information?
Thanks!
Well, a very interesting "rare" sorting algorithm in Smoothsort by Edsger Dijkstra. On paper it is almost a perfect sort:
O(n) best
O(n log n) average
O(n log n) worst
O(1) memory
n comparisons, 0 swaps when input is sorted
It is so rare due to it's complex nature (which makes it hard to optimize).
You can read the paper written by Dijkstra himself here: http://www.cs.utexas.edu/users/EWD/ewd07xx/EWD796a.PDF
And here is the wikipedia link and a very extensive article about smoothsort (by Keith Schwarz).
One of a sorting which may be you say Rare Sorting, is timsorting, It works great in arrays which are have sorted parts, best case is O(n), and worst and average case is O(n log n).
Another fast way of sorting is bitonic sorting, which is base of nearly all parallel sorting algorithms. you can find thousands of papers about in the web, also some books like Parallel algorithm of Quinn you can find extended discussion on it, and related variations of this algorithm.
Also Art of computer programming volume 3 has good discussion on sorting strategies.
Bitonic sort is O(N log^2(N)) (slightly asymptotically slower than the likes of quicksort), but it is parallellizable, with a highly regular structure. This lets you use SIMD vector instruction sets like SSE -- providing a constant-factor boost which makes it an interesting option for "bottom-level" sorts (instead of the more commonly used insertion sort).
As it currently stands, this question is not a good fit for our Q&A format. We expect answers to be supported by facts, references, or expertise, but this question will likely solicit debate, arguments, polling, or extended discussion. If you feel that this question can be improved and possibly reopened, visit the help center for guidance.
Closed 10 years ago.
I am learning algorithms and need you guys to help me. I am a beginner so forgive me if my question is not clear. Whiles am learning i am seeing something like NlogN, N^2 etc.. and something like that.
I don't really understand it clearly when it comes to checking the efficiency/performance of different algorithms using these notations. I understand Logarithms very well but the way they are been used in relation to checking algorithms performance makes me mad.
I am asking if someone can point me to a tutorial where such notations are been explained so that i can get the basics very well. I really want to understand them and am willing to learn.
Thanks for your help.
Kap.
What you've described is called big O notation. Here is a guide explaining it.
An important thing to notice is that the notation ignores insignificant terms. If your algorithm takes 6X^2 + 3X + 12 seconds to execute, where X is the number of data points being processed, just call it O(X^2) because when X gets big, the 6 won't really make a difference, nor will the 3 nor the 12.
Buy Introduction to Algorithms. You can get a second hand version at an affordable price.
And/or view these great online video lectures from MIT built around aforementioned book.
By viewing those lectures, you'll understand how some algorithms have logarithmic time complexity, whereas some have exponential, etc.
Those are just functions, receiving the number of items in input, and returning how many operations it takes to complete the algorithm (usually they return the limiting factor of the algorithm, and not a specific function.. more on that - here).
http://www.amazon.com/Structures-Algorithm-Analysis-Allen-Weiss/dp/0805390529 is one of the best books which will explain Algorithms in excellent way.
--Cheers
You're talking about Big-O notation. This notation is a way of describing the worst possible running time of an algorithm as a function of its input size.
O(n^2) means that if the input has a size of n (such as a list with n elements in it), the algorithm will require n^2 passes through to execute in the worst-case scenarion (Big-O is worst case; there are other notations for best-case and average case). This could happen if you have a a for loop nested inside another, and both run from 1 to n.
O(nlogn) is similar. It usually happens when you are traversing a tree structure (such as a binary tree).
Note that you will probably never see something like O(3n) because for very large values of n, the constant 3 doesn't matter much, so it would be simplified to O(n).
Many of the others have already posted good links to read at.
As it currently stands, this question is not a good fit for our Q&A format. We expect answers to be supported by facts, references, or expertise, but this question will likely solicit debate, arguments, polling, or extended discussion. If you feel that this question can be improved and possibly reopened, visit the help center for guidance.
Closed 10 years ago.
I was having a look at this question and then reading about Tarjan's least common ancestors algorithm. I never came across any applications of LCA algorithms before.
Where are such LCA algorithms commonly used?
In compilers, the LCA of two basic blocks is a place you can put a computation so it is available to both. This might be useful for eliminating common subexpressions, or inserting phi-node for SSA conversion. These algorithms are well evolved and highly optimized, though, so the LCA itself may be hard to see, e.g., SSA and PRE
Don't know where it is used, but I have a couple ideas where it might get used:
computer graphics: often 3D sceneries get split into cubes which form a tree structure. If you have an object which is contained in two such cubes a LCA algorithm gives you the smallest containing larger cube.
analysis of gens in order to find the relationships between species and their lowest common ancestor
merge algorithms of version control systems
I just wrote a blog post about how I had to implement my own algorithm for this problem (extended to a set of nodes with an arbitrary length) for a taxonomy tree in the context of metagenomics:
http://blog.bio4j.com/2012/02/finding-the-lowest-common-ancestor-of-a-set-of-ncbi-taxonomy-nodes-with-bio4j/
Cheers,
Pablo