I came up with an idea to solve another SO question and am hoping for some help with determining the function's complexity since I'm not too knowledgeable about that. Would I be correct in guessing that "unsorting" each of the chunks would be O (n * log 2) ? And then what would be the complexity of the sortBy function that compares the last of one chunk to the head of another? My guess is that that function would only compare pairs and not require finding one chunk's order in terms of the total list. Also, would Haskell offer different complexity because of a lazy optimization of the overall function? Thanks, in advance!
import Data.List.Split (chunksOf)
import Data.List (sortBy)
rearrange :: [Int] -> [Int]
rearrange = concat
. sortBy (\a b -> compare (last a) (head b))
. map (sortBy (\a b -> compare b a))
. chunksOf 2
Well step by step
chunksOf 2 must iterate through the whole list so O(n) and we half the length of the list. However, since constant multiples don't affect complexity we can ignore this.
map (sortBy... iterates through the whole list O(n) doing a constant time operation* O(1)=O(1*n) = O(n)
sortBy with a constant time comparison* is O( n * log n)
concat which is O(n)
So in total O(n + n + n log n + n) = O ((3 + log n) * n) = O(n log n)
*Since the lists are guarenteed to be of length 2 or less, we can say that operations like sorting and accessing the last element are O(2 * log 2) and O(2) respectively, which are both constant time, O(1)
Let's look at the parts in isolation (let n be the length of the list argument):
chunksOf 2 is O(n), resulting in a list of length (n+1) `quot` 2.
map (sortBy ...): Since all lists that are passed to sortBy ... have length <= 2, each of those sorts is O(1), and thus the entire map is O(n) again.
sortBy (\a b -> compare (last a) (head b)): The comparison is always O(1), since the lists whose last element is taken are of bounded length (<= 2), thus the entire sortBy operation is O(n*log n)
concat is O(n) again.
So overall, we have O(n*log n).
Note, however, that
cmp = \a b -> compare (last a) (head b)
is an inconsistent comparison, for two lists a and b (say [30,10] and [25,15]), you can have
cmp a b == cmp b a = LT
I'm not sure that your algorithm always works.
After looking at the implementation of sortBy and tracing the sort a bit in my head, I think that for the given purpose, it works (provided the list elements are distinct) and the inconsistent comparison does no harm. For some sorting algorithms, an inconsistent comparison might cause the sorting to loop, but for merge sort variants, that should not occur.
Related
Suppose I've got a natural number n and I want a list (or whatever) of all primes up to n.
The classic prime sieve algorithm runs in O(n log n) time and O(n) space -- it's fine for more imperative languages, but requires in-place modification to lists and random access, in a fundamental way.
There's a functional version involving priority queues, which is pretty slick -- you can check it out here. This has better space complexity at about O(n / log(n)) (asymptotically better but debatable at practical scales). Unfortunately the time analysis is nasty, but it's very nearly O(n^2) (actually, I think it's about O(n log(n) Li(n)), but log(n) Li(n) is approximately n).
Asymptotically speaking it would actually be better just to check the primality of each number as you generate it, using successive trial division, as that would take only O(1) space and O(n^{3/2}) time. Is there a better way?
Edit: it turns out my calculations were simply incorrect. The algorithm in the article is O(n (log n) (log log n)), which the articles explains and proves (and see the answer below), not the complicated mess I put above. I'd still enjoy seeing a bona-fide O(n log log n) pure algorithm if there is one out there.
Here's a Haskell implementation of Melissa O'Neill's algorithm (from the linked article). Unlike the implementation that Gassa linked to, I've made minimal use of laziness, so that the performance analysis is clear -- O(n log n log log n), i.e., linearithmic in n log log n, the number of writes made by the imperative Sieve of Eratosthenes.
The heap implementation is just a tournament tree. The balancing logic is in push; by swapping the children every time, we ensure that, for every branch, the left subtree is the same size or one bigger compared to the right subtree, which ensures depth O(log n).
module Sieve where
type Nat = Int
data Heap = Leaf !Nat !Nat
| Branch !Nat !Heap !Heap
deriving Show
top :: Heap -> Nat
top (Leaf n _) = n
top (Branch n _ _) = n
leaf :: Nat -> Heap
leaf p = Leaf (3 * p) p
branch :: Heap -> Heap -> Heap
branch h1 h2 = Branch (min (top h1) (top h2)) h1 h2
pop :: Heap -> Heap
pop (Leaf n p) = Leaf (n + 2 * p) p
pop (Branch _ h1 h2)
= case compare (top h1) (top h2) of
LT -> branch (pop h1) h2
EQ -> branch (pop h1) (pop h2)
GT -> branch h1 (pop h2)
push :: Nat -> Heap -> Heap
push p h#(Leaf _ _) = branch (leaf p) h
push p (Branch _ h1 h2) = branch (push p h2) h1
primes :: [Nat]
primes
= let helper n h
= case compare n (top h) of
LT -> n : helper (n + 2) (push n h)
EQ -> helper (n + 2) (pop h)
GT -> helper n (pop h)
in 2 : 3 : helper 5 (leaf 3)
Here it is, if (Haskell's) pure arrays count as pure (they should, IMO). The complexity is obviously O(n log (log n)), provided accumArray indeed spends O(1) time for each entry it is given, as it ought to:
import Data.Array.Unboxed
import Data.List (tails, inits)
primes = 2 : [ n |
(r:q:_, px) <- zip (tails (2 : [p^2 | p <- primes]))
(inits primes),
(n,True) <- assocs ( accumArray (\_ _ -> False) True
(r+1,q-1)
[ (m,()) | p <- px
, let s = div (r+p) p * p
, m <- [s,s+p..q-1]]
:: UArray Int Bool ) ]
Calculates primes by segments between successive squares of primes, generating the composites by enumerating the multiples of the corresponding prefixes of the primes list (using inits), just as any proper sieve of Eratosthenes would, by repeated additions.
So, the primes {2,3} are used to sieve a segment from 10 to 24; {2,3,5} from 26 to 48; and so on. See also.
Also, a Python generator-based sieve might be considered functional as well. Python's dicts are extremely well-performing, empirically, though I'm not sure about the exact cost of the multiples over-producing scheme used there to avoid duplicate composites.
update: testing it indeed produces favorable results, as expected:
{- original heap tweaked nested-feed array-based
(3*p,p) (p*p,2*p) JBwoVL abPSOx
6Uv0cL 2x speed-up another 3x+ speed-up
n^ n^ n^ n^
100K: 0.78s 0.38s 0.13s 0.065s
200K: 2.02s 1.37 0.97s 1.35 0.29s 1.16 0.13s 1.00
400K: 5.05s 1.32 2.40s 1.31 0.70s 1.27 0.29s 1.16
800K: 12.37s 1.29 1M: 2.10s 1.20 0.82s 1.13
2M: 1.71s 1.06
4M: 3.72s 1.12
10M: 9.84s 1.06
overall in the tested range:
1.33 1.21 1.09
-}
with empirical orders of growth calculated for producing n primes, where O(n log log n) is commonly seen as n1.05...1.10 and O(n log n log log n) as n1.20...1.25.
The "nested-feed" variant implements the postponement technique (as also seen in the above linked Python answer) that achieves quadratic reduction of the heap size which evidently has a noticeable impact on the empirical complexity, even if not quite reaching the still better results for the array-based code of this answer, which is able to produce 10 million primes in under 10 seconds on ideone.com (with overall growth rate of just n1.09 in the tested range).
("original heap" is of course the code from the other answer here).
I derived a prime generating function (generates all primes in order) a while back. Created a 6 pages proof for it as well. I think it is the first prime generating function in history actually (At least I couldn't find any other examples).
Here it is:
(-1)^((4*gamma(x)+4)/x)-1
Not sure how fast it can be computed. It returns 0 for all primes (or maybe it was 1, can't remember). Gamma function is essentially factorial so that can be fast early on. Raising negative 1 to a fractional exponent is a whole other beast though, I believe it uses integrals in base_e possible, or maybe some trigonometric functions; can't remember.
I don't know LaTeX so if someone wants to edit my post and include a LaTeX version that would be amazing!
I am having problems with anyalysing the time complexity of an algorithm.
For example the following Haskell code, which will sort a list.
sort xs
|isSorted xs= xs
|otherwise= sort (check xs)
where
isSorted xs=all (==True) (zipWith (<=) xs ( drop 1 xs))
check [] =[]
check [x]=[x]
check (x:y:xs)
|x<=y = x:check (y:xs)
|otherwise=y:check (x:xs)
So for n being the length of the list and t_isSorted(n) the running time function: there is an constant t_drop(n) =c and t_all(n)=n, t_zipWith(n)=n :
t_isSorted(n)= c + n +n
For t_check:
t_check(1)=c1
t_check(n)=c2 + t_check(n-1), c2= for comparing and changing an element
.
.
.
t_check(n)=i*c2 + tcheck_(n-i), with i=n-1
=(n-1)*c2 + t_check(1)
=n*c2 - c2 + c1
And how exactly do I have to combine those to get t_sort(n)? I guess in the worst- case, sort xs has to run n-1 times.
isSorted is indeed O(n), since it's dominated by zipWith which in turn is O(n) since it does a linear pass over its argument.
check itself is O(n), since it only calls itself once per execution, and it always removes a constant number of elements from the list. The fastest sorting algorithm (without knowing something more about the list) runs in O(n*log(n)) (equivalent to O(log(n!)) time). There's a mathematical proof of this, and this algorithm is faster, so it cannot possibly be sorting the whole list.
check only moves things one step; it's effectively a single pass of bubble sort.
Consider sorting this list: [3,2,1]
check [3,2,1] = 2:(check [3,1]) -- since 3 > 2
check [3,1] = 1:(check [3]) -- since 3 > 1
check [3] = [3]
which would return the "sorted" list [2,1,3].
Then, as long as the list is not sorted, we loop. Since we might only put one element in its correct position (as 3 did in the example above), we might need O(n) loop iterations.
This totals at a time complexity of O(n) * O(n) = O(n^2)
The time complexity is O(n^2).
You're right, one step takes O(n) time (for both isSorted and check functions). It is called no more than n times (maybe even n - 1, it doesn't really matter for time complexity) (after the first call the largest element is guaranteed to be the last one, the same is the case for the second largest after the second call. We can prove that the last k elements are the largest and sorted properly after k calls). It swaps only adjacent elements, so it removes at most one inversion per step. As the number of inversions is O(n^2) in the worst case (namely, n * (n - 1) / 2), the time complexity is O(n^2).
I'm reading a book Pearls of Functional Algorithm Design. Tried implementing the divide and conquer solution for smallest free number problem.
minfree xs = minfrom 0 (length xs) xs
minfrom a 0 _ = a
minfrom a n xs = if m == b - a
then minfrom b (n-m) vs
else minfrom a (m) us
where b = a + 1 + (n `div` 2)
(us,vs) = partition (<b) xs
m = length us
But this one works no faster than the solution that one might call "naive" solution. Which is
import Data.List ((\\))
minfree' = head . (\\) [0..]
I don't know why this is like this, what's wrong with the divide and conquer algorithm and how to improve it.
Tried using BangPatterns, implementing the version of partition that also returns first list's length in the tuple, so it eliminates additional traversal for m =length us. None of them made improvement.
First one takes more than 5 seconds, whereas second one does it almost instantly in ghci on input [0..9999999].
You have pathological input on which head . (\\) [0..] performs in O(N) time. \\ is defined as follows:
(\\) = foldl (flip delete)
delete x xs is an O(N) operation that removes the first x from xs. foldl (flip delete) xs ys deletes all elements of ys from xs one by one.
In [0..] \\ [0..9999999], we always find the next element to be deleted at the head of the list, so the result can be evaluated in linear time.
If you instead type minfree' (reverse [0..9999999]) into GHCi, that takes quadratic time and you find that it pretty much never finishes.
The divide-and-conquer algorithm on the other hand would not slow down on the reversed input.
I have the following solution in Haskell to Problem 3:
isPrime :: Integer -> Bool
isPrime p = (divisors p) == [1, p]
divisors :: Integer -> [Integer]
divisors n = [d | d <- [1..n], n `mod` d == 0]
main = print (head (filter isPrime (filter ((==0) . (n `mod`)) [n-1,n-2..])))
where n = 600851475143
However, it takes more than the minute limit given by Project Euler. So how do I analyze the time complexity of my code to determine where I need to make changes?
Note: Please do not post alternative algorithms. I want to figure those out on my own. For now I just want to analyse the code I have and look for ways to improve it. Thanks!
Two things:
Any time you see a list comprehension (as you have in divisors), or equivalently, some series of map and/or filter functions over a list (as you have in main), treat its complexity as Θ(n) just the same as you would treat a for-loop in an imperative language.
This is probably not quite the sort of advice you were expecting, but I hope it will be more helpful: Part of the purpose of Project Euler is to encourage you to think about the definitions of various mathematical concepts, and about the many different algorithms that might correctly satisfy those definitions.
Okay, that second suggestion was a bit too nebulous... What I mean is, for example, the way you've implemented isPrime is really a textbook definition:
isPrime :: Integer -> Bool
isPrime p = (divisors p) == [1, p]
-- p is prime if its only divisors are 1 and p.
Likewise, your implementation of divisors is straightforward:
divisors :: Integer -> [Integer]
divisors n = [d | d <- [1..n], n `mod` d == 0]
-- the divisors of n are the numbers between 1 and n that divide evenly into n.
These definitions both read very nicely! Algorithmically, on the other hand, they are too naïve. Let's take a simple example: what are the divisors of the number 10? [1, 2, 5, 10]. On inspection, you probably notice a couple things:
1 and 10 are pairs, and 2 and 5 are pairs.
Aside from 10 itself, there can't be any divisors of 10 that are greater than 5.
You can probably exploit properties like these to optimize your algorithm, right? So, without looking at your code -- just using pencil and paper -- try sketching out a faster algorithm for divisors. If you've understood my hint, divisors n should run in sqrt n time. You'll find more opportunities along these lines as you continue. You might decide to redefine everything differently, in a way that doesn't use your divisors function at all...
Hope this helps give you the right mindset for tackling these problems!
Let's start from the top.
divisors :: Integer -> [Integer]
divisors n = [d | d <- [1..n], n `mod` d == 0]
For now, let's assume that certain things are cheap: incrementing numbers is O(1), doing mod operations is O(1), and comparisons with 0 are O(1). (These are false assumptions, but what the heck.) The divisors function loops over all numbers from 1 to n, and does an O(1) operation on each number, so computing the complete output is O(n). Notice that here when we say O(n), n is the input number, not the size of the input! Since it takes m=log(n) bits to store n, this function takes O(2^m) time in the size of the input to produce a complete answer. I'll use n and m consistently to mean the input number and input size below.
isPrime :: Integer -> Bool
isPrime p = (divisors p) == [1, p]
In the worst case, p is prime, which forces divisors to produce its whole output. Comparison to a list of statically-known length is O(1), so this is dominated by the call to divisors. O(n), O(2^m)
Your main function does a bunch of things at once, so let's break down subexpressions a bit.
filter ((==0) . (n `mod`))
This loops over a list, and does an O(1) operation on each element. This is O(m), where here m is the length of the input list.
filter isPrime
Loops over a list, doing O(n) work on each element, where here n is the largest number in the list. If the list happens to be n elements long (as it is in your case), this means this is O(n*n) work, or O(2^m*2^m) = O(4^m) work (as above, this analysis is for the case where it produces its entire list).
print . head
Tiny bits of work. Let's call it O(m) for the printing part.
main = print (head (filter isPrime (filter ((==0) . (n `mod`)) [n-1,n-2..])))
Considering all the subexpressions above, the filter isPrime bit is clearly the dominating factor. O(4^m), O(n^2)
Now, there's one final subtlety to consider: throughout the analysis above, I've consistently made the assumption that each function/subexpression was forced to produce its entire output. As we can see in main, this probably isn't true: we call head, which only forces a little bit of the list. However, if the input number itself isn't prime, we know for sure that we must look through at least half the list: there will certainly be no divisors between n/2 and n. So, at best, we cut our work in half -- which has no effect on the asymptotic cost.
Daniel Wagner's answer explains the general strategy of deriving bounds for the runtime complexity rather well. However, as is usually the case for general strategies, it yields too conservative bounds.
So, just for the heck of it, let's investigate this example in some more detail.
main = print (head (filter isPrime (filter ((==0) . (n `mod`)) [n-1,n-2..])))
where n = 600851475143
(Aside: if n were prime, this would cause a runtime error when checking n `mod` 0 == 0, thus I change the list to [n, n-1 .. 2] so that the algorithm works for all n > 1.)
Let's split up the expression into its parts, so we can see and analyse each part more easily
main = print answer
where
n = 600851475143
candidates = [n, n-1 .. 2]
divisorsOfN = filter ((== 0) . (n `mod`)) candidates
primeDivisors = filter isPrime divisorsOfN
answer = head primeDivisors
Like Daniel, I work with the assumption that arithmetic operations, comparisons etc. are O(1) - although not true, that's a good enough approximation for all remotely reasonable inputs.
So, of the list candidates, the elements from n down to answer have to be generated, n - answer + 1 elements, for a total cost of O(n - answer + 1). For composite n, we have answer <= n/2, then that's Θ(n).
Generating the list of divisors as far as needed is then Θ(n - answer + 1) too.
For the number d(n) of divisors of n, we can use the coarse estimate d(n) <= 2√n.
All divisors >= answer of n have to be checked for primality, that's at least half of all divisors.
Since the list of divisors is lazily generated, the complexity of
isPrime :: Integer -> Bool
isPrime p = (divisors p) == [1, p]
is O(smallest prime factor of p), because as soon as the first divisor > 1 is found, the equality test is determined. For composite p, the smallest prime factor is <= √p.
We have < 2√n primality checks of complexity at worst O(√n), and one check of complexity Θ(answer), so the combined work of all prime tests carried out is O(n).
Summing up, the total work needed is O(n), since the cost of each step is O(n) at worst.
In fact, the total work done in this algorithm is Θ(n). If n is prime, generating the list of divisors as far as needed is done in O(1), but the prime test is Θ(n). If n is composite, answer <= n/2, and generating the list of divisors as far as needed is Θ(n).
If we don't consider the arithmetic operations to be O(1), we have to multiply with the complexity of an arithmetic operation on numbers the size of n, that is O(log n) bits, which, depending on the algorithms used, usually gives a factor slightly above log n and below (log n)^2.
(This question is related to my previous question, or rather to my answer to it.)
I want to store all qubes of natural numbers in a structure and look up specific integers to see if they are perfect cubes.
For example,
cubes = map (\x -> x*x*x) [1..]
is_cube n = n == (head $ dropWhile (<n) cubes)
It is much faster than calculating the cube root, but It has complexity of O(n^(1/3)) (am I right?).
I think, using a more complex data structure would be better.
For example, in C I could store a length of an already generated array (not list - for faster indexing) and do a binary search. It would be O(log n) with lower сoefficient than in another answer to that question. The problem is, I can't express it in Haskell (and I don't think I should).
Or I can use a hash function (like mod). But I think it would be much more memory consuming to have several lists (or a list of lists), and it won't lower the complexity of lookup (still O(n^(1/3))), only a coefficient.
I thought about a kind of a tree, but without any clever ideas (sadly I've never studied CS). I think, the fact that all integers are ascending will make my tree ill-balanced for lookups.
And I'm pretty sure this fact about ascending integers can be a great advantage for lookups, but I don't know how to use it properly (see my first solution which I can't express in Haskell).
Several comments:
If you have finitely many cubes, put them in Data.IntSet. Lookup is logarithmic time. Algorithm is based on Patricia trees and a paper by Gill and Okasaki.
If you have infinitely many cubes in a sorted list, you can do binary search. start at index 1 and you'll double it logarithmically many times until you get something large enough, then logarithmically many more steps to find your integer or rule it out. But unfortunately with lists, every lookup is proportional to the size of the index. And you can't create an infinite array with constant-time lookup.
With that background, I propose the following data structure:
A sorted list of sorted arrays of cubes. The array at position i contains exp(2,i) elements.
You then have a slightly more complicated form of binary search.
I'm not awake enough to do the analysis off the top of my head, but I believe this gets you to O((log n)^2) worst case.
You can do fibonacci-search (or any other you wuld like) over lazy infinite tree:
data Tree a = Empty
| Leaf a
| Node a (Tree a) (Tree a)
rollout Empty = []
rollout (Leaf a) = [a]
rollout (Node x a b) = rollout a ++ x : rollout b
cubes = backbone 1 2 where
backbone a b = Node (b*b*b) (sub a b) (backbone (b+1) (a+b))
sub a b | (a+1) == b = Leaf (a*a*a)
sub a b | a == b = Empty
sub a b = subBackbone a (a+1) b
subBackbone a b c | b >= c = sub a c
subBackbone a b c = Node (b*b*b) (sub a b) (subBackbone (b+1) (a+b) c)
is_cube n = go cubes where
go Empty = False
go (Leaf x) = (x == n)
go (Node x a b) = case (compare n x) of
EQ -> True
LT -> go a
GT -> go b