I'm reading a book Pearls of Functional Algorithm Design. Tried implementing the divide and conquer solution for smallest free number problem.
minfree xs = minfrom 0 (length xs) xs
minfrom a 0 _ = a
minfrom a n xs = if m == b - a
then minfrom b (n-m) vs
else minfrom a (m) us
where b = a + 1 + (n `div` 2)
(us,vs) = partition (<b) xs
m = length us
But this one works no faster than the solution that one might call "naive" solution. Which is
import Data.List ((\\))
minfree' = head . (\\) [0..]
I don't know why this is like this, what's wrong with the divide and conquer algorithm and how to improve it.
Tried using BangPatterns, implementing the version of partition that also returns first list's length in the tuple, so it eliminates additional traversal for m =length us. None of them made improvement.
First one takes more than 5 seconds, whereas second one does it almost instantly in ghci on input [0..9999999].
You have pathological input on which head . (\\) [0..] performs in O(N) time. \\ is defined as follows:
(\\) = foldl (flip delete)
delete x xs is an O(N) operation that removes the first x from xs. foldl (flip delete) xs ys deletes all elements of ys from xs one by one.
In [0..] \\ [0..9999999], we always find the next element to be deleted at the head of the list, so the result can be evaluated in linear time.
If you instead type minfree' (reverse [0..9999999]) into GHCi, that takes quadratic time and you find that it pretty much never finishes.
The divide-and-conquer algorithm on the other hand would not slow down on the reversed input.
Related
In the HaskellWiki https://wiki.haskell.org/Performance/Laziness they introduce the merge-sort function as non-lazy
merge_sort [] = []
merge_sort [x] = [x]
merge_sort lst = let (e,o) = cleave lst
in merge (merge_sort e) (merge_sort o) where
merge :: (Ord a) => [a] -> [a] -> [a]
merge xs [] = xs
merge [] ys = ys
merge xs#(x:t) ys#(y:u)
| x <= y = x : merge t ys
| otherwise = y : merge xs u
since you first have to recursively cleave the list
cleave = cleave' ([],[]) where
cleave' (eacc,oacc) [] = (eacc,oacc)
cleave' (eacc,oacc) [x] = (x:eacc,oacc)
cleave' (eacc,oacc) (x:x':xs) = cleave' (x:eacc,x':oacc) xs
and then, going up the reduction layers, merge these. So a merge-sort runs in n(log n) time. But the composition
min xs = head . merge_sort $ xs
supposedly runs in linear time. I can't see why, as you still have to cleave every sublist until you arrive at the singleton/empty lists and then merge these to guarantee the first element of the returned list is the smallest of all. What am I missing?
But lazyness still comes into play with the definitions like min xs = head . merge_sort $ xs. In finding the minimal element this way only the necessary amount of comparisons between elements will be performed (O(n) a.o.t. O(nlogn) comparisons needed to fully sort the whole list).
You are right, it will have a time complexity of O(n log(n)), however if you read the above paragraph carefully you will see, that it is talking about the amount of comparisons. Only O(n) comparisions will be performed, because every merge application only has to produce one element, so it only has to compare the first two elements of its arguments. So you get n/2 comparisons at the leaves of the recursion plus n/4 one level up, then n/4,... all the way up to the top-level of the recursion. If you work it out you get n-1 comparisons.
I am having problems with anyalysing the time complexity of an algorithm.
For example the following Haskell code, which will sort a list.
sort xs
|isSorted xs= xs
|otherwise= sort (check xs)
where
isSorted xs=all (==True) (zipWith (<=) xs ( drop 1 xs))
check [] =[]
check [x]=[x]
check (x:y:xs)
|x<=y = x:check (y:xs)
|otherwise=y:check (x:xs)
So for n being the length of the list and t_isSorted(n) the running time function: there is an constant t_drop(n) =c and t_all(n)=n, t_zipWith(n)=n :
t_isSorted(n)= c + n +n
For t_check:
t_check(1)=c1
t_check(n)=c2 + t_check(n-1), c2= for comparing and changing an element
.
.
.
t_check(n)=i*c2 + tcheck_(n-i), with i=n-1
=(n-1)*c2 + t_check(1)
=n*c2 - c2 + c1
And how exactly do I have to combine those to get t_sort(n)? I guess in the worst- case, sort xs has to run n-1 times.
isSorted is indeed O(n), since it's dominated by zipWith which in turn is O(n) since it does a linear pass over its argument.
check itself is O(n), since it only calls itself once per execution, and it always removes a constant number of elements from the list. The fastest sorting algorithm (without knowing something more about the list) runs in O(n*log(n)) (equivalent to O(log(n!)) time). There's a mathematical proof of this, and this algorithm is faster, so it cannot possibly be sorting the whole list.
check only moves things one step; it's effectively a single pass of bubble sort.
Consider sorting this list: [3,2,1]
check [3,2,1] = 2:(check [3,1]) -- since 3 > 2
check [3,1] = 1:(check [3]) -- since 3 > 1
check [3] = [3]
which would return the "sorted" list [2,1,3].
Then, as long as the list is not sorted, we loop. Since we might only put one element in its correct position (as 3 did in the example above), we might need O(n) loop iterations.
This totals at a time complexity of O(n) * O(n) = O(n^2)
The time complexity is O(n^2).
You're right, one step takes O(n) time (for both isSorted and check functions). It is called no more than n times (maybe even n - 1, it doesn't really matter for time complexity) (after the first call the largest element is guaranteed to be the last one, the same is the case for the second largest after the second call. We can prove that the last k elements are the largest and sorted properly after k calls). It swaps only adjacent elements, so it removes at most one inversion per step. As the number of inversions is O(n^2) in the worst case (namely, n * (n - 1) / 2), the time complexity is O(n^2).
I came up with an idea to solve another SO question and am hoping for some help with determining the function's complexity since I'm not too knowledgeable about that. Would I be correct in guessing that "unsorting" each of the chunks would be O (n * log 2) ? And then what would be the complexity of the sortBy function that compares the last of one chunk to the head of another? My guess is that that function would only compare pairs and not require finding one chunk's order in terms of the total list. Also, would Haskell offer different complexity because of a lazy optimization of the overall function? Thanks, in advance!
import Data.List.Split (chunksOf)
import Data.List (sortBy)
rearrange :: [Int] -> [Int]
rearrange = concat
. sortBy (\a b -> compare (last a) (head b))
. map (sortBy (\a b -> compare b a))
. chunksOf 2
Well step by step
chunksOf 2 must iterate through the whole list so O(n) and we half the length of the list. However, since constant multiples don't affect complexity we can ignore this.
map (sortBy... iterates through the whole list O(n) doing a constant time operation* O(1)=O(1*n) = O(n)
sortBy with a constant time comparison* is O( n * log n)
concat which is O(n)
So in total O(n + n + n log n + n) = O ((3 + log n) * n) = O(n log n)
*Since the lists are guarenteed to be of length 2 or less, we can say that operations like sorting and accessing the last element are O(2 * log 2) and O(2) respectively, which are both constant time, O(1)
Let's look at the parts in isolation (let n be the length of the list argument):
chunksOf 2 is O(n), resulting in a list of length (n+1) `quot` 2.
map (sortBy ...): Since all lists that are passed to sortBy ... have length <= 2, each of those sorts is O(1), and thus the entire map is O(n) again.
sortBy (\a b -> compare (last a) (head b)): The comparison is always O(1), since the lists whose last element is taken are of bounded length (<= 2), thus the entire sortBy operation is O(n*log n)
concat is O(n) again.
So overall, we have O(n*log n).
Note, however, that
cmp = \a b -> compare (last a) (head b)
is an inconsistent comparison, for two lists a and b (say [30,10] and [25,15]), you can have
cmp a b == cmp b a = LT
I'm not sure that your algorithm always works.
After looking at the implementation of sortBy and tracing the sort a bit in my head, I think that for the given purpose, it works (provided the list elements are distinct) and the inconsistent comparison does no harm. For some sorting algorithms, an inconsistent comparison might cause the sorting to loop, but for merge sort variants, that should not occur.
I have the following solution in Haskell to Problem 3:
isPrime :: Integer -> Bool
isPrime p = (divisors p) == [1, p]
divisors :: Integer -> [Integer]
divisors n = [d | d <- [1..n], n `mod` d == 0]
main = print (head (filter isPrime (filter ((==0) . (n `mod`)) [n-1,n-2..])))
where n = 600851475143
However, it takes more than the minute limit given by Project Euler. So how do I analyze the time complexity of my code to determine where I need to make changes?
Note: Please do not post alternative algorithms. I want to figure those out on my own. For now I just want to analyse the code I have and look for ways to improve it. Thanks!
Two things:
Any time you see a list comprehension (as you have in divisors), or equivalently, some series of map and/or filter functions over a list (as you have in main), treat its complexity as Θ(n) just the same as you would treat a for-loop in an imperative language.
This is probably not quite the sort of advice you were expecting, but I hope it will be more helpful: Part of the purpose of Project Euler is to encourage you to think about the definitions of various mathematical concepts, and about the many different algorithms that might correctly satisfy those definitions.
Okay, that second suggestion was a bit too nebulous... What I mean is, for example, the way you've implemented isPrime is really a textbook definition:
isPrime :: Integer -> Bool
isPrime p = (divisors p) == [1, p]
-- p is prime if its only divisors are 1 and p.
Likewise, your implementation of divisors is straightforward:
divisors :: Integer -> [Integer]
divisors n = [d | d <- [1..n], n `mod` d == 0]
-- the divisors of n are the numbers between 1 and n that divide evenly into n.
These definitions both read very nicely! Algorithmically, on the other hand, they are too naïve. Let's take a simple example: what are the divisors of the number 10? [1, 2, 5, 10]. On inspection, you probably notice a couple things:
1 and 10 are pairs, and 2 and 5 are pairs.
Aside from 10 itself, there can't be any divisors of 10 that are greater than 5.
You can probably exploit properties like these to optimize your algorithm, right? So, without looking at your code -- just using pencil and paper -- try sketching out a faster algorithm for divisors. If you've understood my hint, divisors n should run in sqrt n time. You'll find more opportunities along these lines as you continue. You might decide to redefine everything differently, in a way that doesn't use your divisors function at all...
Hope this helps give you the right mindset for tackling these problems!
Let's start from the top.
divisors :: Integer -> [Integer]
divisors n = [d | d <- [1..n], n `mod` d == 0]
For now, let's assume that certain things are cheap: incrementing numbers is O(1), doing mod operations is O(1), and comparisons with 0 are O(1). (These are false assumptions, but what the heck.) The divisors function loops over all numbers from 1 to n, and does an O(1) operation on each number, so computing the complete output is O(n). Notice that here when we say O(n), n is the input number, not the size of the input! Since it takes m=log(n) bits to store n, this function takes O(2^m) time in the size of the input to produce a complete answer. I'll use n and m consistently to mean the input number and input size below.
isPrime :: Integer -> Bool
isPrime p = (divisors p) == [1, p]
In the worst case, p is prime, which forces divisors to produce its whole output. Comparison to a list of statically-known length is O(1), so this is dominated by the call to divisors. O(n), O(2^m)
Your main function does a bunch of things at once, so let's break down subexpressions a bit.
filter ((==0) . (n `mod`))
This loops over a list, and does an O(1) operation on each element. This is O(m), where here m is the length of the input list.
filter isPrime
Loops over a list, doing O(n) work on each element, where here n is the largest number in the list. If the list happens to be n elements long (as it is in your case), this means this is O(n*n) work, or O(2^m*2^m) = O(4^m) work (as above, this analysis is for the case where it produces its entire list).
print . head
Tiny bits of work. Let's call it O(m) for the printing part.
main = print (head (filter isPrime (filter ((==0) . (n `mod`)) [n-1,n-2..])))
Considering all the subexpressions above, the filter isPrime bit is clearly the dominating factor. O(4^m), O(n^2)
Now, there's one final subtlety to consider: throughout the analysis above, I've consistently made the assumption that each function/subexpression was forced to produce its entire output. As we can see in main, this probably isn't true: we call head, which only forces a little bit of the list. However, if the input number itself isn't prime, we know for sure that we must look through at least half the list: there will certainly be no divisors between n/2 and n. So, at best, we cut our work in half -- which has no effect on the asymptotic cost.
Daniel Wagner's answer explains the general strategy of deriving bounds for the runtime complexity rather well. However, as is usually the case for general strategies, it yields too conservative bounds.
So, just for the heck of it, let's investigate this example in some more detail.
main = print (head (filter isPrime (filter ((==0) . (n `mod`)) [n-1,n-2..])))
where n = 600851475143
(Aside: if n were prime, this would cause a runtime error when checking n `mod` 0 == 0, thus I change the list to [n, n-1 .. 2] so that the algorithm works for all n > 1.)
Let's split up the expression into its parts, so we can see and analyse each part more easily
main = print answer
where
n = 600851475143
candidates = [n, n-1 .. 2]
divisorsOfN = filter ((== 0) . (n `mod`)) candidates
primeDivisors = filter isPrime divisorsOfN
answer = head primeDivisors
Like Daniel, I work with the assumption that arithmetic operations, comparisons etc. are O(1) - although not true, that's a good enough approximation for all remotely reasonable inputs.
So, of the list candidates, the elements from n down to answer have to be generated, n - answer + 1 elements, for a total cost of O(n - answer + 1). For composite n, we have answer <= n/2, then that's Θ(n).
Generating the list of divisors as far as needed is then Θ(n - answer + 1) too.
For the number d(n) of divisors of n, we can use the coarse estimate d(n) <= 2√n.
All divisors >= answer of n have to be checked for primality, that's at least half of all divisors.
Since the list of divisors is lazily generated, the complexity of
isPrime :: Integer -> Bool
isPrime p = (divisors p) == [1, p]
is O(smallest prime factor of p), because as soon as the first divisor > 1 is found, the equality test is determined. For composite p, the smallest prime factor is <= √p.
We have < 2√n primality checks of complexity at worst O(√n), and one check of complexity Θ(answer), so the combined work of all prime tests carried out is O(n).
Summing up, the total work needed is O(n), since the cost of each step is O(n) at worst.
In fact, the total work done in this algorithm is Θ(n). If n is prime, generating the list of divisors as far as needed is done in O(1), but the prime test is Θ(n). If n is composite, answer <= n/2, and generating the list of divisors as far as needed is Θ(n).
If we don't consider the arithmetic operations to be O(1), we have to multiply with the complexity of an arithmetic operation on numbers the size of n, that is O(log n) bits, which, depending on the algorithms used, usually gives a factor slightly above log n and below (log n)^2.
(This question is related to my previous question, or rather to my answer to it.)
I want to store all qubes of natural numbers in a structure and look up specific integers to see if they are perfect cubes.
For example,
cubes = map (\x -> x*x*x) [1..]
is_cube n = n == (head $ dropWhile (<n) cubes)
It is much faster than calculating the cube root, but It has complexity of O(n^(1/3)) (am I right?).
I think, using a more complex data structure would be better.
For example, in C I could store a length of an already generated array (not list - for faster indexing) and do a binary search. It would be O(log n) with lower сoefficient than in another answer to that question. The problem is, I can't express it in Haskell (and I don't think I should).
Or I can use a hash function (like mod). But I think it would be much more memory consuming to have several lists (or a list of lists), and it won't lower the complexity of lookup (still O(n^(1/3))), only a coefficient.
I thought about a kind of a tree, but without any clever ideas (sadly I've never studied CS). I think, the fact that all integers are ascending will make my tree ill-balanced for lookups.
And I'm pretty sure this fact about ascending integers can be a great advantage for lookups, but I don't know how to use it properly (see my first solution which I can't express in Haskell).
Several comments:
If you have finitely many cubes, put them in Data.IntSet. Lookup is logarithmic time. Algorithm is based on Patricia trees and a paper by Gill and Okasaki.
If you have infinitely many cubes in a sorted list, you can do binary search. start at index 1 and you'll double it logarithmically many times until you get something large enough, then logarithmically many more steps to find your integer or rule it out. But unfortunately with lists, every lookup is proportional to the size of the index. And you can't create an infinite array with constant-time lookup.
With that background, I propose the following data structure:
A sorted list of sorted arrays of cubes. The array at position i contains exp(2,i) elements.
You then have a slightly more complicated form of binary search.
I'm not awake enough to do the analysis off the top of my head, but I believe this gets you to O((log n)^2) worst case.
You can do fibonacci-search (or any other you wuld like) over lazy infinite tree:
data Tree a = Empty
| Leaf a
| Node a (Tree a) (Tree a)
rollout Empty = []
rollout (Leaf a) = [a]
rollout (Node x a b) = rollout a ++ x : rollout b
cubes = backbone 1 2 where
backbone a b = Node (b*b*b) (sub a b) (backbone (b+1) (a+b))
sub a b | (a+1) == b = Leaf (a*a*a)
sub a b | a == b = Empty
sub a b = subBackbone a (a+1) b
subBackbone a b c | b >= c = sub a c
subBackbone a b c = Node (b*b*b) (sub a b) (subBackbone (b+1) (a+b) c)
is_cube n = go cubes where
go Empty = False
go (Leaf x) = (x == n)
go (Node x a b) = case (compare n x) of
EQ -> True
LT -> go a
GT -> go b