Is there in the Standard Prelude functions which implement the union and the intersection of sets ?
union :: (Eq a) => [a] -> [a] -> [a]
intersect :: (Eq a) => [a] -> [a] -> [a]
If no, may somebody can said if my implementation is efficient, (make good use of laziness and prelude function)
unionSet :: (Eq a) => [a] -> [a] -> [a]
unionSet as bs = foldl (\xs y -> if elem y xs then xs else xs ++ [y]) as bs
intersectSet :: (Eq a) => [a] -> [a] -> [a]
intersectSet as bs = let ns = [ a | a <- as, elem a bs] in [ b | b <- bs, elem b ns]
There are union and intersect functions on lists in the standard libraries, located in Data.List but not in the Prelude itself.
As far as efficiency goes, I'm going to say "no" to all of the above, both yours and the standard library's. There's really no way either can ever be efficient operations on a list with only an Eq constraint. That said, you may still find the implementation in Data.List informative--see the links above, which I've pointed directly to the relevant source.
Edit -- As a brief addendum for the sake of posterity, be sure to see Don's answer for what you actually want to use for this purpose, rather than the narrower question of "do these functions exist at all".
The base library provides list versions, as camccann points out. If you want something a bit more efficient, consider Data.Set, which provides:
union :: Ord a => Set a -> Set a -> Set a
intersection :: Ord a => Set a -> Set a -> Set a
with complexity O(n+m).
Related
Specifically I'm searching for a function 'maximumWith',
maximumWith :: (Foldable f, Ord b) => (a -> b) -> f a -> a
Which behaves in the following way:
maximumWith length [[1, 2], [0, 1, 3]] == [0, 1, 3]
maximumWith null [[(+), (*)], []] == []
maximumWith (const True) x == head x
My use case is picking the longest word in a list.
For this I'd like something akin to maximumWith length.
I'd thought such a thing existed, since sortWith etc. exist.
Let me collect all the notes in the comments together...
Let's look at sort. There are 4 functions in the family:
sortBy is the actual implementation.
sort = sortBy compare uses Ord overloading.
sortWith = sortBy . comparing is the analogue of your desired maximumWith. However, this function has an issue. The ranking of an element is given by applying the given mapping function to it. However, the ranking is not memoized, so if an element needs to compared multiple times, the ranking will be recomputed. You can only use it guilt-free if the ranking function is very cheap. Such functions include selectors (e.g. fst), and newtype constructors. YMMV on simple arithmetic and data constructors. Between this inefficiency, the simplicity of the definition, and its location in GHC.Exts, it's easy to deduce that it's not used that often.
sortOn fixes the inefficiency by decorating each element with its image under the ranking function in a pair, sorting by the ranks, and then erasing them.
The first two have analogues in maximum: maximumBy and maximum. sortWith has no analogy; you may as well write out maximumBy (comparing _) every time. There is also no maximumOn, even though such a thing would be more efficient. The easiest way to define a maximumOn is probably just to copy sortOn:
maximumOn :: (Functor f, Foldable f, Ord r) => (a -> r) -> f a -> a
maximumOn rank = snd . maximumBy (comparing fst) . fmap annotate
where annotate e = let r = rank e in r `seq` (r, e)
There's a bit of interesting code in maximumBy that keeps this from optimizing properly on lists. It also works to use
maximumOn :: (Foldable f, Ord r) => (a -> r) -> f a -> a
maximumOn rank = snd . fromJust . foldl' max' Nothing
where max' Nothing x = let r = rank x in r `seq` Just (r, x)
max' old#(Just (ro, xo)) xn = let rn = rank xn
in case ro `compare` rn of
LT -> Just (rn, xo)
_ -> old
These pragmas may be useful:
{-# SPECIALIZE maximumOn :: Ord r => (a -> r) -> [a] -> a #-}
{-# SPECIALIZE maximumOn :: (a -> Int) -> [a] -> a #-}
HTNW has explained how to do what you asked, but I figured I should mention that for the specific application you mentioned, there's a way that's more efficient in certain cases (assuming the words are represented by Strings). Suppose you want
longest :: [[a]] -> [a]
If you ask for maximumOn length [replicate (10^9) (), []], then you'll end up calculating the length of a very long list unnecessarily. There are several ways to work around this problem, but here's how I'd do it:
data MS a = MS
{ _longest :: [a]
, _longest_suffix :: [a]
, _longest_bound :: !Int }
We will ensure that longest is the first of the longest strings seen thus far, and that longest_bound + length longest_suffix = length longest.
step :: MS a -> [a] -> MS a
step (MS longest longest_suffix longest_bound) xs =
go longest_bound longest_suffix xs'
where
-- the new list is not longer
go n suffo [] = MS longest suffo n
-- the new list is longer
go n [] suffn = MS xs suffn n
-- don't know yet
go !n (_ : suffo) (_ : suffn) =
go (n + 1) suffo suffn
xs' = drop longest_bound xs
longest :: [[a]] -> [a]
longest = _longest . foldl' step (MS [] [] 0)
Now if the second to longest list has q elements, we'll walk at most q conses into each list. This is the best possible complexity. Of course, it's only significantly better than the maximumOn solution when the longest list is much longer than the second to longest.
I've been learning Haskell in my spare time working through LYAH. Would like to improve upon my Haskell (/ Functional programming) skills by solving some problems from the imperative world. One of the problems from EPI is to print an "almost sorted array", in a sorted fashion where it is guaranteed that no element in the array is more than k away from its correct position. The input is a stream of elements and the requirement is to do this in O(n log k) time complexity and O(k) space complexity.
I've attempted to re-implement the imperative solution in Haskell as follows:
import qualified Data.Heap as Heap
-- print the k-sorted list in a sorted fashion
ksorted :: (Ord a, Show a) => [a] -> Int -> IO ()
ksorted [] _ = return ()
ksorted xs k = do
heap <- ksorted' xs Heap.empty
mapM_ print $ (Heap.toAscList heap) -- print the remaining elements in the heap.
where
ksorted' :: (Ord a, Show a) => [a] -> Heap.MinHeap a -> IO (Heap.MinHeap a)
ksorted' [] h = return h
ksorted' (x:xs) h = do let (m, h') = getMinAndBuildHeap h x in
(printMin m >> ksorted' xs h')
printMin :: (Show a) => Maybe a -> IO ()
printMin m = case m of
Nothing -> return ()
(Just item) -> print item
getMinAndBuildHeap :: (Ord a, Show a) => Heap.MinHeap a -> a -> (Maybe a, Heap.MinHeap a)
getMinAndBuildHeap h item= if (Heap.size h) > k
then ((Heap.viewHead h), (Heap.insert item (Heap.drop 1 h)))
else (Nothing, (Heap.insert item h))
I would like to know a better way of solving this in Haskell. Any inputs would be appreciated.
[Edit 1]: The input is stream, but for now I assumed a list instead (with only a forward iterator/ input iterator in some sense.)
[Edit 2]: added Data.Heap import to the code.
Thanks.
I think the main improvement is to separate the production of the sorted list from the printing of the sorted list. So:
import Data.Heap (MinHeap)
import qualified Data.Heap as Heap
ksort :: Ord a => Int -> [a] -> [a]
ksort k xs = go (Heap.fromList b) e where
(b, e) = splitAt (k-1) xs
go :: Ord a => MinHeap a -> [a] -> [a]
go heap [] = Heap.toAscList heap
go heap (x:xs) = x' : go heap' xs where
Just (x', heap') = Heap.view (Heap.insert x heap)
printKSorted :: (Ord a, Show a) => Int -> [a] -> IO ()
printKSorted k xs = mapM_ print (ksort k xs)
If I were feeling extra-special-fancy, I might try to turn go into a foldr or perhaps a mapAccumR, but in this case I think the explicit recursion is relatively readable, too.
I have defined a data type called FS the following way:
type Name= String
data Ext where { Txt::Ext ; Mp3::Ext ; Jar::Ext ; Doc::Ext ; Hs::Ext }
deriving (Eq, Show)
data FS where { A :: (Name,Ext) -> FS;
Dir :: Name-> [FS] -> FS }
deriving (Eq, Show)
(A stands for file and Dir for directory)
And I'm trying to make a function that given a FS (directory) it returns the same FS but ordered alphabetically at all levels, my attempt so far is the following:
orderFS :: FS-> FS
orderFS (Dir x y) = Dir x (map orderFS (sort y));
orderFS (A (x,y)) = A (x,y);
The only piece I'm missing is a function called "sort" that takes a [FS] and returns it ordered alphabetically by the Name field.
I read that there are functions like sort from Data.List that can help, but I have to do this without using anything else than Prelude.
So how should I implement such function? Thanks in advance
I do not believe that there are any sorting functions in Prelude but not in a module like Data.List. Note that Data.List is in the base package that is part of GHC, so in basically any situation where Prelude is available, I would imagine that Data.List would be as well---you shouldn't need to download/include any other packages in order to use it.
That said, if you do want to write your own sorting function, you are probably best off taking an existing simple sorting algorithm and using it. There are very neat/simple ways of writing quicksorts and merge sorts in Haskell, although the obvious implementations sometimes don't have the same exact performance characteristics as you would expect. Merge sort, for example, has roughly the same asymptotics, but partitioning the list into two actually takes some time, since the list is singly-linked and you therefore have to walk through half of it in order to split it. But, it can be a very nice short function that looks a lot like the algorithm, and is probably worth doing as a learning exercise.
Also, I noticed that you are defining your Ext and FS types as GADTs, which I'm not really sure about the motivation for; I would suggest using the non-GADT syntax, which is much simpler for this example:
type Name = String
data Ext = Txt | Mp3 | Jar | Doc | Hs deriving (Eq, Show)
data FS = A Name Ext | Dir Name [FS] deriving (Eq, Show)
In order to sort them by name, it would probably be worth writing a simple accessor function that can get the name of an FS:
name :: FS -> Name
name (A n _) = n
name (Dir n _) = n
Another approach would be to factor out the thing (Name) that is common to both cases:
data FS = NamedFS { name :: Name, fs :: UnnamedFS }
data UnnamedFS = A Ext | Dir [FS]
The first entry here uses record syntax, which, among other things, will automatically make a name :: FS -> Name accessor, as well as an fs :: FS -> UnnamedFS.
For the actual sort, it looks a lot like the algorithmic description of merge sort. To start with, let's write a function to divide a list in two:
split :: [a] -> ([a], [a])
split xs = splitAt (length xs `div` 2) xs
We also need a function to merge two sorted lists:
merge :: Ord a => [a] -> [a] -> [a]
merge [] x = x
merge x [] = x
merge (x:xs) (y:ys) | x < y = x:merge xs (y:ys)
| otherwise = y:merge (x:xs) ys
Actually, this is not what we want, because it always uses the < from the Ord instance; instead, we want something that takes in a comparison function. In this case, we assume that if the comparison function returns true when called with x and y, x is conceptually less than y.
merge :: (a -> a -> Bool) -> [a] -> [a] -> [a]
merge _ [] x = x
merge _ x [] = x
merge le (x:xs) (y:ys) | x `le` y = x:merge le xs (y:ys)
| otherwise = y:merge le (x:xs) ys
Now, we can implement mergesort like usual:
mergeSort :: (a -> a -> Bool) -> [a] -> [a]
mergeSort _ [] = []
mergeSort _ [x] = [x]
mergeSort f l = merge f (mergeSort f left) (mergeSort f right)
where (left, right) = split l
And just call it like:
-- fss is some [FS]
mergeSort (\x y -> name x < name y) fss
If we could use Data.Ord, this could be further reduced to:
mergeSort (comparing name) fss
The function in Data.List that could help you is sortOn
this together with a function getName :: FS -> Name would allow you to sort by comparing the Names.
If you cannot use functions from Data.List you will have to implement a sorting algorithm yourself (of which there are many to choose from). One example is "QuickSort" as implemented in the Learn you a Haskell book:
quicksort :: [FS] -> [FS]
quicksort [] = []
quicksort (x:xs) =
let smallerSorted = quicksort [a | a <- xs, getName a <= getName x]
biggerSorted = quicksort [a | a <- xs, getName a > getName x]
in smallerSorted ++ [x] ++ biggerSorted
Note that I changed the comparisons to compare the getNames instead of the whole nodes.
Another thing: you are using GADT syntax to define your datatypes and I cannot see any reason for you to do so. Here is how I would write them instead using ordinary datatype declarations:
data Ext
= Txt
| Mp3
| Jar
| Doc
| Hs
deriving (Eq, Show)
data FS
= File Name Ext
| Dir Name [FS]
deriving (Eq, Show)
In my opinion, the most natural list-sorting function in Haskell is the bottom-up mergesort (Peter Amidon's answer gives a top-down mergesort). Suppose you're starting with the list
[25,1,22,2,10,8,6,20,13,28,5,3,11]
The first step is to turn the list into a list of lists. The simplest way is with map (:[]), which yields
[[25],[1],[22],[2],[10],[8],[6],[20],[13],[28],[5],[3],[11]]
Next, you merge lists pairwise:
[[1,25],[2,22],[8,10],[6,20],[13,28],[3,5],[11]]
Repeat!
[[1,2,22,25],[6,8,10,20],[3,5,13,28],[11]]
And again!
[[1,2,6,8,10,20,22,25],[3,5,11,13,28]]
And once more:
[[1,2,3,5,6,8,10,11,13,20,22,25,28]]
As we now have just one list, we extract it.
To implement this, I think it's best to use a custom type for the lists of lists in this case, to express the appropriate strictness:
infixr 5 :::
data LL a = ![a] ::: LL a | Nil
Implementation proceeds step by step. While breaking the list up into singletons is the simplest way to start off the process, it's a bit wasteful and will slow down access to the first few elements. So let's go by pairs there too:
breakUp :: (a -> a -> Ordering) -> [a] -> LL a
breakUp _cmp [] = Nil
breakUp _cmp xs#[a] = xs ::: Nil
breakUp cmp (x1 : x2 : xs)
| GT <- cmp x1 x2 = [x2,x1] ::: breakUp cmp xs
| otherwise = [x1,x2] ::: breakUp cmp xs
Now we need to write a merge function:
merge :: (a -> a -> Ordering)
-> [a] -> [a] -> [a]
merge _cmp [] ys = ys
merge _cmp xs [] = xs
merge cmp xss#(x:xs) yss#(y:ys)
| GT <- cmp x y = y : merge cmp xss ys
| otherwise = x : merge cmp xs yss
Next we need to be able to merge a list of lists pairwise:
mergePairwise :: (a -> a -> Ordering)
-> LL a -> LL a
mergePairwise _cmp Nil = Nil
mergePairwise _cmp xs#(_ ::: Nil) = xs
mergePairwise cmp (xs1 ::: xs2 ::: xss)
= merge cmp xs1 xs2 ::: mergePairwise cmp xss
Then we must merge up until done:
mergeAll :: (a -> a -> Ordering)
-> LL a -> [a]
mergeAll _cmp Nil = []
mergeAll _cmp (xs ::: Nil) = xs
mergeAll cmp xss = mergeAll cmp (mergePairwise cmp xss)
Now we're cooking with glass grass bass!
sortBy :: (a -> a -> Ordering) -> [a] -> [a]
sortBy cmp = mergeAll cmp . breakUp cmp
sort :: Ord a => [a] -> [a]
sort = sortBy compare
There's a bit of room to improve the above implementation by recognizing that the lists stored in the lists of lists are never empty. So we might get a small performance improvement using
data NonEmpty a = a :| [a]
data LL a = (:::) {-# UNPACK #-} !(NonEmpty a) (LL a) | Nil
I've done quite a bit of googling to come up with some hints/examples, to no avail so far.
Is there a more generic way to implement a sorting algorithm than one based on lists? I could just go with sortGeneric :: IsList l => l a -> l a but that seems like a bad idea because IsList is nothing more than support for OverloadedLists.
Maybe I should be just asking about sorting a Traversable t => t a?
You certainly can sort an arbitrary Traversable by folding over it to produce a list, vector, or heap, and then using mapAccumL or mapAccumR to put all the elements back. The trouble is that this may be less efficient than sorting the container directly.
import qualified Data.PQueue.Min as Q
import Data.Foldable (toList)
import Data.Traversable (Traversable (..))
import Data.Tuple (swap)
import Control.Monad.Trans.State.Strict
sort xs = mapAccumL' go (Q.fromList . toList $ xs) xs where
go h _ = swap $ Q.deleteFindMin h
mapAccumL' :: Traversable t =>
(a -> b -> (a, c)) -> a -> t b -> t c
mapAccumL' f s t = flip evalState s $
traverse (\q -> state $ swap . flip f q) t
Note that the uses of toList and fromList are purely for convenience, and the lists involved are pretty likely never to actually be allocated.
How about
sortGeneric :: (Ord a, Traversable t) => t a -> t a
I am just curious if there are any (first order polymorphic only) optimisations with folds.
For maps, there's deforestation: map g (map f ls) => map (g . f) ls, and rev (map f ls) => rev_map f ls (faster in Ocaml).
But fold is so powerful, it seems to defy any optimisation.
The obvious ones:
fold_left f acc (List.map g li) => fold_left (fun acc x -> f acc (g x)) acc li
fold_right f li acc => fold_left f acc li (* if (f,acc) is a monoid *)
You may be interested in the classical paper on the topic, "Functional Programming with Bananas, Lenses, Envelopes and Barbed Wire". Beware however that it is technical and has impenetrable notation.
http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.41.125
Edit: my first version of the first rule was wrong, edited thanks to vincent-hugot.
You can use deforestation on folds. In fact, map/map fusion is a special case of that.
The trick is to replace list construction by a special build function:
build :: (forall b. (a -> b -> b) -> b -> b) -> [a]
build g = g (:) []
Now, using the standard definition of foldr
foldr :: (a -> b -> b) -> b -> [a] -> b
foldr c n [] = n
foldr c n (x:xs) = c x (foldr c n xs)
We have the following equivalence:
foldr c n (build g) == g c n
(Actually this is only true under certain, but common, circumstances. For details see "Correctness of short-cut fusion").
If you write your list producing functions (including map) using build and your consumers using foldr, then the above equality can remove most intermediate lists. Haskell's list comprehensions are translated into combinations of build and foldr.
The downside of this approach is that it cannot handle left folds. Stream Fusion handles this just fine, though. It expresses list producers and transformers as streams (coinductive datatypes, kind of like iterators). The above paper is very readable, so I recommend taking a look.
The "bananas" paper mentioned by gasche goes into more details about kinds of folds and their equivalences.
Finally, there is Bird and Moor's "Algebra of Programming", which mentions transformations such as combining two folds into one.
If you're interested going a bit deeper into theory, I suggest you to read something about catamorphisms, anamorphisms and hylomorphisms. While the category theory surrounding it may seem to be a bit scary, the concept isn't that difficult.
Catamorphisms are functions that consume recursive data structures and produce some kind of a value. Anamorphisms are functions that given some value (a kind of a seed) produce recursive data structures. In particular, foldr and build mentioned in the other anwers are functions to build catamorphisms and anamorphisms on lists. But this concept can be applied to basically any recursive data structure, such as different kinds of trees etc.
Now if you build a recursive data structure with an anamorphism and then consume it with a catamorphism, you get what is called a hylomorphism. In such a case, you actually don't need the intermediate structure. You can skip creating it and destroying it. This is often called deforestation.
Concerning map: This function is interesting that it's both a catamorphism and an anamorphism:
map consumes a list and produces something; but also
map produces a list, consuming something.
So you can view the composition of two maps map f . map g as a composition of an anamorphism (map g) with a catamorphism (map f), forming a hylomorphism. So you know can optimize (deforest) by not creating the intermediate list.
To be specific: You could write map in two ways, one using foldr and the other using build:
mapAna :: (a -> b) -> [a] -> [b]
mapAna f xs = build (mapAna' f xs)
mapAna' :: (a -> b) -> [a] -> (b -> c -> c) -> c -> c
mapAna' f [] cons nil = nil
mapAna' f (x : xs) cons nil = (f x) `cons` (mapAna' f xs cons nil)
mapCata :: (a -> b) -> [a] -> [b]
mapCata f xs = foldr (\x ys -> f x : ys) [] xs
and the composition map f (map g zs) as mapCata f (mapAna g zs), which after some simplifications and applying foldr c n (build g) == g c n results in map (f . g).