open System
let rec quick (cast: int list) mmm =
match mmm with
| [] -> []
| first::rest ->
let small = (rest |> List.filter (fun x -> x < first))
let large = (rest |> List.filter (fun x -> x >= first))
quick small |> ignore
quick large |> ignore
//[small # [first] # large]
List.concat [small; [first]; large]
[<EntryPoint>]
let main argv =
printfn "%A" (quick [3;5;6;7;8;7;5;4;3;4;5;6]);;
0
Trying to implement a simple quicksort function in F#.
Relatively new to the language, but by all account from what I've read and my understanding of the syntax this should present an integer list but is instead presenting the ambiguous "unit list".
Why does this give a unit list and not an int list?
It errors out at "%A" saying the types do not match.
As given in the OP, quick is a function that takes two parameters: cast and mmm. The type of the function is int list -> int list -> int list.
The function call quick [3;5;6;7;8;7;5;4;3;4;5;6], however, only supplies one argument. Since F# functions are curried, the return value is a new function:
> quick [3;5;6;7;8;7;5;4;3;4;5;6];;
val it : (int list -> int list) = <fun:it#3-4>
This function (in my F# Interactive window called it#3-4) has the type int list -> int list - that is: It's a function that 'still waits' for an int list argument before it runs.
When you print it with the %A format specifier, it prints <fun:it#4-5> to the console. The return value of printfn is () (unit):
> printfn "%A" (quick [3;5;6;7;8;7;5;4;3;4;5;6]);;
<fun:it#4-5>
val it : unit = ()
You probably only want the function to take a single list parameter. Additionally, the steps you ignore are having no effect, so you might consider another way to recursively call quick.
From an unordered list of int, I want to have the smallest difference between two elements. I have a code that is working but way to slow. Can anyone sugest some change to improve the performance? Please explain why you did the change and what will be the performance gain.
let allInt = [ 5; 8; 9 ]
let sortedList = allInt |> List.sort;
let differenceList = [ for a in 0 .. N-2 do yield sortedList.Item a - sortedList.Item a + 1 ]
printfn "%i" (List.min differenceList) // print 1 (because 9-8 smallest difference)
I think I'm doing to much list creation or iteration but I don't know how to write it differently in F#...yet.
Edit: I'm testing this code on list with 100 000 items or more.
Edit 2: I believe that if I can calculte the difference and have the min in one go it should improve the perf a lot, but I don't know how to do that, anay idea?
Thanks in advance
The List.Item performs in O(n) time and is probably the main performance bottle neck in your code. The evaluation of differenceList iterates the elements of sortedList by index, which means the performance is around O((N-2)(2(N-2))), which simplifies to O(N^2), where N is the number of elements in sortedList. For long lists, this will eventually perform badly.
What I would do is to eliminate calls to Item and instead use the List.pairwise operation
let data =
[ let rnd = System.Random()
for i in 1..100000 do yield rnd.Next() ]
#time
let result =
data
|> List.sort
|> List.pairwise // convert list from [a;b;c;...] to [(a,b); (b,c); ...]
|> List.map (fun (a,b) -> a - b |> abs) // Calculates the absolute difference
|> List.min
#time
The #time directives lets me measure execution time in F# Interactive and the output I get when running this code is:
--> Timing now on
Real: 00:00:00.029, CPU: 00:00:00.031, GC gen0: 1, gen1: 1, gen2: 0
val result : int = 0
--> Timing now off
F#'s built-in list type is implemented as a linked list, which means accessing elements by index has to enumerate the list all the way to the index each time. In your case you have two index accesses repeated N-2 times, getting slower and slower with each iteration, as the index grows and each access needs to go through longer part of the list.
First way out of this would be using an array instead of a list, which is a trivial change, but grants you faster index access.
(*
[| and |] let you define an array literal,
alternatively use List.toArray allInt
*)
let allInt = [| 5; 8; 9 |]
let sortedArray = allInt |> Array.sort;
let differenceList = [ for a in 0 .. N-2 do yield sortedArray.[a] - sortedArray.[a + 1] ]
Another approach might be pairing up the neighbours in the list, subtracting them and then finding a min.
let differenceList =
sortedList
|> List.pairwise
|> List.map (fun (x,y) -> x - y)
List.pairwise takes a list of elements and returns a list of the neighbouring pairs. E.g. in your example List.pairwise [ 5; 8; 9 ] = [ (5, 8); (8, 9) ], so that you can easily work with the pairs in the next step, the subtraction mapping.
This way is better, but these functions from List module take a list as input and produce a new list as the output, having to pass through the list 3 times (1 for pairwise, 1 for map, 1 for min at the end). To solve this, you can use functions from the Seq module, which work with .NETs IEnumerable<'a> interface allowing lazy evaluation resulting usually in fewer passes.
Fortunately in this case Seq defines alternatives for all the functions we use here, so the next step is trivial:
let differenceSeq =
sortedList
|> Seq.pairwise
|> Seq.map (fun (x,y) -> x - y)
let minDiff = Seq.min differenceSeq
This should need only one enumeration of the list (excluding the sorting phase of course).
But I cannot guarantee you which approach will be fastest. My bet would be on simply using an array instead of the list, but to find out, you will have to try it out and measure for yourself, on your data and your hardware. BehchmarkDotNet library can help you with that.
The rest of your question is adequately covered by the other answers, so I won't duplicate them. But nobody has yet addressed the question you asked in your Edit 2. To answer that question, if you're doing a calculation and then want the minimum result of that calculation, you want List.minBy. One clue that you want List.minBy is when you find yourself doing a map followed by a min operation (as both the other answers are doing): that's a classic sign that you want minBy, which does that in one operation instead of two.
There's one gotcha to watch out for when using List.minBy: It returns the original value, not the result of the calculation. I.e., if you do ints |> List.pairwise |> List.minBy (fun (a,b) -> abs (a - b)), then what List.minBy is going to return is a pair of items, not the difference. It's written that way because if it gives you the original value but you really wanted the result, you can always recalculate the result; but if it gave you the result and you really wanted the original value, you might not be able to get it. (Was that difference of 1 the difference between 8 and 9, or between 4 and 5?)
So in your case, you could do:
let allInt = [5; 8; 9]
let minPair =
allInt
|> List.pairwise
|> List.minBy (fun (x,y) -> abs (x - y))
let a, b = minPair
let minDifference = abs (a - b)
printfn "The difference between %d and %d was %d" a b minDifference
The List.minBy operation also exists on sequences, so if your list is large enough that you want to avoid creating an intermediate list of pairs, then use Seq.pairwise and Seq.minBy instead:
let allInt = [5; 8; 9]
let minPair =
allInt
|> Seq.pairwise
|> Seq.minBy (fun (x,y) -> abs (x - y))
let a, b = minPair
let minDifference = abs (a - b)
printfn "The difference between %d and %d was %d" a b minDifference
EDIT: Yes, I see that you've got a list of 100,000 items. So you definitely want the Seq version of this. The F# seq type is just IEnumerable, so if you're used to C#, think of the Seq functions as LINQ expressions and you'll have the right idea.
P.S. One thing to note here: see how I'm doing let a, b = minPair? That's called destructuring assignment, and it's really useful. I could also have done this:
let a, b =
allInt
|> Seq.pairwise
|> Seq.minBy (fun (x,y) -> abs (x - y))
and it would have given me the same result. Seq.minBy returns a tuple of two integers, and the let a, b = (tuple of two integers) expression takes that tuple, matches it against the pattern a, b, and thus assigns a to have the value of that tuple's first item, and b to have the value of that tuple's second item. Notice how I used the phrase "matches it against the pattern": this is the exact same thing as when you use a match expression. Explaining match expressions would make this answer too long, so I'll just point you to an excellent reference on them if you haven't already read it:
https://fsharpforfunandprofit.com/posts/match-expression/
Here is my solution:
let minPair xs =
let foo (x, y) = abs (x - y)
xs
|> List.allPairs xs
|> List.filter (fun (x, y) -> x <> y)
|> List.minBy foo
|> foo
Maybe it's already implemented in F# ?
basically i would like to define a class of generic filter functions with infix operators, so something which would look like
type Filter<'T> = ('T -> bool) with
static member (|*) (f:Filter<'T>) (g:Filter<'T>) = (fun x -> (f x) ||
(g x)) // OR operator
but this is not the right syntax it seems
Stopped due to error System.Exception: Operation could not be
completed due to earlier error Type abbreviations cannot have
augmentations at 2,5 Type abbreviations cannot have members at 3,19
thanks
What you are defining there is a type abbreviation, which, as the errors will indicate, can neither have augmentations nor members. You could fix that by using a single case discriminated union:
type Filter<'T> = Filter of ('T -> bool) with
static member (|* ) (Filter f, Filter g) =
Filter(fun x -> f x || g x) // OR operator
Of course, you now need to unwrap the predicate function prior to the boolean operation, and wrap the composed function again afterwards. A simple test...
let odd x = x % 2 <> 0
let big x = x > 10
let (Filter f) = Filter odd |* Filter big in [8..12] |> List.filter f
// val it : int list = [9; 11; 12]
If you define a While method of the builder-object, you can use while-loops in your computation expressions. The signature of the While method is:
member b.While (predicate:unit->bool, body:M<'a>) : M<'a>
For comparison, the signature of the For method is:
member b.For (items:seq<'a>, body:unit->M<'a>) : M<'a>
You should notice that, in the While-method, the body is a simple type, and not a function as in the For method.
You can embed some other statements, like let and function-calls inside your computation-expressions, but those can impossibly execute in a while-loop more than once.
builder {
while foo() do
printfn "step"
yield bar()
}
Why is the while-loop not executed more than once, but merely repeated? Why the significant difference from for-loops? Better yet, is there some intended strategy for using while-loops in computation-expressions?
If you look at how computation expressions are evaluated, you'll see that
while foo() do
printfn "step"
yield bar()
is translated to something like
builder.While(fun () -> foo(),
builder.Delay(fun () ->
printfn "step"
builder.Yield(bar()))))
This translation allows the body of the while loop to be evaluated multiple times. While your type signatures are accurate for some computation expressions (such as seq or async), note that the insertion of the call to Delay may result in a different signature. For instance, you could define a list builder like this:
type ListBuilder() =
member x.Delay f = f
member x.While(f, l) = if f() then l() # (x.While(f, l)) else []
member x.Yield(i) = [i]
member x.Combine(l1,l2) = l1 # l2()
member x.Zero() = []
member x.Run f = f()
let list = ListBuilder()
Now you can evaluate an expression like:
list {
let x = ref 0
while !x < 10 do
yield !x
x := !x + 1
}
to get the equivalent of [0 .. 9].
Here, our While method has the signature (unit -> bool) * (unit -> 'a list) -> 'a list, rather than (unit -> bool) * 'a list -> 'a list. In general, when the Delay operation has type (unit -> M<'a>) -> D<M<'a>>, the While method's signature will be (unit -> bool) * D<M<'a>> -> M<'a>.
I've recently written a piece of code to read some data from a file, store it in a tuple and sort all the collected data by the first element of the tuple. After some tests I've noticed that using Seq.sortBy (and Array.sortBy) is extremely slower than using IEnumerable.OrderBy.
Below are two snippets of code which should show the behaviour I'm talking about:
(filename
|> File.ReadAllLines
|> Array.Parallel.map(fun ln -> let arr = ln.Split([|' '|], StringSplitOptions.RemoveEmptyEntries)
|> Array.map(double)
|> Array.sort in arr.[0], arr.[1])
).OrderBy(new Func(fun (a,b) -> a))
and
filename
|> File.ReadAllLines
|> Array.Parallel.map(fun ln -> let arr = ln.Split([|' '|], StringSplitOptions.RemoveEmptyEntries) |> Array.map(double) |> Array.sort in arr.[0], arr.[1])
|> Seq.sortBy(fun (a,_) -> a)
On a file containing 100000 lines made of two doubles, on my computer the latter version takes over twice as long as the first one (no improvements are obtained if using Array.sortBy).
Ideas?
the f# implementation uses a structural comparison of the resulting key.
let sortBy keyf seq =
let comparer = ComparisonIdentity.Structural
mkDelayedSeq (fun () ->
(seq
|> to_list
|> List.sortWith (fun x y -> comparer.Compare(keyf x,keyf y))
|> to_array) :> seq<_>)
(also sort)
let sort seq =
mkDelayedSeq (fun () ->
(seq
|> to_list
|> List.sortWith Operators.compare
|> to_array) :> seq<_>)
both Operators.compare and the ComparisonIdentity.Structural.Compare become (eventually)
let inline GenericComparisonFast<'T> (x:'T) (y:'T) : int =
GenericComparisonIntrinsic x y
// lots of other types elided
when 'T : float = if (# "clt" x y : bool #)
then (-1)
else (# "cgt" x y : int #)
but the route to this for the Operator is entirely inline, thus the JIT compiler will end up inserting a direct double comparison instruction with no additional method invocation overhead except for the (required in both cases anyway) delegate invocation.
The sortBy uses a comparer so will go through an additional virtual method call but is basically about the same.
In comparison the OrderBy function also must go through virtual method calls for the equality (Using EqualityComparer<T>.Default) but the significant difference is that it sorts in place and uses the buffer created for this as the result. In comparison if you take a look at the sortBy you will see that it sorts the list (not in place, it uses the StableSortImplementation which appears to be merge sort) and then creates a copy of it as a new array. This additional copy (given the size of your input data) is likely the principle cause of the slow down though the differing sort implementations may also have an effect.
That said this is all guessing. If this area is a concern for you in performance terms then you should simply profile to find out what is taking the time.
If you wish to see what effect the sorting/copying change would have try this alternate:
// these are taken from the f# source so as to be consistent
// beware doing this, the compiler may know about such methods
open System.Collections.Generic
let mkSeq f =
{ new IEnumerable<'b> with
member x.GetEnumerator() = f()
interface System.Collections.IEnumerable with
member x.GetEnumerator() = (f() :> System.Collections.IEnumerator) }
let mkDelayedSeq (f: unit -> IEnumerable<'T>) =
mkSeq (fun () -> f().GetEnumerator())
// the function
let sortByFaster keyf seq =
let comparer = ComparisonIdentity.Structural
mkDelayedSeq (fun () ->
let buffer = Seq.to_array seq
Array.sortInPlaceBy (fun x y -> comparer.Compare(keyf x,keyf y)) buffer
buffer :> seq<_>)
I get some reasonable percentage speedups within the repl with very large (> million) input sequences but nothing like an order of magnitude. Your mileage, as always, may vary.
A difference of x2 is not much when sorts are O(n.log(n)).
Small differences in data structures (e.g. optimising for input being ICollection<T>) could make this scale of difference.
And F# is currently Beta (not so much focus on optimisation vs. getting the language and libraries right), plus the generality of F# functions (supporting partial application etc.) could lead to a slight slow down in calling speed: more than enough to account for the different.