Given that in F#
Tuples have an automatically defined equality operation
and therefore
let myQueryable = [| (1, 1); (2,2)|].AsQueryable()
let check =
query {
for x in myQueryable do
} |> Seq.where(fun x -> x = (2,2))
is {(2,2)}, then why
let check =
query {
for x in myQueryable do
where(x = (2, 2))
}
is Empty?
It seems a compatibility issue between C# System.Tuple and F# reference tuples.
It is interesting that the compatibility issue seems to be solved between C# 7 and F# 4.1 with the new ValueTuple equivalent to F# struct tuples.
let myQueryableNew = [| ValueTuple.Create(1, 1); ValueTuple.Create(2,2) |].AsQueryable()
let resultFSharp1 =
query {
for x in myQueryableNew do
} |> Seq.where(fun x -> x = ValueTuple.Create(2,2))
let resultFSharp2 =
query {
for x in myQueryableNew do
where (x.Equals(ValueTuple.Create(2, 2)))
}
Btw, notice also that the new type has a more coherent default (in case of this related issue)
let testDefault =
query {
for x in myQueriable1 do
where (x.Equals(ValueTuple.Create(1, 2)))
select x
headOrDefault
}
You probably have System.Linq open.
The following returns (2,2) for me in both cases:
//open System
//open System.Linq
let myQueryable = [| (1, 1); (2,2) |] //.AsQueryable()
let check =
query {
for x in myQueryable do
select x
} |> Seq.where(fun x -> x = (2,2))
check //val it : seq<int * int> = seq [(2, 2)]
let check' =
query {
for x in myQueryable do
where (x = (2, 2))
select x
}
check' //val it : seq<int * int> = seq [(2, 2)]
So within an IQueryable this structural equality doesn't hold. I don't know if it's because of the IQueryable type, i.e. it becomes reference equality, or where gets redefined, or a possible bug.
Related
Cannot assign a position to View.Pin from a list of tuples (There are 300 tuples in the real program), please comment how to fix it.
Error : The type 'a list' is not compatible with the type 'View Element'
let tokyo = Position(35.652832, 139.839478)
// create sample tuples
let t1 = ("35", 48.856, 2.3522)
let t2 = ("32", 51.5074, -0.1278)
let t3 = ("25", 50.9513, 1.8587)
// create sample list
let lst = [ t1; t2; t3 ]
let map =
View.Map
(hasZoomEnabled = true, hasScrollEnabled = true,
pins = [ lst |> List.map (fun (_, a, b) -> View.Pin(Position(a, b), label = "Dummy", pinType = PinType.Place)) ],
requestedRegion = MapSpan.FromCenterAndRadius(tokyo, Distance.FromKilometers(300.)))
I found a solution from "https://msdn.microsoft.com/en-us/visualfsharpdocs/conceptual/list.map%5B%27t%2C%27u%5D-function-%5Bfsharp%5D"
To my surprise, F# is an awesome functional language to handle large data set in only a few lines of codes.
let view (model : Model) dispatch =
// A list of 300 tuples
let jpnDatas = jsonData.jpnData
let pins =
jpnDatas
|> List.map (fun (_, a, _, _, b, c) -> View.Pin(Position(b, c), label = a, pinType = PinType.Place))
let tokyo = Position(35.652832, 139.839478)
let map =
View.Map
(hasZoomEnabled = true, hasScrollEnabled = true,
pins = [ for pin in pins do yield pin ],
requestedRegion = MapSpan.FromCenterAndRadius(tokyo, Distance.FromKilometers(300.)))
What would be a more performant way to process this 2DArray without 3rd party?
#time
let ar = array2D[[5.0; 6.0; 7.0; 8.0]; [1.0; 2.0; 3.0; 4.0]]
[0..5000000]
let a2 = ar |> Array2D.mapi(fun rowi coli value -> (value + 1.6) * double(coli + 6) * double(rowi + 7))
If you run the above code, it takes about 0ms, so I it really depends on the context in which you are calling it. If you just run it in a loop 1M times, then it takes about 600ms on my machine:
for i in 0 .. 1000000 do
let a2 = ar |> Array2D.mapi(fun rowi coli value ->
(value + 1.6) * double ((coli + 6) * (rowi + 7)))
()
Here, most of the time is spent allocating the result array - for each iteration, we need to allocate a new 2D array to store the result. This gives you nice functional properties (the results can be shared because they're not mutated) but it is why it takes longer.
You can use some mutation and avoid this. This depends on the context, and so that's why you probably won't get a useful answer here.
For example, in this artificial 1M loop example, I could just allocate one array to store the results and then write there repeatedly:
let res = ar |> Array2D.map id
for i in 0 .. 1000000 do
for x in 0 .. ar.GetLength(0) - 1 do
for y in 0 .. ar.GetLength(1) - 1 do
res.[x, y] <- (ar.[x, y] + 1.6) * double ((x + 6) * (y + 7))
This takes about 100ms, so that gives you an idea about the cost of the allocation. But then, you should not do this change if it can break your program because now you'd be using mutable arrays...
I did some measurements of this problem which I thought could be interesting.
I created 8 different test cases and ran over 3 differently sized matrixes; 1000x1000, 100x100 and 10x10.
In addition I ran the tests in x64 as well as x86.
In the end I ended up with 48 test results presented in two graphs. The y-axis is the execution time in milliseconds.
Creating Zero Matrix - the cost of creating a zero matrix
Copying Matrix - the cost of copying a matrix with Array2D.copy
Mapping Matrix with id - the cost of copying a matrix with Array2D.copy map id
Original Algorithm - the cost of the algorithm posted by OP
Tomas Petricek Algorithm - the cost of the algorithm by Tomas
Modified Tomas Petricek Algorithm - the cost of the modified algorithm to use Array.zeroCreate
Reverse Algorithm - the cost of iterating over the matrix in reverse
Flipped x,y Algorithm - the cost of the modified algorithm but flipping x,y iteration order
Some observations
Tomas wanted to demonstrate the cost of the copy compared to the computation so in his example the copy was not part of the inner loop. I wanted to include his code so I moved the copy into the inner loop to be able to compare with the others. The modified Tomas algorithm is the same code but uses Array2D.zeroCreate to create a fresh matrix. When writing this I realize it would have been better to call both of them modified.
On .NET 4.5.2 x64 is doing significantly better in general
There are performance benefits of using Array2D.zeroCreate and populate the matrix over using Array2D.copy
For large matrixes the x,y iteration order is extremely important. For small matrixes it doesn't matter. This is because how CPU caches works
Iterating reverse order over a the array seems to give a small benefit. The reason is that it's cheaper to check y >= 0 than y < xl.
The reverse algorithm has to use tail-recursion as F# for y = (yl - 1) downto 0 uses y > variable_that_is_always_minus_1 to check for loop end. With tail-recursion we can force y >= 0
For smaller sized Matrixes the cost of creating them and the cost of the GC is increasing.
The code used to generate the measurements.
open System
open System.IO
open System.Diagnostics
let clock =
let sw = Stopwatch ()
sw.Start ()
sw
let collectionCount () =
GC.CollectionCount 0 + GC.CollectionCount 1 + GC.CollectionCount 2
let timeIt (n : string) (outer : int) (a : unit -> 'T) : 'T*int64 =
printfn "Timing '%s'..." n
let v = a ()
let t = clock.ElapsedMilliseconds
for i in 1..outer do
a () |> ignore
let e = clock.ElapsedMilliseconds - t
printfn " took %d ms" e
v, e
[<EntryPoint>]
let main argv =
let random = Random 19740531
let total = 100000000
let outers = [|100;10000;1000000|]
use output = new StreamWriter ".\output.tsv"
"Dimensions\tName\tSum\tCollectionCounts\tMilliseconds" |> output.WriteLine
for outer in outers do
let inner = total / outer
let dim = inner |> float |> sqrt |> int32
let ar = Array2D.init dim dim (fun _ _ -> random.NextDouble ())
printfn "New test run, matrix dimensions are %dx%d" dim dim
let run = sprintf "%d_%d" dim dim
let perf_zero () : float[,] =
let xl = ar.GetLength(0)
let yl = ar.GetLength(1)
let res = Array2D.zeroCreate xl yl
res
let perf_copy () : float[,] =
Array2D.copy ar
let perf_id () : float[,] =
ar |> Array2D.map id
let perf_op () : float[,] =
ar |> Array2D.mapi(fun rowi coli value -> (value + 1.6) * double(coli + 6) * double(rowi + 7))
let perf_tp () : float[,] =
let res = ar |> Array2D.map id
for x in 0 .. ar.GetLength(0) - 1 do
for y in 0 .. ar.GetLength(1) - 1 do
res.[x, y] <- (ar.[x, y] + 1.6) * double ((x + 6) * (y + 7))
res
let perf_tpm () : float[,] =
let xl = ar.GetLength(0)
let yl = ar.GetLength(1)
let res = Array2D.zeroCreate xl yl
for x in 0 .. xl - 1 do
for y in 0 .. yl - 1 do
res.[x, y] <- (ar.[x, y] + 1.6) * double ((x + 6) * (y + 7))
res
let perf_tpmf () : float[,] =
let xl = ar.GetLength(0)
let yl = ar.GetLength(1)
let res = Array2D.zeroCreate xl yl
for y in 0 .. yl - 1 do
for x in 0 .. xl - 1 do
res.[x, y] <- (ar.[x, y] + 1.6) * double ((x + 6) * (y + 7))
res
let perf_tr () : float[,] =
let xl = ar.GetLength(0)
let yl = ar.GetLength(1)
let res = Array2D.zeroCreate xl yl
let rec loopy x y =
if y >= 0 then
res.[x, y] <- (ar.[x, y] + 1.6) * double ((x + 6) * (y + 7))
loopy x (y - 1)
else
()
and loopx x =
if x >= 0 then
loopy x (yl - 1)
loopx (x - 1)
else
()
loopx (xl - 1)
res
let testCases =
[|
"Creating Zero Matrix" , perf_zero
"Copying Matrix" , perf_copy
"Mapping Matrix with id" , perf_id
"Original Algorithm" , perf_op
"Tomas Petricek Algorithm" , perf_tp
"Modified Tomas Petricek Algorithm" , perf_tpm
"Reverse Algoritm" , perf_tr
"Flipped x,y Algoritm" , perf_tpmf
|]
for name, a in testCases do
let pcc = collectionCount ()
let vs, t = timeIt name outer a
let sum = ref 0.
vs |> Array2D.iter (fun v -> sum := !sum + v)
let dcc = collectionCount () - pcc
sprintf "%s\t%s\t%f\t%d\t%d" run name !sum dcc t |> output.WriteLine
0
As OP specified that his problem dealt with smaller Matrixes like 9x4 I did another set of metrics. However since I thought my previous answers held some interesting points on metrics with larger sizes I decided to create a new answer
I did some measurements of this problem which I thought could be interesting.
I created 9 different test cases and ran over it over a 10x5 matrix. All tests run in Release(obviously)/x64.
The first graph shows the execution time in milliseconds:
The second graph shows the number of GC collections during test run:
Creating Zero Matrix - the cost of creating a zero matrix
Copying Matrix - the cost of copying a matrix with Array2D.copy
Mapping Matrix with id - the cost of copying a matrix with Array2D.copy map id
Original Algorithm - the cost of the algorithm posted by OP
Tomas P Algorithm with Zero Init - the cost of the algorithm by Tomas with Array2D.zeroInit
Creating Zero Fixed Size Matrix - the cost of creating a zero fixed size matrix
Copying Fixed Size Matrix - the cost of creating a zero fixed size matrix
Fixed Size Algorithm - the cost of OP:s algorithm adapted to fixed size matrix
Fixed Size Updater - the cost of OP:s algorithm using an updater function
The Fixed Size Matrix is a struct that uses unsafe code to avoid GC allocations. It's written in C# but might be portable to F#. It should not be seen as production worthy code, more like an inspiration for something of your own creation.
Some observations:
Copying a Fixed Size matrix is quick
The Fixed Size Algorithm doesn't perform as good as one hoped. Potentially because JIT:er have to perform some extra lifting because of unsafe code
The Fixed Size Updater (which is similar to Array2D.iteri) has the best performance
As expected Fixed Size Matrixes don't create any GC pressure as it don't rely on GC allocation.
It's hard to judge for me if the Fixed Size Matrix is a viable path for OP but it's an option that might be worth considering.
F# code:
open System
open System.IO
open System.Diagnostics
open Unsafe
let clock =
let sw = Stopwatch ()
sw.Start ()
sw
let collectionCount () =
GC.CollectionCount 0 + GC.CollectionCount 1 + GC.CollectionCount 2
let createTimer (n : string) (a : unit -> 'T) (r : 'T -> 'TResult) : string*(int -> 'TResult*int64*int) =
n, fun outer ->
printfn "Timing '%s'..." n
let v = a () |> r
GC.Collect ()
GC.WaitForFullGCComplete () |> ignore
let pcc = collectionCount ()
let t = clock.ElapsedMilliseconds
for i in 1..outer do
a () |> ignore
let e = clock.ElapsedMilliseconds - t
let dcc = collectionCount () - pcc
printfn " took %d ms, collected %d times, result is %A" e dcc v
v, e, dcc
[<EntryPoint>]
let main argv =
let random = Random 19740531
let total = 300000000
use output = new StreamWriter ".\output.tsv"
"Name\tSum\tCollectionCounts\tMilliseconds" |> output.WriteLine
let cols = 5
let rows = 10
let inner = cols*rows
let outer = total / inner
let ar = Array2D.init rows cols (fun _ _ -> random.NextDouble ())
let mtx5x10 =
let mutable m = Matrix5x10 ()
ar |> Array2D.iteri (fun row col v -> (m.[col, row] <- v))
m
printfn "New test run, matrix dimensions are %dx%d" cols rows
let perf_zero () =
let xl = ar.GetLength(0)
let yl = ar.GetLength(1)
let res = Array2D.zeroCreate xl yl
res
let perf_copy () =
Array2D.copy ar
let perf_id () =
ar |> Array2D.map id
let perf_op () =
ar |> Array2D.mapi(fun rowi coli value -> (value + 1.6) * double(rowi + 6) * double(coli + 7))
let perf_tpm () =
let xl = ar.GetLength(0)
let yl = ar.GetLength(1)
let res = Array2D.zeroCreate xl yl
for x in 0 .. xl - 1 do
for y in 0 .. yl - 1 do
res.[x, y] <- (ar.[x, y] + 1.6) * double ((x + 6) * (y + 7))
res
let perf_fzero () =
let m = Matrix5x10()
m
let perf_fcopy () =
let m = mtx5x10
m
let perf_fs () =
let mutable m = Matrix5x10 ()
for row = 0 to Matrix5x10.Rows - 1 do
for col = 0 to Matrix5x10.Columns - 1 do
m.[col, row] <- (mtx5x10.[col, row] + 1.6) * double ((row + 6) * (col + 7))
m
let perf_fsui = Func<int, int, double, double> (fun col row v -> (v + 1.6) * double ((row + 6) * (col + 7)))
let perf_fsu () =
let mutable m = mtx5x10
m.Update perf_fsui
m
let sumArray vs =
let sum = ref 0.
vs |> Array2D.iter (fun v -> sum := !sum + v)
!sum
let sumMatrix (mtx : Matrix5x10) =
let sum = ref 0.
mtx.Update (fun _ _ v -> sum := !sum + v; v)
!sum
let testCases =
[|
createTimer "Creating Zero Matrix" perf_zero sumArray
createTimer "Copying Matrix" perf_copy sumArray
createTimer "Mapping Matrix with id" perf_id sumArray
createTimer "Original Algorithm" perf_op sumArray
createTimer "Tomas P Algorithm with Zero Init" perf_tpm sumArray
createTimer "Creating Zero Fixed Size Matrix" perf_fzero sumMatrix
createTimer "Copying Fixed Size Matrix" perf_fcopy sumMatrix
createTimer "Fixed Size Algorithm" perf_fs sumMatrix
createTimer "Fixed Size Updater" perf_fsu sumMatrix
|]
for name, a in testCases do
let sum, t, dcc = a outer
sprintf "%s\t%f\t%d\t%d" name sum dcc t |> output.WriteLine
0
C# code (for those that care I generated this with T4):
namespace Unsafe
{
using System;
using System.Diagnostics;
using System.Runtime.CompilerServices;
using System.Runtime.InteropServices;
[StructLayout(LayoutKind.Sequential)]
public struct Matrix5x10
{
double m_c0_r0;
double m_c1_r0;
double m_c2_r0;
double m_c3_r0;
double m_c4_r0;
double m_c0_r1;
double m_c1_r1;
double m_c2_r1;
double m_c3_r1;
double m_c4_r1;
double m_c0_r2;
double m_c1_r2;
double m_c2_r2;
double m_c3_r2;
double m_c4_r2;
double m_c0_r3;
double m_c1_r3;
double m_c2_r3;
double m_c3_r3;
double m_c4_r3;
double m_c0_r4;
double m_c1_r4;
double m_c2_r4;
double m_c3_r4;
double m_c4_r4;
double m_c0_r5;
double m_c1_r5;
double m_c2_r5;
double m_c3_r5;
double m_c4_r5;
double m_c0_r6;
double m_c1_r6;
double m_c2_r6;
double m_c3_r6;
double m_c4_r6;
double m_c0_r7;
double m_c1_r7;
double m_c2_r7;
double m_c3_r7;
double m_c4_r7;
double m_c0_r8;
double m_c1_r8;
double m_c2_r8;
double m_c3_r8;
double m_c4_r8;
double m_c0_r9;
double m_c1_r9;
double m_c2_r9;
double m_c3_r9;
double m_c4_r9;
public const int Columns = 5;
public const int Rows = 10;
unsafe public double this[int x, int y]
{
[MethodImpl (MethodImplOptions.AggressiveInlining)]
get
{
var i = 5 * y + x;
if (i < 0 || i >= 50)
{
throw new IndexOutOfRangeException ("0 <= x <= 5 && 0 <= y <= 10");
}
fixed (double * ms = &m_c0_r0)
{
return ms[i];
}
}
[MethodImpl (MethodImplOptions.AggressiveInlining)]
set
{
var i = 5 * y + x;
if (i < 0 || i >= 50)
{
throw new IndexOutOfRangeException ("0 <= x <= 5 && 0 <= y <= 10");
}
fixed (double * ms = &m_c0_r0)
{
ms[i] = value;
}
}
}
public void Update (Func<int, int, double, double> updater)
{
if (updater == null)
{
return;
}
m_c0_r0 = updater (0, 0, m_c0_r0);
m_c1_r0 = updater (1, 0, m_c1_r0);
m_c2_r0 = updater (2, 0, m_c2_r0);
m_c3_r0 = updater (3, 0, m_c3_r0);
m_c4_r0 = updater (4, 0, m_c4_r0);
m_c0_r1 = updater (0, 1, m_c0_r1);
m_c1_r1 = updater (1, 1, m_c1_r1);
m_c2_r1 = updater (2, 1, m_c2_r1);
m_c3_r1 = updater (3, 1, m_c3_r1);
m_c4_r1 = updater (4, 1, m_c4_r1);
m_c0_r2 = updater (0, 2, m_c0_r2);
m_c1_r2 = updater (1, 2, m_c1_r2);
m_c2_r2 = updater (2, 2, m_c2_r2);
m_c3_r2 = updater (3, 2, m_c3_r2);
m_c4_r2 = updater (4, 2, m_c4_r2);
m_c0_r3 = updater (0, 3, m_c0_r3);
m_c1_r3 = updater (1, 3, m_c1_r3);
m_c2_r3 = updater (2, 3, m_c2_r3);
m_c3_r3 = updater (3, 3, m_c3_r3);
m_c4_r3 = updater (4, 3, m_c4_r3);
m_c0_r4 = updater (0, 4, m_c0_r4);
m_c1_r4 = updater (1, 4, m_c1_r4);
m_c2_r4 = updater (2, 4, m_c2_r4);
m_c3_r4 = updater (3, 4, m_c3_r4);
m_c4_r4 = updater (4, 4, m_c4_r4);
m_c0_r5 = updater (0, 5, m_c0_r5);
m_c1_r5 = updater (1, 5, m_c1_r5);
m_c2_r5 = updater (2, 5, m_c2_r5);
m_c3_r5 = updater (3, 5, m_c3_r5);
m_c4_r5 = updater (4, 5, m_c4_r5);
m_c0_r6 = updater (0, 6, m_c0_r6);
m_c1_r6 = updater (1, 6, m_c1_r6);
m_c2_r6 = updater (2, 6, m_c2_r6);
m_c3_r6 = updater (3, 6, m_c3_r6);
m_c4_r6 = updater (4, 6, m_c4_r6);
m_c0_r7 = updater (0, 7, m_c0_r7);
m_c1_r7 = updater (1, 7, m_c1_r7);
m_c2_r7 = updater (2, 7, m_c2_r7);
m_c3_r7 = updater (3, 7, m_c3_r7);
m_c4_r7 = updater (4, 7, m_c4_r7);
m_c0_r8 = updater (0, 8, m_c0_r8);
m_c1_r8 = updater (1, 8, m_c1_r8);
m_c2_r8 = updater (2, 8, m_c2_r8);
m_c3_r8 = updater (3, 8, m_c3_r8);
m_c4_r8 = updater (4, 8, m_c4_r8);
m_c0_r9 = updater (0, 9, m_c0_r9);
m_c1_r9 = updater (1, 9, m_c1_r9);
m_c2_r9 = updater (2, 9, m_c2_r9);
m_c3_r9 = updater (3, 9, m_c3_r9);
m_c4_r9 = updater (4, 9, m_c4_r9);
}
}
}
I've this code that iterate some samples and build a simple linear interpolation between the points:
foreach sample:
base = floor(index_pointer)
frac = index_pointer - base
out = in[base] * (1 - frac) + in[base + 1] * frac
index_pointer += speed
// restart
if(index_pointer >= sample_length)
{
index_pointer = 0
}
using "speed" equal to 1, the game is done. But if the index_pointer is different than 1 (i.e. got fractional part) I need to wrap last/first element keeping the translation consistent.
How would you do this? Double indexes?
Here's an example of values I have. Let say in array of 4 values: [8, 12, 16, 20].
It will be:
1.0*in[0] + 0.0*in[1]=8
0.28*in[0] + 0.72*in[1]=10.88
0.56*in[1] + 0.44*in[2]=13.76
0.84*in[2] + 0.14*in[3]=16.64
0.12*in[2] + 0.88*in[3]=19.52
0.4*in[3] + 0.6*in[4]=8 // wrong; here I need to wrapper
the last point is wrong. [4] will be 0 because I don't have [4], but the first part need to take care of 0.4 and the weight of first sample (I think?).
Just wrap around the indices:
out = in[base] * (1 - frac) + in[(base + 1) % N] * frac
, where % is the modulo operator and N is the number of input samples.
This procedure generates the following line for your sample data (the dashed lines are the interpolated sample points, the circles are the input values):
I think I understand the problem now (answer only applies if I really did...):
You sample values at a nominal speed sn. But actually your sampler samples at a real speed s, where s != sn. Now, you want to create a function which re-samples the series, sampled at speed s, so it yields a series as if it were sampled with speed sn by means of linear interpolation between 2 adjacent samples. Or, your sampler jitters (has variances in time when it actually samples, which is sn + Noise(sn)).
Here is my approach - a function named "re-sample". It takes the sample data and a list of desired re-sample-points.
For any re-sample point which would index outside the raw data, it returns the respective border value.
let resample (data : float array) times =
let N = Array.length data
let maxIndex = N-1
let weight (t : float) =
t - (floor t)
let interpolate x1 x2 w = x1 * (1.0 - w) + x2 * w
let interp t1 t2 w =
//printfn "t1 = %d t2 = %d w = %f" t1 t2 w
interpolate (data.[t1]) (data.[t2]) w
let inter t =
let t1 = int (floor t)
match t1 with
| x when x >= 0 && x < maxIndex ->
let t2 = t1 + 1
interp t1 t2 (weight t)
| x when x >= maxIndex -> data.[maxIndex]
| _ -> data.[0]
times
|> List.map (fun t -> t, inter t)
|> Array.ofList
let raw_data = [8; 12; 16; 20] |> List.map float |> Array.ofList
let resampled = resample raw_data [0.0..0.2..4.0]
And yields:
val resample : data:float array -> times:float list -> (float * float) []
val raw_data : float [] = [|8.0; 12.0; 16.0; 20.0|]
val resampled : (float * float) [] =
[|(0.0, 8.0); (0.2, 8.8); (0.4, 9.6); (0.6, 10.4); (0.8, 11.2); (1.0, 12.0);
(1.2, 12.8); (1.4, 13.6); (1.6, 14.4); (1.8, 15.2); (2.0, 16.0);
(2.2, 16.8); (2.4, 17.6); (2.6, 18.4); (2.8, 19.2); (3.0, 20.0);
(3.2, 20.0); (3.4, 20.0); (3.6, 20.0); (3.8, 20.0); (4.0, 20.0)|]
Now, I still fail to understand the "wrap around" part of your question. In the end, interpolation - in contrast to extrapolation is only defined for values in [0..N-1]. So it is up to you to decide if the function should produce a run time error or simply use the edge values (or 0) for time values out of bounds of your raw data array.
EDIT
As it turned out, it is about how to use a cyclic (ring) buffer for this as well.
Here, a version of the resample function, using a cyclic buffer. Along with some operations.
update adds a new sample value to the ring buffer
read reads the content a ring buffer element as if it were a normal array, indexed from [0..N-1].
initXXX functions which create the ring buffer in various forms.
length which returns the length or capacity of the ring buffer.
The ring buffer logics is factored into a module to keep it all clean.
module Cyclic =
let wrap n x = x % n // % is modulo operator, just like in C/C++
type Series = { A : float array; WritePosition : int }
let init (n : int) =
{ A = Array.init n (fun i -> 0.);
WritePosition = 0
}
let initFromArray a =
let n = Array.length a
{ A = Array.copy a;
WritePosition = 0
}
let initUseArray a =
let n = Array.length a
{ A = a;
WritePosition = 0
}
let update (sample : float ) (series : Series) =
let wrapper = wrap (Array.length series.A)
series.A.[series.WritePosition] <- sample
{ series with
WritePosition = wrapper (series.WritePosition + 1) }
let read i series =
let n = Array.length series.A
let wrapper = wrap (Array.length series.A)
series.A.[wrapper (series.WritePosition + i)]
let length (series : Series) = Array.length (series.A)
let resampleSeries (data : Cyclic.Series) times =
let N = Cyclic.length data
let maxIndex = N-1
let weight (t : float) =
t - (floor t)
let interpolate x1 x2 w = x1 * (1.0 - w) + x2 * w
let interp t1 t2 w =
interpolate (Cyclic.read t1 data) (Cyclic.read t2 data) w
let inter t =
let t1 = int (floor t)
match t1 with
| x when x >= 0 && x < maxIndex ->
let t2 = t1 + 1
interp t1 t2 (weight t)
| x when x >= maxIndex -> Cyclic.read maxIndex data
| _ -> Cyclic.read 0 data
times
|> List.map (fun t -> t, inter t)
|> Array.ofList
let input = raw_data
let rawSeries0 = Cyclic.initFromArray input
(resampleSeries rawSeries0 [0.0..0.2..4.0]) = resampled
I have a F# list and I'm taking two elements of that list.
If the list has 10 elements in it :
let rnd = new Random()
let elem1 = list.Item(rnd.Next(0,9))
let elem2 = list.Item(rnd.Next(0,9))
There is a chance elem1 and elem2 are equal.
I have checked some workarounds and most of them work using a do while, but I don't want to implement a function that may never end in F#.
Is there a way to create a restriction in the random function?
First random : 0 <= x <= 9
Second random : 0 <= y <= 9 <> x
A simple solution:
let rnd = new Random()
let ndx1 = rnd.Next(9)
let ndx2 =
let x = rnd.Next(8)
if x < ndx1 then x else x + 1
let elem1, elem2 = list.[ndx1], list.[ndx2]
Another way, using maths and calling the random function once:
let r = Random().Next(9 * 8)
let x = 1 + r + r / 9
let elem1, elem2 = list.[x / 9], list.[x % 9]
which may be generalised to:
let getTwoElements lst =
let c = List.length lst
let x, y = Math.DivRem(Random().Next(c * (c-1)) * (c+1) / c + 1, c)
lst.[x], lst.[y]
A more declarative approach, taking into account your comment about points in the image:
let rnd = System.Random()
/// this will build you a list of 10 pairs of indices where a <> b.
let indices =
Seq.initInfinite (fun _ -> rnd.Next(0,10), rnd.Next(0,10))
|> Seq.filter (fun (a,b) -> a <> b)
|> Seq.take 10
|> List.ofSeq
/// map indices into actual points.
let elems =
let points = list |> Array.ofList
List.map (fun (a, b) -> points.[a], points.[b]) indices
As a side note, do not use random access on lists. They're not made for that and performance of that is poor. Convert them to an array first.
There are lots of way to achieve this. A simple one would be something like this:
open System
open System.Linq
let rnd = new Random()
let elem1 = list.Item(rnd.Next(0,9))
let elem2 = list.Where(fun x->x <> elem1).ElementAt(rnd.Next(0,8))
I'm trying to learn F# by rewriting some C# algorithms I have into idiomatic F#.
One of the first functions I'm trying to rewrite is a batchesOf where:
[1..17] |> batchesOf 5
Which would split the sequence into batches with a max of five in each, i.e:
[[1; 2; 3; 4; 5]; [6; 7; 8; 9; 10]; [11; 12; 13; 14; 15]; [16; 17]]
My first attempt at doing this is kind of ugly where I've resorted to using a mutable ref object after running into errors trying to use mutable type inside the closure. Using ref is particularly unpleasant since to dereference it you have to use the ! operator which when inside a condition expression can be counter intuitive to some devs who will read it as logical not. Another problem I ran into is where Seq.skip and Seq.take are not like their Linq aliases in that they will throw an error if size exceeds the size of the sequence.
let batchesOf size (sequence: _ seq) : _ list seq =
seq {
let s = ref sequence
while not (!s |> Seq.isEmpty) do
yield !s |> Seq.truncate size |> List.ofSeq
s := System.Linq.Enumerable.Skip(!s, size)
}
Anyway what would be the most elegant/idiomatic way to rewrite this in F#? Keeping the original behaviour but preferably without the ref mutable variable.
Implementing this function using the seq<_> type idiomatically is difficult - the type is inherently mutable, so there is no simple nice functional way. Your version is quite inefficient, because it uses Skip repeatedly on the sequence. A better imperative option would be to use GetEnumerator and just iterate over elements using IEnumerator. You can find various imperative options in this snippet: http://fssnip.net/1o
If you're learning F#, then it is better to try writing the function using F# list type. This way, you can use idiomatic functional style. Then you can write batchesOf using pattern matching with recursion and accumulator argument like this:
let batchesOf size input =
// Inner function that does the actual work.
// 'input' is the remaining part of the list, 'num' is the number of elements
// in a current batch, which is stored in 'batch'. Finally, 'acc' is a list of
// batches (in a reverse order)
let rec loop input num batch acc =
match input with
| [] ->
// We've reached the end - add current batch to the list of all
// batches if it is not empty and return batch (in the right order)
if batch <> [] then (List.rev batch)::acc else acc
|> List.rev
| x::xs when num = size - 1 ->
// We've reached the end of the batch - add the last element
// and add batch to the list of batches.
loop xs 0 [] ((List.rev (x::batch))::acc)
| x::xs ->
// Take one element from the input and add it to the current batch
loop xs (num + 1) (x::batch) acc
loop input 0 [] []
As a footnote, the imperative version can be made a bit nicer using computation expression for working with IEnumerator, but that's not standard and it is quite advanced trick (for example, see http://fssnip.net/37).
A friend asked me this a while back. Here's a recycled answer. This works and is pure:
let batchesOf n =
Seq.mapi (fun i v -> i / n, v) >>
Seq.groupBy fst >>
Seq.map snd >>
Seq.map (Seq.map snd)
Or an impure version:
let batchesOf n =
let i = ref -1
Seq.groupBy (fun _ -> i := !i + 1; !i / n) >> Seq.map snd
These produce a seq<seq<'a>>. If you really must have an 'a list list as in your sample then just add ... |> Seq.map (List.ofSeq) |> List.ofSeq as in:
> [1..17] |> batchesOf 5 |> Seq.map (List.ofSeq) |> List.ofSeq;;
val it : int list list = [[1; 2; 3; 4; 5]; [6; 7; 8; 9; 10]; [11; 12; 13; 14; 15]; [16; 17]]
Hope that helps!
This can be done without recursion if you want
[0..20]
|> Seq.mapi (fun i elem -> (i/size),elem)
|> Seq.groupBy (fun (a,_) -> a)
|> Seq.map (fun (_,se) -> se |> Seq.map (snd));;
val it : seq<seq<int>> =
seq
[seq [0; 1; 2; 3; ...]; seq [5; 6; 7; 8; ...]; seq [10; 11; 12; 13; ...];
seq [15; 16; 17; 18; ...]; ...]
Depending on how you think this may be easier to understand. Tomas' solution is probably more idiomatic F# though
Hurray, we can use List.chunkBySize, Seq.chunkBySize and Array.chunkBySize in F# 4, as mentioned by Brad Collins and Scott Wlaschin.
This isn't perhaps idiomatic but it works:
let batchesOf n l =
let _, _, temp', res' = List.fold (fun (i, n, temp, res) hd ->
if i < n then
(i + 1, n, hd :: temp, res)
else
(1, i, [hd], (List.rev temp) :: res))
(0, n, [], []) l
(List.rev temp') :: res' |> List.rev
Here's a simple implementation for sequences:
let chunks size (items:seq<_>) =
use e = items.GetEnumerator()
let rec loop i acc =
seq {
if i = size then
yield (List.rev acc)
yield! loop 0 []
elif e.MoveNext() then
yield! loop (i+1) (e.Current::acc)
else
yield (List.rev acc)
}
if size = 0 then invalidArg "size" "must be greater than zero"
if Seq.isEmpty items then Seq.empty else loop 0 []
let s = Seq.init 10 id
chunks 3 s
//output: seq [[0; 1; 2]; [3; 4; 5]; [6; 7; 8]; [9]]
My method involves converting the list to an array and recursively chunking the array:
let batchesOf (sz:int) lt =
let arr = List.toArray lt
let rec bite curr =
if (curr + sz - 1 ) >= arr.Length then
[Array.toList arr.[ curr .. (arr.Length - 1)]]
else
let curr1 = curr + sz
(Array.toList (arr.[curr .. (curr + sz - 1)])) :: (bite curr1)
bite 0
batchesOf 5 [1 .. 17]
[[1; 2; 3; 4; 5]; [6; 7; 8; 9; 10]; [11; 12; 13; 14; 15]; [16; 17]]
I found this to be a quite terse solution:
let partition n (stream:seq<_>) = seq {
let enum = stream.GetEnumerator()
let rec collect n partition =
if n = 1 || not (enum.MoveNext()) then
partition
else
collect (n-1) (partition # [enum.Current])
while enum.MoveNext() do
yield collect n [enum.Current]
}
It works on a sequence and produces a sequence. The output sequence consists of lists of n elements from the input sequence.
You can solve your task with analog of Clojure partition library function below:
let partition n step coll =
let rec split ss =
seq {
yield(ss |> Seq.truncate n)
if Seq.length(ss |> Seq.truncate (step+1)) > step then
yield! split <| (ss |> Seq.skip step)
}
split coll
Being used as partition 5 5 it will provide you with sought batchesOf 5 functionality:
[1..17] |> partition 5 5;;
val it : seq<seq<int>> =
seq
[seq [1; 2; 3; 4; ...]; seq [6; 7; 8; 9; ...]; seq [11; 12; 13; 14; ...];
seq [16; 17]]
As a premium by playing with n and step you can use it for slicing overlapping batches aka sliding windows, and even apply to infinite sequences, like below:
Seq.initInfinite(fun x -> x) |> partition 4 1;;
val it : seq<seq<int>> =
seq
[seq [0; 1; 2; 3]; seq [1; 2; 3; 4]; seq [2; 3; 4; 5]; seq [3; 4; 5; 6];
...]
Consider it as a prototype only as it does many redundant evaluations on the source sequence and not likely fit for production purposes.
This version passes all my tests I could think of including ones for lazy evaluation and single sequence evaluation:
let batchIn batchLength sequence =
let padding = seq { for i in 1 .. batchLength -> None }
let wrapped = sequence |> Seq.map Some
Seq.concat [wrapped; padding]
|> Seq.windowed batchLength
|> Seq.mapi (fun i el -> (i, el))
|> Seq.filter (fun t -> fst t % batchLength = 0)
|> Seq.map snd
|> Seq.map (Seq.choose id)
|> Seq.filter (fun el -> not (Seq.isEmpty el))
I am still quite new to F# so if I'm missing anything - please do correct me, it will be greatly appreciated.