Asynchronous function calls in Parallel - parallel-processing

Following version is calling all functions synchronously,
I'm looking to find out how to call asynchronous functions in parallel and return all results and errors to the caller.
Request
let requestAsync (url: string) : Async<Result<string, Error>> =
async {
Console.WriteLine ("Simulating request " + url)
try
do! Async.Sleep(1000)
return Ok (url + ": body...")
with :? WebException as e ->
return Error {code = 500; message = "Internal Server Error";}
}
Test
[<TestMethod>]
member this.TestrequestAsync() =
let urls = [|
"http://www.example.com/1";
"http://www.example.com/2";
"http://www.example.com/3";
"http://www.example.com/4";
"http://www.example.com/5";
"http://www.example.com/6";
"http://www.example.com/7";
"http://www.example.com/8";
"http://www.example.com/9";
"http://www.example.com/10";
|]
urls
|> Array.map (fun url -> requestAsync url |> Async.RunSynchronously) // Async.Parallel some mismatch
// Iterate results
Ideally to be able to match Ok and Error results while iterating through results
Edit based on the answer.
let result =
urls
|> Seq.map Entity.requestDetailAsync2
|> Async.Parallel
|> Async.RunSynchronously
result
|> Array.iter Console.WriteLine // match x with Ok and Error?
Attempt
result |> Array.iter (fun data -> match data with
| Ok result -> Console.WriteLine(result)
| Error error -> Console.WriteLine(error) )
Iteration using For in
for r in result do
match r with
| Ok re -> Console.WriteLine(re)
| Error error -> Console.WriteLine(error)

You can use Async.Parallel to run many async operations in parallel:
let results =
urls
|> Seq.map requestAsync // seq<Async<'T>>
|> Async.Parallel // async<T' []>
|> Async.RunSynchronously // T' []
Here's a very similar example on MSDN.
There may be an issue with your requestAsync function return type, or a missing type definition in your example. Here's what I used to verify the solution:
type RequestError = {
code : int
message : string
}
let requestAsync (url: string) =
async {
Console.WriteLine ("Simulating request " + url)
try
do! Async.Sleep(1000)
return Ok (url + ": body...")
with :? WebException as e ->
return Error {code = 500; message = "Internal Server Error";}
}

Related

F# | How to manage WebSocketClient ReceiveAsync on multithread scenario?

Looking for WebSocketClient example I only found simple example with a single request/response scenario.
Kind of:
type WSClientSimple (url) =
let ws = new ClientWebSocket()
let lockConnection = Object()
let connect() =
lock lockConnection ( fun () ->
if not (ws.State = WebSocketState.Open) then
ws.ConnectAsync(Uri(url), CancellationToken.None)
|> Async.AwaitTask |> Async.RunSynchronously // await
else ()
)
let receive () =
lock lockConnection ( fun () ->
let rec readStream finalText endOfMessage =
let buffer = ArraySegment(Array.zeroCreate<byte> 1024)
let result = ws.ReceiveAsync(buffer, CancellationToken.None) |> Async.AwaitTask |> Async.RunSynchronously
let text = finalText + Encoding.UTF8.GetString (buffer.Array |> Array.take result.Count)
if result.EndOfMessage then text
else readStream text true
readStream "" false
)
let sendRequest jsonMessage =
let bytes = Encoding.UTF8.GetBytes(jsonMessage:string)
let bytesMessage = ArraySegment(bytes, 0, bytes.Length)
if not (ws.State = WebSocketState.Open) then
connect()
// send request...
ws.SendAsync(bytesMessage, WebSocketMessageType.Text, true, CancellationToken.None) |> Async.AwaitTask |> Async.RunSynchronously
// ... read response
receive()
member this.SendRequest request = sendRequest request
Obviously it works with:
[<Test>]
member this.``Receive sequentially`` () =
let client = WSClientSimple("url")
for i in 1..100 do
client.SendRequest "aaa" |> ignore
and also (thanks to the orrible lock) with multiple thread using the same Client:
[<Test>]
member this.``Receive parallel on same client`` () =
let client = WSClientSimple("url")
for _ in 1..100 do
async {
client.SendRequest "aaa" |> ignore
} |> Async.Start
Now, if I really want to get the beast from WebSocket "duplex" cpmmunication I would continuosly read from the socket, send requests without any block, and distribute the received messages to the right call.
So, this is an ongoing receive function that collect all the inbound messages.
type WSClientTest2 (url:string) =
let onMessageReceived = new Event<string>()
let responseMessage = new Event<ResponseMessage>()
let receivedMesasages = System.Collections.Concurrent.ConcurrentQueue<ResponseMessage>()
let responseCallbacks = Map.empty<int, (string -> unit)>
let manageMessage (message:string) =
match message.Split(':') with
| [|id;message|] ->
responseMessage.Trigger {Id=int(id);Message=message}
receivedMesasages.Enqueue {Id=int(id);Message=message}
| _ -> ()
let startReceiving() =
let mutable counter = 1
async {
// simulate receiving from a WebSocket
while true do
System.Threading.Tasks.Task.Delay 100 |> Async.AwaitTask |> Async.RunSynchronously
onMessageReceived.Trigger (sprintf "message %d" counter)
manageMessage (sprintf "%d:message" counter)
counter <- counter + 1
} |> Async.Start
do
startReceiving()
How can I send a request and wait for the correlated response message?
This is my try:
let mutable requestId = 0
let sendRequest message: string =
let requestId = requestId+1
let received = new Event<string>()
let receivedCall = fun (msg:string) ->
received.Trigger msg
responseCallbacks.Add(requestId, receivedCall) |> ignore
let cancel = fun () -> failwith "Timeout"
async {
System.Threading.Thread.Sleep 500 // wait x seconds
cancel()
} |> Async.Start
// simulate send/receive messsage after some time
let generateRequest () =
System.Threading.Thread.Sleep 100 // wait x time for the response
responseMessage.Trigger {Id=requestId; Message=message}
generateRequest()
Async.AwaitEvent(received.Publish, cancel)
|> Async.RunSynchronously
Async.AwaitWaitHandle seems the right thing to use but I don't know how to create a WaitHandle.
I'm using Async.AwaitEvent but it seems not to work.
The cancel() is always called but it does not raise any Exception!
What could be a proper way to wait for an Event while executing a function and then check and return its content?
I also tried to use a Map<id, response> populatd with any inbound message but still I don't know how to "wait" for the proper message and also it probably requires a check for orphan response messages (add complexity).
More in general, if the resulting code is so crappy I would prefer to use a simple API for this Request/Response scenario and use the WebSocket only for a realtime update.
I'm looking for a nice solution, otherwise I think it is not really worth for the sake of performance, not for my needs.

F# Fabulous Xamarin: external event subscription

I'm new to Fabulous and MUV model, and I'm trying to implement application that works with BLE. I'm also a bit new to F#, mostly worked with erlang and C# in the past, so a bit lost with external events processing. CrossBluetoothLE.Current.Adapter has DeviceDiscovered event handler (IEvent). What's the most correct way of linking this event handler to the Fabulous update function?
E.g. after I will call CrossBluetoothLE.Current.Adapter.StartScanningForDevicesAsync(), I want that this event handler supply newly discovered devices to the update function.
And if I will do something like this (this is not working):
type MyApp () as app =
inherit Application ()
let deviceDiscovered dispatch =
CrossBluetoothLE.Current.Adapter.DeviceDiscovered.Subscribe (fun x -> dispatch (App.Msg.Discovered x.Device) )
let runner =
App.program
|> Program.withConsoleTrace
|> Program.withSubscription (fun _ -> Cmd.ofSub deviceDiscovered)
|> XamarinFormsProgram.run app
if it works, it will be ok for device discovery because CrossBluetoothLE.Current.Adapter is static. However after device will be discovered, I will need to work with (e.g. receive notifications or replies from it), and it will not be possible to include dynamic device handler into Program.withSubscription.
Not sure whether the Fabulous is applicable here.
Ok, I was able to find some solution and it works now, but the overall architecture looks a bit weird. So generic approach is to create an external mailbox, that will dispatch messages to the MUV loop.
Describe all messages of the MUV in the external module, e.g.:
type Msg =
| Scan
| Discovered of IDevice
| Connect of IDevice
| ClockMsg of System.DateTime
| TextMsg of string
Create type that encapsulates mailbox:
type DispatchFunc = Msgs.Msg -> unit
type State =
| Initialized of DispatchFunc
| NotInitialized
type Mail =
| Dispatch of DispatchFunc
| Msg of Msgs.Msg
| None
let rand = System.Random()
let id = rand.NextDouble()
let postbox = MailboxProcessor.Start(fun inbox ->
let rec messageLoop (state:State) = async{
let! mail = inbox.Receive()
let new_state =
match mail with
| None ->
state
| Msg msg ->
match state with
| NotInitialized -> NotInitialized
| Initialized df ->
df msg
state
| Dispatch df ->
Initialized df
return! messageLoop (new_state)
}
messageLoop (NotInitialized))
let post(o) =
postbox.Post o
Here, mailbox starts with NotInitialized state and wait while application will start. When everything is done, mailbox received dispatch function, that will be used in further dispatching of the external messages to the MUV main loop.
Pass dispatch handler to the mailbox:
type MyApp () as app =
inherit Application ()
// generate initial events + start threads + pass dispatch reference to the mailbox
let initThreads dispatch =
// init & start external (e.g. bluetooth receiver) threads here
// or start them asynchronously from MUV loop
Postbox.post (Postbox.Dispatch dispatch)
()
let runner =
App.program
|> Program.withConsoleTrace
|> Program.withSubscription (fun _ -> Cmd.ofSub initThreads)
|> XamarinFormsProgram.run app
So now, if you want to send event to the MUV from external thread, just start it inside initThreads (or, e.g. from within MUV loop) and use something like: Postbox.post (Postbox.Msg (Msgs.TextMsg "It works!")).
E.g. for my purposes (BLE discovery) it will look like this:
let update msg model =
match msg with
| Msgs.Scan ->
CrossBluetoothLE.Current.Adapter.StopScanningForDevicesAsync() |> Async.AwaitTask |> ignore
CrossBluetoothLE.Current.Adapter.DeviceDiscovered.Subscribe (
fun (a) ->
Postbox.post (Postbox.Msg (Msgs.Discovered a.Device))
()
) |> ignore
CrossBluetoothLE.Current.Adapter.StartScanningForDevicesAsync() |> Async.AwaitTask |> ignore
model, Cmd.none
| Msgs.ClockMsg msg ->
{ model with label = msg.ToString() }, Cmd.none
| Msgs.TextMsg msg ->
{ model with label = msg }, Cmd.none
| Msgs.Discovered d ->
{ model with gattDevices = d::model.gattDevices; label = "Discovered " + d.ToString() }, Cmd.none
| Msgs.Connect d -> { model with connectedDevice = d }, Cmd.none
This is for sure a very ugly solution, but I wasn't able to imagine something more beautiful :(.

how to get f# signature - visual studio or vs code

I want to be able to explicitly write type signatures in my code.
VS code will (eventually) generate sort of ghost signatures, but I actually want to explicitly take these generated signatures and type the code.
Any ideas? I could use FSI, but that can be quite a cumbersome technique.
Ideally I'd right click and "generate signature"..though that doesn't always fit peoples coding style...I tend to write code;
let f : int -> string =
fun i -> i.ToString()
You can get the type of an F# function using the Compiler Services SDK. This would require writing a custom analyzer for your projects, but it should be a reusable component that you can integrate into your development process once implemented. The basic steps to resolve every function's type signature would be:
Create an F# Type Checker (FSharpChecker) instance.
Load your project options (FSharpProjectOptions).
Parse and check each file (FSharpChecker.parseAndCheckFileInProject).
Retrieve the Declarations list from each type-checker result (FSharpCheckFileAnswer).
Print the type signature (FSharpType) for each declaration.
Here's a quick solution I put together as a starting point:
#r #"FSharp.Compiler.Service.25.0.1\lib\net45\FSharp.Compiler.Service.dll"
#r #"FSharp.Compiler.Service.ProjectCracker.25.0.1\lib\net45\FSharp.Compiler.Service.ProjectCracker.dll"
open Microsoft.FSharp.Compiler.SourceCodeServices
open System
open System.IO
type Namespace =
{
Name: string
XmlDoc: System.Collections.Generic.IList<string>
}
type Declaration =
| Namespace of Namespace * Declaration list
| Module of FSharpEntity * Declaration list
| Class of FSharpEntity * Declaration list
| Interface of FSharpEntity * Declaration list
| Enum of FSharpEntity * Declaration list
| Record of FSharpEntity * Declaration list
| Union of FSharpEntity * Declaration list
| Function of FSharpMemberOrFunctionOrValue
| Binding of FSharpMemberOrFunctionOrValue
let checker = FSharpChecker.Create(1, true)
let getProject projectFile =
ProjectCracker.GetProjectOptionsFromProjectFile(projectFile)
let private isNamespace (declaration: FSharpImplementationFileDeclaration) =
match declaration with
| FSharpImplementationFileDeclaration.Entity (entity, children) -> entity.IsNamespace
| _ -> false
let rec private getDeclaration nsSoFar (declaration: FSharpImplementationFileDeclaration) =
[
match declaration with
| FSharpImplementationFileDeclaration.Entity (entity, children) ->
if entity.IsNamespace then
if children.Length = 1 && children.Head |> isNamespace
then match nsSoFar with
| Some ns -> yield! getDeclaration (Some <| sprintf "%s.%s" ns entity.DisplayName) children.Head
| None -> yield! getDeclaration (Some entity.DisplayName) children.Head
else match nsSoFar with
| Some ns ->
let nsEntity = {Name = sprintf "%s.%s" ns entity.DisplayName; XmlDoc = entity.XmlDoc}
yield Namespace (nsEntity, children |> List.collect (getDeclaration nsSoFar))
| None ->
let nsEntity = {Name = entity.DisplayName; XmlDoc = entity.XmlDoc}
yield Namespace (nsEntity, children |> List.collect (getDeclaration nsSoFar))
elif entity.IsClass then
yield Class (entity, children |> List.collect (getDeclaration nsSoFar))
elif entity.IsInterface then
yield Interface (entity, children |> List.collect (getDeclaration nsSoFar))
elif entity.IsEnum then
yield Enum (entity, children |> List.collect (getDeclaration nsSoFar))
elif entity.IsFSharpModule then
yield Module (entity, children |> List.collect (getDeclaration nsSoFar))
elif entity.IsFSharpRecord then
yield Record (entity, children |> List.collect (getDeclaration nsSoFar))
elif entity.IsFSharpUnion then
yield Union (entity, children |> List.collect (getDeclaration nsSoFar))
else
()
| FSharpImplementationFileDeclaration.MemberOrFunctionOrValue (func, _, _) ->
if func.IsValCompiledAsMethod
then yield Function func
else yield Binding func
| _ -> ()
]
let getDeclarations (project: FSharpProjectOptions) file =
async {
let source = File.ReadAllText file
let! (parseResults, checkResults) = checker.ParseAndCheckFileInProject(file, 1, source, project)
return
match checkResults with
| FSharpCheckFileAnswer.Succeeded checkInfo ->
match checkInfo.ImplementationFile with
| Some implementation -> implementation.Declarations |> List.collect (getDeclaration None)
| None -> failwithf "No Implementation Available for File %s" file
| error -> failwithf "Error Checking File %s:\r\n%A" file error
}
let getDeclarationsForScript file =
async {
let source = File.ReadAllText file
let! (project, _) = checker.GetProjectOptionsFromScript(file, source)
return! getDeclarations project file
}
Then, if we have a sample script file called "Test.fsx" with a function like your example inside it (let f i = sprintf "%d" i), we can print the function's signature like so:
let getTypeName (t: FSharpType) =
t.Format(FSharpDisplayContext.Empty).Replace("Microsoft.FSharp.Core.", "")
let rec printFunctionSignatures declarations =
for declaration in declarations do
match declaration with
| Namespace (_, ds) -> printFunctionSignatures ds
| Module (_, ds) -> printFunctionSignatures ds
| Function f -> f.FullType |> getTypeName |> printfn "%s: %s" f.DisplayName
| _ -> () // Handle all the other cases
getDeclarationsForScript "Test.fsx"
|> Async.RunSynchronously
|> printFunctionSignatures
This will pint out:
f: int -> string

Two sequential HTTP requests

Sorry for newbie's question (and for my english) :)
I tries to write the following function:
the function downloads a content from URL1 (it's received as argument)
the function parses this content and extract URL2
the function downloads a content from URL2
the content from URL2 is a result of this function
if an error was occured, this function should return Nothing
I know how to execute the HTTP requests. I have a function to parse the request from URL1. But I don't know how:
to execute new request with extracted URL2
to ignore second request if URL2 isn't extracted (or error in URL1 is occured)
I principle you want something like this:
import Maybe
import Http
type Url = String
getContentFromUrl : Maybe Url -> Maybe String
getContentFromUrl url = --your implementation
extractUrlFromContent : Maybe String -> Maybe Url
extractUrlFromContent content = --your implementation
content = getContentFromUrl (Just "http://example.com")
|> extractUrlFromContent
|> getContentFromUrl
Sending an Http means talking to the outside world, which involves Signals in Elm. So the final result from URL2 will come packed in a Signal. As long as you're ok with that, you can use maybe to return the content of in a Maybe in a Signal. For example:
import Maybe
import Http
-- helper functions
isSuccess : Http.Response a -> Bool
isSuccess response = case response of
Http.Success _ -> True
_ -> False
responseToMaybe : Http.Response a -> Maybe.Maybe a
responseToMaybe response = case response of
Http.Success a -> Just a
_ -> Nothing
parseContentAndExtractUrl : String -> String
parseContentAndExtractUrl = identity -- this still requires your implementation
-- URL1
startUrl : String
startUrl = "www.example.com" -- replace this with your URL1
result1 : Signal (Http.Response String)
result1 = Http.sendGet <| constant startUrl
-- URL2
secondUrl : Signal String
secondUrl = result1
|> keepIf isSuccess (Http.Success "")
|> lift (\(Http.Success s) -> s)
|> lift parseContentAndExtractUrl
result2 : Signal (Maybe String)
result2 = secondUrl
|> Http.sendGet
|> lift responseToMaybe
Note that there are plans to make all of this easier to work with: https://groups.google.com/d/topic/elm-discuss/BI0D2b-9Fig/discussion

How to create new syntax block for try with finally?

I really often use:
try
try
with
finally
so I'm interesting if is possible to make new syntax operator to not write "try" two times.
let mytry foo bar foobar =
try
try
foo
with
| _ -> bar // weird part here, I want to have a match
finally foobar
mytry
<| foo
<| | :? SocketException ->
| _ -> // ok it looks funny but how to realize it?
<| foobar
the problems I see here are
non-common syntax, in mytry there is no try with finally keywords, just <| <| <| for each, but it's lesser trouble I guess
with: I don't know how can I realize this part. even how it will look if I can realize it...
The question is whether you really need try/finally. Most of the time try/finally is used for disposing resources even when exceptions occur. But you can always replace it by the use keyword.
For example:
open System.IO
let openFile(url: string) =
let fileStream = File.OpenText(url)
try
try
let readline = fileStream.ReadLine()
printfn "Readline: %s" readline
with
| :? IOException as ex ->
printfn "IOException: %A" ex
| ex -> printfn "Another exception: %A" ex
finally
fileStream.Dispose()
can be rewritten as:
let openFile(url: string) =
use fileStream = File.OpenText(url)
try
let readline = fileStream.ReadLine()
printfn "Readline: %s" readline
with
| :? IOException as ex ->
printfn "IOException: %A" ex
| ex -> printfn "Another exception: %A" ex
For the learning purpose, you can define mytry using high-order functions as follows:
let mytry foo bar foobar =
try
try
foo ()
with
| exn -> bar exn
finally foobar ()
But it doesn't look really nice on above example:
let myOpenFile(url: string) =
let fileStream = File.OpenText(url)
mytry (fun () -> let readline = fileStream.ReadLine()
printfn "Readline: %s" readline)
(fun ex -> match ex with
| :? IOException ->
printfn "IOException: %A" ex
| _ -> printfn "Another exception: %A" ex)
(fun () -> fileStream.Dispose())
You can write a higher-order function that takes the three parts as separate function. The body of the try would be a function unit -> 'R where 'R is the result. The exception handler will need to handle only some exceptions, so you can return option to say whether you handled the result or if you want the exception to be rethrown. The type of handler will be exn -> 'R option. The finalizer is then simply a function unit -> unit.
The usage is not as elegant as using built-in language feature, but it does the trick:
tryWithFinally
(fun () ->
1/0 ) // The nested body
(function
| :? DivideByZeroException -> Some -1 // Handle division by zero
| _ -> None ) // Rethrow any other exceptions
(fun () ->
printfn "done" )
The implementation is quite easy once you know the structure, but for completeness, here it is:
let tryWithFinally f handler finalizer =
try
try f()
with e ->
match handler e with
| Some r -> r
| None -> reraise()
finally
finalizer()
Anyway, I agree with #pad that in most of the cases, you should be fine with just use and try .. with.

Resources