Two sequential HTTP requests - frp

Sorry for newbie's question (and for my english) :)
I tries to write the following function:
the function downloads a content from URL1 (it's received as argument)
the function parses this content and extract URL2
the function downloads a content from URL2
the content from URL2 is a result of this function
if an error was occured, this function should return Nothing
I know how to execute the HTTP requests. I have a function to parse the request from URL1. But I don't know how:
to execute new request with extracted URL2
to ignore second request if URL2 isn't extracted (or error in URL1 is occured)

I principle you want something like this:
import Maybe
import Http
type Url = String
getContentFromUrl : Maybe Url -> Maybe String
getContentFromUrl url = --your implementation
extractUrlFromContent : Maybe String -> Maybe Url
extractUrlFromContent content = --your implementation
content = getContentFromUrl (Just "http://example.com")
|> extractUrlFromContent
|> getContentFromUrl

Sending an Http means talking to the outside world, which involves Signals in Elm. So the final result from URL2 will come packed in a Signal. As long as you're ok with that, you can use maybe to return the content of in a Maybe in a Signal. For example:
import Maybe
import Http
-- helper functions
isSuccess : Http.Response a -> Bool
isSuccess response = case response of
Http.Success _ -> True
_ -> False
responseToMaybe : Http.Response a -> Maybe.Maybe a
responseToMaybe response = case response of
Http.Success a -> Just a
_ -> Nothing
parseContentAndExtractUrl : String -> String
parseContentAndExtractUrl = identity -- this still requires your implementation
-- URL1
startUrl : String
startUrl = "www.example.com" -- replace this with your URL1
result1 : Signal (Http.Response String)
result1 = Http.sendGet <| constant startUrl
-- URL2
secondUrl : Signal String
secondUrl = result1
|> keepIf isSuccess (Http.Success "")
|> lift (\(Http.Success s) -> s)
|> lift parseContentAndExtractUrl
result2 : Signal (Maybe String)
result2 = secondUrl
|> Http.sendGet
|> lift responseToMaybe
Note that there are plans to make all of this easier to work with: https://groups.google.com/d/topic/elm-discuss/BI0D2b-9Fig/discussion

Related

F# | How to manage WebSocketClient ReceiveAsync on multithread scenario?

Looking for WebSocketClient example I only found simple example with a single request/response scenario.
Kind of:
type WSClientSimple (url) =
let ws = new ClientWebSocket()
let lockConnection = Object()
let connect() =
lock lockConnection ( fun () ->
if not (ws.State = WebSocketState.Open) then
ws.ConnectAsync(Uri(url), CancellationToken.None)
|> Async.AwaitTask |> Async.RunSynchronously // await
else ()
)
let receive () =
lock lockConnection ( fun () ->
let rec readStream finalText endOfMessage =
let buffer = ArraySegment(Array.zeroCreate<byte> 1024)
let result = ws.ReceiveAsync(buffer, CancellationToken.None) |> Async.AwaitTask |> Async.RunSynchronously
let text = finalText + Encoding.UTF8.GetString (buffer.Array |> Array.take result.Count)
if result.EndOfMessage then text
else readStream text true
readStream "" false
)
let sendRequest jsonMessage =
let bytes = Encoding.UTF8.GetBytes(jsonMessage:string)
let bytesMessage = ArraySegment(bytes, 0, bytes.Length)
if not (ws.State = WebSocketState.Open) then
connect()
// send request...
ws.SendAsync(bytesMessage, WebSocketMessageType.Text, true, CancellationToken.None) |> Async.AwaitTask |> Async.RunSynchronously
// ... read response
receive()
member this.SendRequest request = sendRequest request
Obviously it works with:
[<Test>]
member this.``Receive sequentially`` () =
let client = WSClientSimple("url")
for i in 1..100 do
client.SendRequest "aaa" |> ignore
and also (thanks to the orrible lock) with multiple thread using the same Client:
[<Test>]
member this.``Receive parallel on same client`` () =
let client = WSClientSimple("url")
for _ in 1..100 do
async {
client.SendRequest "aaa" |> ignore
} |> Async.Start
Now, if I really want to get the beast from WebSocket "duplex" cpmmunication I would continuosly read from the socket, send requests without any block, and distribute the received messages to the right call.
So, this is an ongoing receive function that collect all the inbound messages.
type WSClientTest2 (url:string) =
let onMessageReceived = new Event<string>()
let responseMessage = new Event<ResponseMessage>()
let receivedMesasages = System.Collections.Concurrent.ConcurrentQueue<ResponseMessage>()
let responseCallbacks = Map.empty<int, (string -> unit)>
let manageMessage (message:string) =
match message.Split(':') with
| [|id;message|] ->
responseMessage.Trigger {Id=int(id);Message=message}
receivedMesasages.Enqueue {Id=int(id);Message=message}
| _ -> ()
let startReceiving() =
let mutable counter = 1
async {
// simulate receiving from a WebSocket
while true do
System.Threading.Tasks.Task.Delay 100 |> Async.AwaitTask |> Async.RunSynchronously
onMessageReceived.Trigger (sprintf "message %d" counter)
manageMessage (sprintf "%d:message" counter)
counter <- counter + 1
} |> Async.Start
do
startReceiving()
How can I send a request and wait for the correlated response message?
This is my try:
let mutable requestId = 0
let sendRequest message: string =
let requestId = requestId+1
let received = new Event<string>()
let receivedCall = fun (msg:string) ->
received.Trigger msg
responseCallbacks.Add(requestId, receivedCall) |> ignore
let cancel = fun () -> failwith "Timeout"
async {
System.Threading.Thread.Sleep 500 // wait x seconds
cancel()
} |> Async.Start
// simulate send/receive messsage after some time
let generateRequest () =
System.Threading.Thread.Sleep 100 // wait x time for the response
responseMessage.Trigger {Id=requestId; Message=message}
generateRequest()
Async.AwaitEvent(received.Publish, cancel)
|> Async.RunSynchronously
Async.AwaitWaitHandle seems the right thing to use but I don't know how to create a WaitHandle.
I'm using Async.AwaitEvent but it seems not to work.
The cancel() is always called but it does not raise any Exception!
What could be a proper way to wait for an Event while executing a function and then check and return its content?
I also tried to use a Map<id, response> populatd with any inbound message but still I don't know how to "wait" for the proper message and also it probably requires a check for orphan response messages (add complexity).
More in general, if the resulting code is so crappy I would prefer to use a simple API for this Request/Response scenario and use the WebSocket only for a realtime update.
I'm looking for a nice solution, otherwise I think it is not really worth for the sake of performance, not for my needs.

decode image data from a HTTP string response. (Elm 0.18)

I'd like to fetch an image from a remote server and load it into the following data structure:
type alias Pixel = { r : Int, g : Int, b: Int, a: Float }
type alias ImageData = { pixels : Array Pixel, width: Int }
In the project for some reasons I'm still using Elm 0.18, and elm-lang/http only let's you handle strings as responses from http requests.
init : ( Model, Cmd Msg )
init =
( {}
, Http.send
(\response ->
case response of
Ok data ->
let
_ =
Debug.log "ok" data
in
NoOp
Err error ->
let
_ =
Debug.log "error" error
in
NoOp
)
(Http.getString "https://upload.wikimedia.org/wikipedia/commons/thumb/f/fb/Creamer_MET_186969.jpg/2px-Creamer_MET_186969.jpg")
)
This logs:
ok: "����\0C\0\n\r\t\n\n\r\r��\0\v\0\0\"\0��\0\0\0\0\0\0\0\0\0\0\0\t\n\v��\0�\0\0\0}\0!1AQa\"q2���#B��R��$3br�\t\n%&'()*456789:CDEFGHIJSTUVWXYZcdefghijstuvwxyz���������������������������������������������������������������������������\0\0\0?\0�9�(�.t�/���=~���"
But how to convert it into the above data structure? Also, does it matter if the image is a jpg, png, gif?
What I'm looking for is the implementation of a function imageDataFromString : String -> Maybe ImageData
note: I'd prefer a non-port solution even if performance will be lower, the images I'm trying to process are not large.

Asynchronous function calls in Parallel

Following version is calling all functions synchronously,
I'm looking to find out how to call asynchronous functions in parallel and return all results and errors to the caller.
Request
let requestAsync (url: string) : Async<Result<string, Error>> =
async {
Console.WriteLine ("Simulating request " + url)
try
do! Async.Sleep(1000)
return Ok (url + ": body...")
with :? WebException as e ->
return Error {code = 500; message = "Internal Server Error";}
}
Test
[<TestMethod>]
member this.TestrequestAsync() =
let urls = [|
"http://www.example.com/1";
"http://www.example.com/2";
"http://www.example.com/3";
"http://www.example.com/4";
"http://www.example.com/5";
"http://www.example.com/6";
"http://www.example.com/7";
"http://www.example.com/8";
"http://www.example.com/9";
"http://www.example.com/10";
|]
urls
|> Array.map (fun url -> requestAsync url |> Async.RunSynchronously) // Async.Parallel some mismatch
// Iterate results
Ideally to be able to match Ok and Error results while iterating through results
Edit based on the answer.
let result =
urls
|> Seq.map Entity.requestDetailAsync2
|> Async.Parallel
|> Async.RunSynchronously
result
|> Array.iter Console.WriteLine // match x with Ok and Error?
Attempt
result |> Array.iter (fun data -> match data with
| Ok result -> Console.WriteLine(result)
| Error error -> Console.WriteLine(error) )
Iteration using For in
for r in result do
match r with
| Ok re -> Console.WriteLine(re)
| Error error -> Console.WriteLine(error)
You can use Async.Parallel to run many async operations in parallel:
let results =
urls
|> Seq.map requestAsync // seq<Async<'T>>
|> Async.Parallel // async<T' []>
|> Async.RunSynchronously // T' []
Here's a very similar example on MSDN.
There may be an issue with your requestAsync function return type, or a missing type definition in your example. Here's what I used to verify the solution:
type RequestError = {
code : int
message : string
}
let requestAsync (url: string) =
async {
Console.WriteLine ("Simulating request " + url)
try
do! Async.Sleep(1000)
return Ok (url + ": body...")
with :? WebException as e ->
return Error {code = 500; message = "Internal Server Error";}
}

Method missing when running FSharp.Data app

I have a mono/.Net 4.5 app that compiles just fine. But whe I run it I get a Method missing Http.Request. The code in question is this
let private post url parser body =
let res = Http.Request (
url,
body = (body |> TextRequest),
silentHttpErrors = true,
headers = [
Accept HttpContentTypes.Json
ContentType HttpContentTypes.Json
]
)
let body =
match res.Body with
HttpResponseBody.Text str -> str
| _ -> failwith "Only text replies are supported"
if res.StatusCode >= 200 && res.StatusCode < 300 then
body |> parser
else
body |> errorParser
It doesn't seem to be related with the actual method because all method calls from FSharp.Data seems to fail.
I'm experiencing this both when running some standard nunit tests or when executing.
I would seem that the issue was that I had FSharp.Data.TypeProviders installed in the GAC. removing that
gacutil -u FSharp.Data.TypeProviders
solved it

Unintuitive empty formatted string

I want to add a listener mechanism to a Format-based logging facility, and I ended up in a situation where my program is typed by OCaml and compiles, but the formatted string just disappeared, and I don't understand exactly why this happens (it's related to formatters returning unit when they should return something else, but I expected the program not to type-check in that case).
This comes from a real use case; its simplification may however have led into a somewhat contrived program.
The basic need is this: to devise a Format.printf-like function (with variadic arguments) that is easy to use but also allows other formatters to be notified (e.g. duplicating their outputs).
I've been told this is not possible due to typing constraints, and indeed if I further simplify my example below, I do get typing errors, but for some reason the program below does type-check but does not produce the expected result.
open Format
let observers : formatter list ref = ref []
let add_observer o : unit =
observers := o :: !observers
let print_to_fmt (fmt: formatter) (text: ('a, formatter, unit) format) : unit =
Format.fprintf fmt "<";
Format.fprintf fmt text;
Format.fprintf fmt ">#."
let notify text : unit =
List.iter (fun fmt ->
Format.printf "MESSAGE: {";
Format.printf text;
Format.printf "}#.";
print_to_fmt fmt text
) !observers
let buffer = ref ""
let append text _ _ = buffer := text
let print text =
let fmt = Format.make_formatter append (fun () -> ()) in
Format.kfprintf (fun f -> ()) fmt text
let log text =
notify text;
print text
let () =
add_observer (Format.err_formatter);
log "this works";
log "this does not %d" 42;
log "this also works"
Any help on how to (1) change the program to display this does not 42, or (2) an explanation on why the program type-checks when it seems it shouldn't, would be much appreciated.
You're trying to do a very strange magic with formatters, that I would classify as an abuse, honestly. Formatter is a formatted channel, not data, so they impose all problems of channels, like non-persistent data that disappear suddenly.
If you want to have a log function, that will dispatch data between registered formatters, then the following will work:
open Format
let observers : formatter list ref = ref []
let add_observer o : unit =
observers := o :: !observers
let notify (text : string) : unit =
List.iter (fun fmt ->
fprintf fmt "MESSAGE: {%s}#." text) !observers
let log text = ksprintf notify text
let () =
add_observer Format.err_formatter;
log "this works";
log "this does not %d" 42;
log "this also works"
Will rend the following output:
MESSAGE: {this works}
MESSAGE: {this does not 42}
MESSAGE: {this also works}

Resources