OCaml - How to serialize and deserialize Yaml to records - yaml

What are the current community preferred libraries to parse and work with YAML and how do you use them to serialize and deserialize a record like this:
type book = {
title: string;
authors: string list
}

This is how I got string -> record and back.
$ opam update
$ opam install yaml ppx_deriving_yaml
Update dune with the preprocess clause:
; `dune` file
(executable
(name main)
(libraries yaml)
(preprocess
(pps ppx_deriving_yaml)))
Short version:
let serialize_book (book_rec : book) : (string, string) result =
let res = Yaml.to_string (book_to_yaml book_rec) in
map_error ~f:(fun (`Msg m) -> m) res
let deserialize_book (book_str : string) : (book, string) result =
let res =
Yaml.of_string book_str >>= fun yaml_value -> book_of_yaml yaml_value
in
map_error ~f:(fun (`Msg m) -> m) res
More verbose/descriptive version:
(* Define a record *)
(* `[##deriving yaml]` generates a bunch of functions, one being `book_to_yaml` to convert the record into a Yaml type, another `book_of_yaml` to convert Yaml type to record *)
type book = {
title: string;
authors: string list
} [##deriving yaml]
let serialize =
let (v: book) = { title = "Cryptonomicon"; authors = [ "Neal Stephenson" ] } in
(* `book_to_yaml` converts from record to `yaml res` where res is a Result *)
let yaml_structure = book_to_yaml v in
(* `to_string` converts from a `yaml` type'ed data structure to string *)
match Yaml.to_string yaml_structure with
| Ok s ->
print_endline ("Serialize:");
print_endline (s)
| Error (`Msg e) -> print_endline e
let deserialize =
let str = "title: Cryptonomicon\nauthors:\n- Neal Stephenson" in
(* `of_string converts from string to a `yaml res` data structure, where `res` is Result *)
match Yaml.of_string str with
| Ok yaml_value ->
(* `book_of_yaml` is generated by `[##deriving yaml]` *)
(* `book_of_yaml` converts from `yaml` type to `book res` where res is Result *)
(match book_of_yaml yaml_value with
| Ok t ->
print_endline ("Deserialize:");
print_endline ("Title: " ^ t.title);
print_endline ("Authors: " ^ String.concat ", " t.authors);
| Error `Msg e -> print_endline ("Error - convert to book: " ^ e))
| Error `Msg e -> print_endline ("Error - parsing: " ^ e)

Related

Pandoc `writeMarkdown` does not include metadata, i.e. write (read a) is not a - what extension is required?

I want to creat an utilty to transform some text in a markdown file and want to procude a new markdown file. The metadata is not changed.
The operations read and write should be inverse (or at least idempotent), but I cannot find a way to have pandoc reproduce the input file including the metadata. What combination of Extensions and Options are required?
Here my minimal working example with the newest pandoc from lst-15.13.
-- ---------------------------------------------------------------------------
--
-- Module : a test for pandoc output of yaml
-- ---------------------------------------------------------------------------
{-# LANGUAGE ScopedTypeVariables #-}
{-# LANGUAGE OverloadedStrings #-}
module Main where
import qualified Data.Text as T
import Text.Pandoc
import Control.Monad.IO.Class (liftIO )
main :: IO ()
main = do
putStrLns $ ["input", showT inputMd]
res <- sub inputMd
case res of
Left st -> putStrLns $ ["error", show st]
Right q ->
do
putStrLns $ ["result q\n", showT $ q]
putStrLns $ ["should be the same as input p\n", showT inputMd]
putStrLns $ ["same", show (inputMd == ( q))]
return ()
return ()
showT = show . T.unpack
putStrLns = putStrLn . unwords
sub :: T.Text -> IO (Either PandocError T.Text)
sub input1 = do
res <- runIO $
do
let readOptions = def{readerStandalone = True
, readerExtensions = extensionsFromList
[Ext_yaml_metadata_block ]
}
p <- readMarkdown readOptions ( input1)
-- def does not analyse metadata
-- readOptions analysis metadata
-- output is then only the markdown
-- t1 :: String <- liftIO $readFile "/home/frank/Workspace8/pandocTest/temp.tpl"
-- t2 <- compileDefaultTemplate (T.pack t1)
let writeOptions = def {writerSetextHeaders = False
, writerExtensions = extensionsFromList
[Ext_yaml_metadata_block]
-- , writerTemplate = Just t2
}
q <- writeMarkdown writeOptions p
-- def gives only the markdown part
-- but not including the titel
liftIO $ putStrLns ["AST \n", show p]
return q
res2 <- handleError res
return res
inputMd = T.unlines ["---"
,"title: The Future of AA"
,"..."
,""
,"## second level"
,"text for nothing"
] :: T.Text
The use of a template comiles but gives a runtime error:
Could not find data file /home/frank/.stack/snapshots/x86_64-linux/668b320207ef95ba5255b2b20895a7f7315ff61076bb3ab82e76f7ef56076320/8.8.3/share/x86_64-linux-ghc-8.8.3/pandoc-2.9.1.1/data/templates/default.
$if(titleblock)$ $titleblock$

how to get f# signature - visual studio or vs code

I want to be able to explicitly write type signatures in my code.
VS code will (eventually) generate sort of ghost signatures, but I actually want to explicitly take these generated signatures and type the code.
Any ideas? I could use FSI, but that can be quite a cumbersome technique.
Ideally I'd right click and "generate signature"..though that doesn't always fit peoples coding style...I tend to write code;
let f : int -> string =
fun i -> i.ToString()
You can get the type of an F# function using the Compiler Services SDK. This would require writing a custom analyzer for your projects, but it should be a reusable component that you can integrate into your development process once implemented. The basic steps to resolve every function's type signature would be:
Create an F# Type Checker (FSharpChecker) instance.
Load your project options (FSharpProjectOptions).
Parse and check each file (FSharpChecker.parseAndCheckFileInProject).
Retrieve the Declarations list from each type-checker result (FSharpCheckFileAnswer).
Print the type signature (FSharpType) for each declaration.
Here's a quick solution I put together as a starting point:
#r #"FSharp.Compiler.Service.25.0.1\lib\net45\FSharp.Compiler.Service.dll"
#r #"FSharp.Compiler.Service.ProjectCracker.25.0.1\lib\net45\FSharp.Compiler.Service.ProjectCracker.dll"
open Microsoft.FSharp.Compiler.SourceCodeServices
open System
open System.IO
type Namespace =
{
Name: string
XmlDoc: System.Collections.Generic.IList<string>
}
type Declaration =
| Namespace of Namespace * Declaration list
| Module of FSharpEntity * Declaration list
| Class of FSharpEntity * Declaration list
| Interface of FSharpEntity * Declaration list
| Enum of FSharpEntity * Declaration list
| Record of FSharpEntity * Declaration list
| Union of FSharpEntity * Declaration list
| Function of FSharpMemberOrFunctionOrValue
| Binding of FSharpMemberOrFunctionOrValue
let checker = FSharpChecker.Create(1, true)
let getProject projectFile =
ProjectCracker.GetProjectOptionsFromProjectFile(projectFile)
let private isNamespace (declaration: FSharpImplementationFileDeclaration) =
match declaration with
| FSharpImplementationFileDeclaration.Entity (entity, children) -> entity.IsNamespace
| _ -> false
let rec private getDeclaration nsSoFar (declaration: FSharpImplementationFileDeclaration) =
[
match declaration with
| FSharpImplementationFileDeclaration.Entity (entity, children) ->
if entity.IsNamespace then
if children.Length = 1 && children.Head |> isNamespace
then match nsSoFar with
| Some ns -> yield! getDeclaration (Some <| sprintf "%s.%s" ns entity.DisplayName) children.Head
| None -> yield! getDeclaration (Some entity.DisplayName) children.Head
else match nsSoFar with
| Some ns ->
let nsEntity = {Name = sprintf "%s.%s" ns entity.DisplayName; XmlDoc = entity.XmlDoc}
yield Namespace (nsEntity, children |> List.collect (getDeclaration nsSoFar))
| None ->
let nsEntity = {Name = entity.DisplayName; XmlDoc = entity.XmlDoc}
yield Namespace (nsEntity, children |> List.collect (getDeclaration nsSoFar))
elif entity.IsClass then
yield Class (entity, children |> List.collect (getDeclaration nsSoFar))
elif entity.IsInterface then
yield Interface (entity, children |> List.collect (getDeclaration nsSoFar))
elif entity.IsEnum then
yield Enum (entity, children |> List.collect (getDeclaration nsSoFar))
elif entity.IsFSharpModule then
yield Module (entity, children |> List.collect (getDeclaration nsSoFar))
elif entity.IsFSharpRecord then
yield Record (entity, children |> List.collect (getDeclaration nsSoFar))
elif entity.IsFSharpUnion then
yield Union (entity, children |> List.collect (getDeclaration nsSoFar))
else
()
| FSharpImplementationFileDeclaration.MemberOrFunctionOrValue (func, _, _) ->
if func.IsValCompiledAsMethod
then yield Function func
else yield Binding func
| _ -> ()
]
let getDeclarations (project: FSharpProjectOptions) file =
async {
let source = File.ReadAllText file
let! (parseResults, checkResults) = checker.ParseAndCheckFileInProject(file, 1, source, project)
return
match checkResults with
| FSharpCheckFileAnswer.Succeeded checkInfo ->
match checkInfo.ImplementationFile with
| Some implementation -> implementation.Declarations |> List.collect (getDeclaration None)
| None -> failwithf "No Implementation Available for File %s" file
| error -> failwithf "Error Checking File %s:\r\n%A" file error
}
let getDeclarationsForScript file =
async {
let source = File.ReadAllText file
let! (project, _) = checker.GetProjectOptionsFromScript(file, source)
return! getDeclarations project file
}
Then, if we have a sample script file called "Test.fsx" with a function like your example inside it (let f i = sprintf "%d" i), we can print the function's signature like so:
let getTypeName (t: FSharpType) =
t.Format(FSharpDisplayContext.Empty).Replace("Microsoft.FSharp.Core.", "")
let rec printFunctionSignatures declarations =
for declaration in declarations do
match declaration with
| Namespace (_, ds) -> printFunctionSignatures ds
| Module (_, ds) -> printFunctionSignatures ds
| Function f -> f.FullType |> getTypeName |> printfn "%s: %s" f.DisplayName
| _ -> () // Handle all the other cases
getDeclarationsForScript "Test.fsx"
|> Async.RunSynchronously
|> printFunctionSignatures
This will pint out:
f: int -> string

Asynchronous function calls in Parallel

Following version is calling all functions synchronously,
I'm looking to find out how to call asynchronous functions in parallel and return all results and errors to the caller.
Request
let requestAsync (url: string) : Async<Result<string, Error>> =
async {
Console.WriteLine ("Simulating request " + url)
try
do! Async.Sleep(1000)
return Ok (url + ": body...")
with :? WebException as e ->
return Error {code = 500; message = "Internal Server Error";}
}
Test
[<TestMethod>]
member this.TestrequestAsync() =
let urls = [|
"http://www.example.com/1";
"http://www.example.com/2";
"http://www.example.com/3";
"http://www.example.com/4";
"http://www.example.com/5";
"http://www.example.com/6";
"http://www.example.com/7";
"http://www.example.com/8";
"http://www.example.com/9";
"http://www.example.com/10";
|]
urls
|> Array.map (fun url -> requestAsync url |> Async.RunSynchronously) // Async.Parallel some mismatch
// Iterate results
Ideally to be able to match Ok and Error results while iterating through results
Edit based on the answer.
let result =
urls
|> Seq.map Entity.requestDetailAsync2
|> Async.Parallel
|> Async.RunSynchronously
result
|> Array.iter Console.WriteLine // match x with Ok and Error?
Attempt
result |> Array.iter (fun data -> match data with
| Ok result -> Console.WriteLine(result)
| Error error -> Console.WriteLine(error) )
Iteration using For in
for r in result do
match r with
| Ok re -> Console.WriteLine(re)
| Error error -> Console.WriteLine(error)
You can use Async.Parallel to run many async operations in parallel:
let results =
urls
|> Seq.map requestAsync // seq<Async<'T>>
|> Async.Parallel // async<T' []>
|> Async.RunSynchronously // T' []
Here's a very similar example on MSDN.
There may be an issue with your requestAsync function return type, or a missing type definition in your example. Here's what I used to verify the solution:
type RequestError = {
code : int
message : string
}
let requestAsync (url: string) =
async {
Console.WriteLine ("Simulating request " + url)
try
do! Async.Sleep(1000)
return Ok (url + ": body...")
with :? WebException as e ->
return Error {code = 500; message = "Internal Server Error";}
}

OCaml: new_line equivalent before 3.11

I'm trying to compile Libra toolkit on a machine running Ubuntu Hardy with OCaml 3.10, I can't upgrade the OS nor update OCaml, and I don't know anything about OCaml. There is only one line that gives me an unbound value error because it uses the new_line function, which was introduced in OCaml 3.11 (http://caml.inria.fr/pub/docs/manual-ocaml/libref/Lexing.html), could someone tell me how to change it to be compatible with OCaml 3.10? It's the line near the end of this code:
{
open MnParseTypes;;
open MnParser;;
(* Raised when parsing ends *)
exception Eof;;
module L = Lexing
let linenum lexbuf = lexbuf.L.lex_curr_p.L.pos_lnum
let line = ref 1;;
let keywords = Hashtbl.create 10
let _ =
List.iter2 (Hashtbl.add keywords)
["mn"; "features"; "tree"; "table"; "w"; "eof"]
[Tmn; Tfeatures; Ttree; Ttable; Tweight; EOF];;
}
let digits = ['0'-'9']+
let identifier = ['a'-'z' 'A'-'Z']+
rule lexer = parse
(* eat blank characters *)
[' ' '\t'] {lexer lexbuf}
(* | "Feature list:" {lexer lexbuf} *)
| '{' {Tlbrace}
| '}' {Trbrace}
| '(' {Tlparen}
| ')' {Trparen}
| ('-')? "inf" {Tfloat( float_of_string(L.lexeme lexbuf))}
| identifier {
let x = String.lowercase (Lexing.lexeme lexbuf) in
try Hashtbl.find keywords x
with Not_found ->
failwith((Lexing.lexeme lexbuf)
^ ": unknown identifier on line " ^ string_of_int (linenum lexbuf))}
| digits {Tint (int_of_string (L.lexeme lexbuf))}
| ('-')? digits ('.' digits)? (['e' 'E'] ['+' '-']? digits)?
{Tfloat( float_of_string(L.lexeme lexbuf))}
| '+' 'v' (digits as var) '_' (digits as value)
{Tcond(true, int_of_string var, int_of_string value)}
| '-' 'v' (digits as var) '_' (digits as value)
{Tcond(false, int_of_string var, int_of_string value)}
| 'v' (digits as var) '_' (digits as value)
{Tvar( int_of_string var, int_of_string value)}
| ['\n' '\r']+ {L.new_line lexbuf; TEOL} (* THIS GIVES THE ERROR *)
| eof {EOF}
| _ {failwith((L.lexeme lexbuf) ^
": mistake on line " ^ string_of_int lexbuf.L.lex_curr_p.L.pos_lnum)}
In the directory of the OCaml sources (from SVN or a relase tarball), the source of the module Foo of the standard library will be in stdlib/foo.{ml,mli} (.mli is the interface file, .ml the implementation file). Looking at stdlib/lexing.ml gives you:
let new_line lexbuf =
let lcp = lexbuf.lex_curr_p in
lexbuf.lex_curr_p <- { lcp with
pos_lnum = lcp.pos_lnum + 1;
pos_bol = lcp.pos_cnum;
}
You can implement this in your code as well, using open Lexing to have the field names in scope, or using lexbuf.Lexing.lex_curr_p, and { lcp with Lexing.pos_lnum = lcp.Lexing.pos_lnum ... instead.
Edit: as you probably don't plan to hack the OCaml code yourself, let's give you the full thing:
let new_line lexbuf =
let lcp = lexbuf.Lexing.lex_curr_p in
lexbuf.Lexing.lex_curr_p <- { lcp with
Lexing.pos_lnum = lcp.Lexing.pos_lnum + 1;
Lexing.pos_bol = lcp.Lexing.pos_cnum;
}
add this at the top of the file that uses new_line (if it says Lexing.new_line, turn it into new_line), and you should be fine.
You can implement new_line yourself but I think upgrading OCaml would be better. I know that you said you can't upgrade the OS but installing the newer version of Ocaml can be done in your home directory without the need of any superuser privilege. OPAM is a packet manager for OCaml that makes it very easy to install the latest version of OCaml.

How to create new syntax block for try with finally?

I really often use:
try
try
with
finally
so I'm interesting if is possible to make new syntax operator to not write "try" two times.
let mytry foo bar foobar =
try
try
foo
with
| _ -> bar // weird part here, I want to have a match
finally foobar
mytry
<| foo
<| | :? SocketException ->
| _ -> // ok it looks funny but how to realize it?
<| foobar
the problems I see here are
non-common syntax, in mytry there is no try with finally keywords, just <| <| <| for each, but it's lesser trouble I guess
with: I don't know how can I realize this part. even how it will look if I can realize it...
The question is whether you really need try/finally. Most of the time try/finally is used for disposing resources even when exceptions occur. But you can always replace it by the use keyword.
For example:
open System.IO
let openFile(url: string) =
let fileStream = File.OpenText(url)
try
try
let readline = fileStream.ReadLine()
printfn "Readline: %s" readline
with
| :? IOException as ex ->
printfn "IOException: %A" ex
| ex -> printfn "Another exception: %A" ex
finally
fileStream.Dispose()
can be rewritten as:
let openFile(url: string) =
use fileStream = File.OpenText(url)
try
let readline = fileStream.ReadLine()
printfn "Readline: %s" readline
with
| :? IOException as ex ->
printfn "IOException: %A" ex
| ex -> printfn "Another exception: %A" ex
For the learning purpose, you can define mytry using high-order functions as follows:
let mytry foo bar foobar =
try
try
foo ()
with
| exn -> bar exn
finally foobar ()
But it doesn't look really nice on above example:
let myOpenFile(url: string) =
let fileStream = File.OpenText(url)
mytry (fun () -> let readline = fileStream.ReadLine()
printfn "Readline: %s" readline)
(fun ex -> match ex with
| :? IOException ->
printfn "IOException: %A" ex
| _ -> printfn "Another exception: %A" ex)
(fun () -> fileStream.Dispose())
You can write a higher-order function that takes the three parts as separate function. The body of the try would be a function unit -> 'R where 'R is the result. The exception handler will need to handle only some exceptions, so you can return option to say whether you handled the result or if you want the exception to be rethrown. The type of handler will be exn -> 'R option. The finalizer is then simply a function unit -> unit.
The usage is not as elegant as using built-in language feature, but it does the trick:
tryWithFinally
(fun () ->
1/0 ) // The nested body
(function
| :? DivideByZeroException -> Some -1 // Handle division by zero
| _ -> None ) // Rethrow any other exceptions
(fun () ->
printfn "done" )
The implementation is quite easy once you know the structure, but for completeness, here it is:
let tryWithFinally f handler finalizer =
try
try f()
with e ->
match handler e with
| Some r -> r
| None -> reraise()
finally
finalizer()
Anyway, I agree with #pad that in most of the cases, you should be fine with just use and try .. with.

Resources