Can I get AST for wolfram language expressions? - wolfram-mathematica

Question

In Mathematica, we use FullForm or TreeForm or Developer'WriteExpressionJSONString to get the syntax details for given expressions. How can I get a complete AST (Abstract Syntax Tree) for any expression? For example, is there any function toAST such that
toAST["a +b c\nSin[%];"]
which will give the result like this:
{
{
Plus,
0,
6,
{a, 0, 1},
{Multiply, 3, 6, {b, 3, 4}, {c, 5, 6}}
},
{
CompoundExpression,
7,
14,
{Sin, 7, 13, {Out, 11, 12}},
{Null, 14, 14}
}
}

Answer 1

Probably your best option ATM is codeparser. It'll be bundled with the next version of Mathematica, but you can use PacletInstall["CodeParser"] to install it for now.
The function you want to use (i.e. for ASTs) is CodeParse. (You can get CSTs with CodeConcreteParse.) The documentation seems to be a bit scarce.
Needs["CodeParser`"];
ast = CodeParse[parseStr]
(* Output *)
ContainerNode[String, {CallNode[
LeafNode[Symbol,
"Plus", <||>], {LeafNode[Symbol,
"a", <|Source -> {{1, 1}, {1, 2}}|>],
CallNode[
LeafNode[Symbol,
"Times", <||>], {LeafNode[Symbol,
"b", <|Source -> {{1, 4}, {1, 5}}|>],
LeafNode[Symbol,
"c", <|Source -> {{1, 6}, {1, 7}}|>]}, <|Source -> {{1, 4}, {1,
7}}|>]}, <|Source -> {{1, 1}, {1, 7}}|>],
CallNode[LeafNode[Symbol,
"CompoundExpression", <||>], {CallNode[
LeafNode[Symbol,
"Sin", <|Source -> {{2, 1}, {2, 4}}|>], {CallNode[
LeafNode[Symbol,
"Out", <||>], {}, <|Source -> {{2, 5}, {2,
6}}|>]}, <|Source -> {{2, 1}, {2, 7}}|>],
LeafNode[Symbol,
"Null", <|Source -> {{2, 8}, {2, 8}}|>]}, <|Source -> {{2,
1}, {2, 8}}|>]}, <||>]
You can use Developer`WriteExpressionJSONString that you mentioned or ExportString[ast, "ExpressionJSON"] to get output pretty close to what you wanted, albeit more verbose (so I've squashed it down here):
ExportString[ast[[2;;]], "ExpressionJSON", Compact -> 3]
(* Output *)
[
"ContainerNode",
[
"List",
[
"CallNode",
["LeafNode","Symbol","'Plus'",["Association"]],
["List",["LeafNode","Symbol","'a'",["Association",["Rule","Source",["List",["List",1,1],["List",1,2]]]]],["CallNode",["LeafNode","Symbol","'Times'",["Association"]],["List",["LeafNode","Symbol","'b'",["Association",["Rule","Source",["List",["List",1,4],["List",1,5]]]]],["LeafNode","Symbol","'c'",["Association",["Rule","Source",["List",["List",1,6],["List",1,7]]]]]],["Association",["Rule","Source",["List",["List",1,4],["List",1,7]]]]]],
["Association",["Rule","Source",["List",["List",1,1],["List",1,7]]]]
],
[
"CallNode",
["LeafNode","Symbol","'CompoundExpression'",["Association"]],
["List",["CallNode",["LeafNode","Symbol","'Sin'",["Association",["Rule","Source",["List",["List",2,1],["List",2,4]]]]],["List",["CallNode",["LeafNode","Symbol","'Out'",["Association"]],["List"],["Association",["Rule","Source",["List",["List",2,5],["List",2,6]]]]]],["Association",["Rule","Source",["List",["List",2,1],["List",2,7]]]]],["LeafNode","Symbol","'Null'",["Association",["Rule","Source",["List",["List",2,8],["List",2,8]]]]]],
["Association",["Rule","Source",["List",["List",2,1],["List",2,8]]]]
]
],
[
"Association"
]
]