I am playing with the Racket recently, currently implementing a lexer for a subset of scripting language...and wondering whether there are more examples to show how to use the lexer generator in Racket?
I understand the calculator example in the doc, but it is too simple to show the full features...
I am especially curious how to reference position of each character....
The libraries at PLaneT are a great resource for more elaborate examples.
Try for example the Infix Parser .
Related
Is it possible to write documentation in source files like in Common Lisp or Go, for example, and extract it from source files? Or everybody uses Scribble to document their code?
The short answer is you can write in-source documentation by using scribble/srcdoc.
Unlike the other languages you mentioned, these aren't "doc strings":
Although you can write plain text, you have full Racket at-expressions and may use scribble/manual forms and functions.
Not only does this allow for "richer" documentation, the documentation goes into its own documentation submodule -- similar to how you can put tests into test submodules. This means the documentation or tests add no runtime overhead.
You do need one .scrbl file, in which you use scribble/extract to include the documentation submodule(s). However you probably want such a file, anyway, for non-function-specific documentation (topics such as introduction, installation, or "user's guide" prose as opposed to "reference" style documentation).
Of course you can define your own syntax to wrap scribble/srcdoc. For example, in one project I defined a little define/doc macro, which expands into proc-doc/names as well as a (module+ test ___) form. That way, doc examples can also be used as unit tests.
How Racket handles in-source documentation intersects a few interesting aspects of Racket:
Submodules let you define things like "test-time" and "doc-time" as well as run-time.
At-expressions are a different syntax for s-expressions, especially good when writing text.
Scribble is an example of using a custom language -- documentation-as-a-program -- demonstrating Racket's ability to be not just a programming language, but a programming-language programming language.
I am currently studying Datalog for my report in class, I only need to discuss basic syntax and a basic example.
I know Datalog is usually used as a query language and is usually only implemented to other languages such as Java, Lua, C, etc. but is it possible to teach Datalog only by itself, or am I required to use another language implementing it to show a simple working program?
There are a few online demos you can use:
http://iris-reasoner.org/demo
http://ysangkok.github.io/mitre-datalog.js/wrapper.html
https://repl.logicblox.com/ (docs here:
https://developer.logicblox.com/content/docs4/tutorial/repl/section/split.html )
I haven't tried it myself yet, but http://abcdatalog.seas.harvard.edu also looks nice and easy to run.
A complete overview is on Wikipedia: https://en.wikipedia.org/wiki/Datalog
I have been curious lately about DSLs, specifically, how to implement them in Lisp,
since it looks like a piece of cake compare to the alternatives.
Looking for information I cannot find any evidence of a non-lisp DSEL in Lisp in internet.
So my question is:
Is it possible to implement a DSL with non-lisp syntax in lisp with the use of macros?
How is this achieved?
Can the reader of lisp be replaced by a custom reader that translates code to lisp structure?
If the former is true: is this a common way to implement "non-lispy" DSELs?
Short version: Racket does this.
In more detail: Racket, a descendant of Scheme, has a really well-thought-out story here. A Racket module/file can begin with a language declaration, e.g.
#lang algol60
... and then the rest of the file can be written in the given language. (Yes, algol60 is built in.)
In order to develop your own language, you need to write a package that is a language specification, that shows how to expand the syntax of this language into the syntax of the underlying language (in this case, Racket). Anyone can write such packages, and then distribute them to allow others to write programs in this language. There are examples of such language specifications included with Racket, e.g. the algol 60 example mentioned earlier.
I think this is exactly what you're asking for?
ObDisclaimer: Yes, I am a Racket developer.
How do you implement the surface language of a programming language? You write a parser or use a parser generator. You can do that in Lisp, too.
There are many examples of general purpose and domain specific languages written in Lisp - not using s-expression syntax.
Historically the first ML (an extension language for a theorem prover) was written in Lisp. Macsyma (a language for computer algebra) is written in Lisp. In many cases there is some kind of 'end user', for which a non-s-expression language needs to be written/supported. Sometimes there are languages which exist and need to be supported.
Using macros and read macros you can implement some languages or extend the Lisp language. For example it is easy to add JSON syntax to Lisp using a read macro. Also some kind of infix syntax. XML (example: XMLisp).
There's no problem in supporing non-Lisp syntax DSLs in Lisp. You'll need to use some parser/parser generator library as Rainer has mentioned. A good example is esrap that is used to parse markdown (see 3bmd) and also for the pgloader command language which is just an example of an external DSL you're asking about.
From Let Over Lambda, there is an implementation of Perl style regular expressions: http://letoverlambda.com/index.cl/guest/chap4.html#sec_4.
Also there are several attempts at making a "non-lispy" version of Lisp, the main one being the Readable Lisp S-expression Project: http://readable.sourceforge.net/.
One implementation-specific solution that sticks out (if you want to use Scheme rather than CL) is Gambit Scheme's built-in support for infix syntax via its SIX-script extension.
This provides a rich set of loosely C-like operators and syntax forms, which can either be used out-of-the-box to write code in a C-like style, or redefined to mean whatever you want (you can easily redefine e.g. the function definition format, if you aren't a fan of type name(args) {}). for, case, := and so on (even goto) are all already present and ready to mean whatever you need.
The actual core of the syntax (operator precedence, expressions vs. statements) is fixed, but you can assign things like a Scheme binding construct to the s-expression produced by an operator for a reasonably large amount of freedom.
a = b * c;
is translated by the reader into
(six.x=y (six.identifier a) (six.x*y (six.identifier b) (six.identifier c)))
You can then override the definitions of those macros with your own to make the syntax do whatever you want. Turning the C-style base into a Haskell-looking functional language isn't too hard (strategically redefine = and -> and you're halfway there...).
I have no idea how to build S-exp.
I want to do it, because I need to build AST for my langauge.
At the beginning I used RubyParser to parse it to sexp then code gen.
But it must be ruby's subset I think.I cant define the language what I want.
Now I need to implement parser for my language.
So anyone could recommend any ruby tool that building AST for S-expression ?
Thanks!
It is not very clear from your question what exactly do you need, but simple Google search gives some interesting links to check. Maybe after checking these links, if they are not the answer to your question, you can edit question and make it more precise and concrete.
http://thingsaaronmade.com/blog/writing-an-s-expression-parser-in-ruby.html
https://github.com/aarongough/sexpistol
You might try the sxp-ruby gem at http://github.com/bendiken/sxp-ruby. I use it for SPARQL S-Expressions (SSE) and similar methods for managing Abstract Syntax Trees in Ruby.
Maybe you could have a look at this gem named Astrapi.
This is just an experiment :
describe your language elements (concepts) in a "mm" file (abstract syntax)
run astrapi on this file
astrapi generates a parser that is able to fill up your AST, from your input source expressed in s-expression (concrete syntax of your concepts).
I have put a modest documentation here.
I am learning my way around Scheme, and I am especially interested in how the language is constructed. I'm trying to find a nice description of the core syntax for a Scheme implementation. I don't know enough about the standards, but I assume that they all contain macro systems. If not, I'd like to read about a standard that also includes macros (they can't possibly be implemented in simpler Scheme constructs, can they?).
Does anyone have a good reference for the minimal syntax needed for a Scheme dialect?
Just an update:
I also stumbled upon this: http://matt.might.net/articles/compiling-to-java/#sec1. If you also add define-syntax and delay then it seems like it might be a good start.
In the R5RS specification, the following page appears to be what I was looking for: formal syntax
Although it may be a bit dry, you should read over the R5RS spec or the R6RS spec.
The docs really do not take that long to read through and you can just skim most of the sections until you need more detail. But either document does cover all of the minimal syntax required, including macros.