Related
If you evaluate (list-tail '(1 2) 3) at guile scheme. You will get an exception.
It would be smarter to have an '() as answer.
Overall why do we haven't closure property with respect to cdr combinator? What complications may arise?
Examples to make my point clearer
Now (cdr (cdr (cdr '(1 2))) -> raise-exception
Should be (cdr (cdr (cdr ... (cdr '(1 2))...))) -> ()
Then we would automatically have properly working list-tail
(define (list-tail list n)
(if (= n 0)
list
(list-tail (cdr list) (- n 1)))
Group-by then could be written elegantly and exceptionless
(define (group-by list-arg n)
(if (null? list-arg)
'()
(cons (list-head n) (list-tail n))))
The historic answer is that:
Originally, the Lisp 1 and 1.5 languages created by John MacCarthy did not allow (CDR NIL). The CDR function required a cons cell argument.
The idea that it would be convenient for (CDR NIL) to just return NIL came from a dialect called Interlisp (but may have been present elsewhere).
In the 1960's, there was another major dialect of Lisp called MacLisp (two decades before the Apple Mac, unrelated).
According to The Evolution of Lisp by Peter Gabriel and Guy Steele, some MacLisp people held a pow-wow with Interlisp people in 1974:
In 1974, about a dozen persons attended a meeting at MIT between the MacLisp and Interlisp implementors, including Warren Teitelman, Alice Hartley, Jon L White, Jeff Golden, and Guy Steele. There was some hope of finding substantial common ground, but the meeting actually served to illustrate the great chasm separating the two groups, in everything from implementation details to overall design philosophy. [...] In the end only a trivial exchange of features resulted from “the great MacLisp/Interlisp summit”: MacLisp adopted from Interlisp the behavior (CAR NIL) → NIL and (CDR NIL) → NIL, and Interlisp adopted the concept
of a read table.
Both Interlisp and MacLisp are ancestral dialects to Common Lisp, which also has the forgiving car and cdr.
Further remarks are made in the above paper on this matter, begining with:
The adoption of the Interlisp treatment of NIL was not received with universal warmth.
You can see from this fifty, sixty years ago, Lisp people were already divided into camps, and didn't agree on everything. Whether the car of an empty list should just yield the empty list, or error out is a very old issue.
Ashwin Ram, presently director of AI at Google, put in his own opinion in favor of forgiving cdr in 1986, when he composed this poem.
It still remains a divisive issue that is a matter of opinion.
It is undeniable that the flexible car, cdr and their derivatives can help you "code golf" list processing code.
It's also true that such code-golfed code sometimes handles only happy cases without error checking, which can cause problems in some circumstances.
For instance, some list that is assumed to always have three items is subject to (caddr list) to get the third item. But, due to some bug, it has only two. Now the code just ran off with the nil value, which may cause a problem somewhwere else. For instance, suppose the value is expected to be a string, and in some totally different function elsewhere, nil is passed to some API that needs a string and blows up. Now you're hunting through the code to discover where this nil came from.
People who write Lisp interpreters or compilers that rely on the forgiving destructuring performed by car and cdr end up producing something that accepts bad syntax silently.
For instance
(defun interpret-if (form env)
(let ((test (car form))
(then (cadr form))
(else (caddr form)))
(if (interpret-expr test env)
(interpret-expr then env)
(interpret-expr else env))))
This is actually a very nice example for discussing both sides of the issue.
On the one hand, the code is succinct, and nicely supports the optional else clause: the user of this interpreter can do:
(if (> x y)
(print "x is greater than y"))
In interpret-if, the else variable will pull out a nil, and that will get handed off to (eval expr else env) where it just evaluates to nil, and everything is cool; the optionality of else was obtained from free thanks to caddr not complaining.
On the other hand, the interpreter doesn't diagnose this:
(if) ;; no arguments at all
or this:
(if (> x y)) ;; spec says "then" is required, but no error!
However, all these issues have nice solutions and ways of working that don't require tightening up the list accessor functions, so that we can resort to the succinct coding when we need to. For instance, the interpreter could use pattern matching of some kind, like Common Lisp's rudimentary destructuring-bind:
(defun interpret-if (form env)
(destructuring-bind (test then &optional else) form
(if (interpret-expr test env)
(interpret-expr then env)
(interpret-expr else env))))
destructuring-bind has strict checking. It generates code with car, caddr and other functions under the hood, but also error checking code. The list (1 2 3) will not be destructured by the pattern (a b).
You have to look at the entire language and how it is used and what else is in it.
Introducing forgiving car and cdr into Scheme might give you less mileage than you think. There is another issue, which is that the only Boolean false value in Scheme is #f. The empty list () in Scheme is not false.
Therefore, even if car is forgiving, code like this cannot work.
Suppose the third element of a list is always a number, or else it doesn't exist. In Lisp, we can do this to default to zero:
(or (third list) 0)
for that to work in Scheme in the default 0 case, (third list) would have to return the Boolean false value #f.
A plausible approach might be to have different default values for car and cdr:
(car ()) -> #f
(cdr ()) -> ()
However, that is rather arbitrary: it works in some circumstances, but fails in situations like:
;; if list has more than two items ...
(if (cddr list) ...)
If cddr returns () by default, then that is always true, and so the test is useless. Different defaulting for car and cdr would probably be more error prone than common defaulting.
In Lisp, the forgiving list accessors work in a synergistic way with the empty list being false, which is why once upon a time I was quite surprised to learn that the forgiving list accessors came in fairly late into the game.
Early Scheme was implemented as a project written in Lisp, and therefore to interoperate smoothly with the host language, it used the same convention: () being NIL being the empty list and false. This was eventually changed, and so if you're wishing to have that back, you're asking for Scheme to revert a many-decades-old decision which is next to impossible now.
Object-oriented programming weighs in on this also. The fact that (car nil) does something instead of failing is an instance of the Null Object Pattern, which is something useful and good. We can express this in the Common Lisp object system, in which it practically disappears:
Suppose we had a car function which blows up on non-conses. We could write a generic function kar which doesn't, like this:
;; generic fun
(defgeneric kar (obj))
;; method specialization for cons class: delegate to car.
(defmethod kar ((obj cons))
(car obj))
;; specialization for null class:
(defmethod kar ((obj null))) ;; return nil
;; catch all specialization for any type
(defmethod kar ((obj t))
:oops)
Test:
[1]> (kar nil)
NIL
[2]> (kar '(a . b))
A
[3]> (kar "string")
:OOPS
In CLOS, the class null is that class whose only instance is the object nil. When a method parameter specializes to null, that method is only eligible when the argument for that parameter nil.
The class t is the superclass of everything: the top of the type spindle. (There is a bottom of the type spindle also, the class named nil, which contains no instances and is a subclass of everything.)
Specializing methods on null lets us catch method calls with nil parameters. Thanks to CLOS multiple dispatch, these can be thus handled in any parameter position. Because of that, and null being a class, the Null Object Pattern disappears in CLOS.
If you're debating Lisp with OOP people, you can present (car nil) can be spoken about as being the Null Object pattern.
The inconvenience of explicit null handling is recognized in numerous newer programming languages.
A common feature nowadays is to have null safe object access. For instance
foo.bar
might blow up if foo is null. So the given language provides
foo?.bar
or similar, which will only dereference .bar if foo isn't nil, otherwise the expression yields nil.
When languages add foo?.bar, they do not throw away foo.bar, or make foo.bar behave like foo?.bar. Sometimes you want the error (foo being nil is a programming error you want to catch in testing) and sometimes you want the default.
Sometimes you want the default, so you can collapse multiple levels of default and catch an error:
if (foo?.bar?.xyzzy?.fun() == nil) {
// we coudn't have fun(); handle it
// this was because either foo was nil, or else bar was nil,
// or else xyzzy was nil, or else fun() returned nil.
} else {
// happy case
}
cdr is only allowed on pairs. When you reach the end of the list, the value is (), which is not a pair, so you get an error.
You can check for this in your list-tail procedure to allow it to be more permissive.
(define (list-tail list n)
(if (or (= n 0) (not (pair? list)))
list
(list-tail (cdr list) (- n 1)))
Using (not (pair? list)) will also allow it to work for improper lists like (1 2 . 3). It will keep returning 3 for any n >= 2.
"You will get an exception."
This is a problem with many core libraries, and not just Scheme. Take Haskell's core library:
tail [1] -- []
tail [] -- error
head [1] -- 1
head [] -- error
As you know, the technical name for a function like this is a Partial Function. It is a function that doesn't work for some inputs, leading to errors.
So, yes, you can define your own version. One thing, though - what should be returned in the end condition? Should (list-tail '(1 2) 3) return () or should it return 0? If I'm trying to get a value to add to another number, then 0 would be appropriate. If I'm using cons to gather values then () would be appropriate. I guess that's why the function is left as partial.
"Right. I was interested why scheme was designed that way, without car/cdr closure property. Is it feature or just design flaw. It's more like scheme is less consistent than Common Lisp, rather strict."
Common Lisp returns NIL when it runs out of list:
(car '(1)) ; 1
(car '()) ; NIL
(cdr '(1)) ; 1
(cdr '()) ; NIL
In this case you would have to test for NIL, and if you wanted a zero instead make the replacement.
Why Scheme didn't have this is due to its minimalistic design. The report was so under-specified you could do pointer arithmetic and just let the program segfault since any faulty scheme code was deemed not scheme and pigs could fly. Later reports, like R7RS, requires much more error checking since it is required to signal errors in many situations where just undefined behavior would be OK in early reports.
With today's Scheme we can easily create car and cdr that does what you want:
#!r7rs
(define-library
(sylwester pair-accessors)
(export car cdr)
(import (rename (scheme base) (car base:car) (cdr base:cdr))
(except (scheme base) (car cdr)))
(begin
(define (car v)
(if (pair? v)
(base:car v)
'()))
(define (cdr v)
(if (pair? v)
(base:cdr v)
'()))))
So in your library or program you just import (scheme) (or (scheme base)) without car and cdr and also import (sylwester pair-accessors) and you're in business. Alternatively you can make a (scheme base) or (scheme) that replaces all accessors with you own safe ones using a macro to produce them all.
The only thing you cannot do is inject your version of car/cdr into already defined libraries since that would require some late binding or monkey-patching, but that isn't supported by the language. I'm fascinated by these things and would love to make a OO-scheme where you can augment standard procedures with some CLOS-ish late binding where all core functions under the hood are indeed methods so that you can define your own objects and accessors and that standard libraries and user libraries created for normal pairs would just work out of the box for your new data structures that has pair like features.
Suppose I have a list of arguments args and a macro/syntax f that takes a variable number of arguments. How do I apply f to args? Apparently apply doesn't work here.
For example, suppose I have a list of values bs and I want to know if they're all true, so I try (apply and bs) but I get the error "and: bad syntax". The workaround I came up with is (eval `(and . ,bs)) but I'm wondering if there is some standard way to achieve this sort of thing.
Update
A bunch of possible duplicates have been suggested, but most of them are just about the and example. This suggested question seems to be the same as mine, but the answer there is not very helpful: it basically says "don't do this!".
So maybe the point is that in practice this "apply + macro" question only comes up for macros like and and or, and there is no useful general question? I certainly ran into this issue with and, and don't have much Scheme experience, certainly no other examples of this phenomenon.
This is an XY problem. Macros are part of the syntax of the language: they're not functions and you can't apply them to arguments. Conceptually, macros are like functions which map source code to other source code, and which are called at compile time, not run time: trying to use them at run time is a category error. (In Common Lisp macros are, quite literally, functions which map source code to other source code: in Scheme I'm not quite so clear about that).
So if you have a list and you want to know if all its elements are true, you call a function on the list to do that.
It's easy to write such a function:
(define (all-true? things)
(cond [(null? things)
#t]
[(first things)
(all-true? (rest things))]
[else #f]))
However Racket provides a more general function: andmap: (andmap identity things) will either return false if one of things is not true, or it will return the value of the last thing in the list (or #t if the list is empty). (andmap (lambda (x) (and (integer? x) (even? x))) ...) will tell you if all the elements in a list are even integers, for instance.
There is also every which comes from SRFI 1 and which you can use in Racket after (require srfi/1). It is mostly (exactly?) the same as andmap.
One thing people sometimes try to do (and which you seem to be tempted to do) is to use eval. It may not be immediately clear how awful the eval 'solution; is. It is awful because
it doesn't, in fact, work at all;
insofar as it does work it prevents any kind of compilation and optimisation;
last but not least, it's a pathway to code injection attacks.
Let's see how bad it is. Start with this:
> (let ([args '(#t #t #f)])
(eval `(and ,#args)))
#f
OK, that looks good, right? Well, what if I have a list which is (a a a b): none of the elements in that are false, so, let's try that:
> (let ([args '(a a b)])
(eval `(and ,#args)))
; a: undefined;
; cannot reference an identifier before its definition
Oh, well, can I fix that?
> (let ([args '(a a b)]
[a 1] [b 2])
(eval `(and ,#args)))
; a: undefined;
; cannot reference an identifier before its definition
No, I can't. To make that work, I'd need this:
> (define a 1)
> (define b 2)
> (let ([args '(a a b)])
(eval `(and ,#args)))
2
or this
> (let ([args '('a 'a 'b)])
(eval `(and ,#args)))
'b
Yes, those are quotes inside the quoted list.
So that's horrible: the only two cases it's going to work for is where everything is either defined at the top level as eval has no access to the lexical scope where it is called, or a literal within the object which may already be a literal as it is here, because everything is now getting evaluated twice.
So that's just horrible. To make things worse, eval evaluates Scheme source code. So forget about compiling, performance, or any of that good stuff: it's all gone (maybe if you have a JIT compiler, maybe it might not be so awful).
Oh, yes, and eval evaluates Scheme source code, and it evaluates all of it.
So I have this convenient list:
(define args
'((begin (delete-all-my-files)
(publish-all-my-passwords-on-the-internet)
(give-all-my-money-to-tfb)
#t)
(launch-all-the-nuclear-missiles)))
It's just a list of lists of symbols and #t, right? So
> (eval `(and #,args)
; Error: you are not authorized to launch all the missiles
; (but all your files are gone, your passwords are now public,
; and tfb thanks you for your kind donation of all your money)
Oops.
It would be nice to be able to say, if I have a list that I want to check some property of that doing so would not send all my money to some person on the internet. Indeed, it would be nice to know that checking the property of the list would simply halt, at all. But if I use eval I can't know that: checking that every element of the list is (evaluates to) true may simply never terminate, or may launch nuclear weapons, and I can generally never know in advance whether it will terminate, or whether it will launch nuclear weapons. That's an ... undesirable property.
At the very least I would need to do something like this to heavily restrict what can appear in the list:
(define (safely-and-list l)
(for ([e (in-list l)])
(unless
(or (number? e)
(boolean? e))
(error 'safely-and-list "bad list")))
(eval `(and ,#l)))
But ... wait: I've just checked every element of the list: why didn't I just, you know, check they were all true then?
This is why eval is never the right solution for this problem. The thing eval is the right solution for is, well, evaluating Scheme. If you want to write some program that reads user input and evaluates, it, well, eval is good for that:
(define (repl (exit 'exit))
(display "feed me> ")
(flush-output)
(let ([r (read)])
(unless (eqv? r exit)
(writeln (eval r))
(repl exit))))
But if you think you want to apply a macro to some arguments then you almost certainly have an XY problem: you want to do something else, and you likely don't understand macros.
This is not a duplicate of
set-car!, set-cdr! unbound in racket? or
of
Implement SICP evaluator using Racket
or of
How to install sicp package module in racket?,
but rather a follow-up question because the solutions proposed therein do not
work for me. First, the need: Section 5.5.5 of SICP, the compiler plus
explicit-control evaluator (code here in "ch5-eceval-compiler.scm"), are
entirely dependent on set-car! and set-cdr! into explicit quoted lists. I
would like to copy and modify this code without a complete, bottom-up rewrite in
immutable form. I'd also accept a reference to a scheme implementation that can
run the code out-of-the-box or with some minimal, straightforward adaptation,
i.e., a scheme that has set-car! and set-cdr! or some
work-around. Neither guile nor racket give me an easy time running this code.
EDIT: mit scheme will load the eceval compiler. I'm leaving the question up for those who might want to get it going in racket (I would rather, for example).
Here is a deeper explanation, including the things I explored and tried out, and
how I diagnosed the quoted list as the deepest problem. When I hand-converted
the quoted list into an mquoted nest of mlists, the code broke in much worse
ways and the rabbit hole got much deeper. I had to revert after a couple of
hours of delicate brain surgery that failed.
Here is an MVE of the kind of structure that section 5.5.5 relies on. This is small, but structurally like the real thing:
(define foo '(a b))
(set-cdr! foo '(c))
The real thing starts like this:
(define eceval
(make-machine
'(exp env val proc argl continue unev
compapp ;*for compiled to call interpreted
)
eceval-operations ;; ----------------------------------------------
'( ;; <<<<<<<<======== BIG QUOTED LIST CAUSING TROUBLE / NOT MCONSES!
;;SECTION 5.4.4, as modified in 5.5.7 ;; -------------------------------
;;*for compiled to call interpreted (from exercise 5.47)
(assign compapp (label compound-apply))
;;*next instruction supports entry from compiler (from section 5.5.7)
(branch (label external-entry))
read-eval-print-loop
(perform (op initialize-stack))
(perform
(op prompt-for-input) (const ";;; EC-Eval input:"))
...
and goes on for quite a while. The evaluator is the "machine-code" in the quoted
list, and various generated code does set-car! and set-cdr! into registers
and environment frames and other things. The code fails on load.
There seems to be no easy way to convert the evaluator into immutable form
without a full rewrite, and I'm trying to avoid that. Of course, set-car! and
set-cdr! are not available in #lang racket, and I don't think they're in
guile, either (at least guile refused to load "ch5-eceval-compiler.scm,"
throwing a mutability error, on which I did not dig deeper).
One solution proposed in
set-car!, set-cdr! unbound in racket? is
to rewrite the code using mcons, mcar, mlist, etc. according to (require
compatibility/mlist) (require rnrs/mutable-pairs-6). Those compatibility
packages have no replacement for quote, so I tried writing my own mquote. I
spent a couple of hours on such a refactoring, but the exercise was not
converging, just going deeper and deeper down the rabbit hole and ending up with
even deeper problems. It seems that to pursue even a refactoring I must
understand more semantics about "ch5-eceval-compiler.scm," and if I must, I
might as well rewrite it in immutable form.
Easier solutions proposed in
set-car!, set-cdr! unbound in racket?
are to use #lang sicp or #lang r5rs. There follows three experiments that
referenced other answers on stack overflow:
#lang r5rs
(define foo '(a b))
(set-cdr! foo '(c))
foo
Error: struct:exn:fail:contract:variable
set-cdr!: undefined;
cannot reference an identifier before its definition
in module: "/usr/share/racket/pkgs/r5rs-lib/r5rs/main.rkt"
-----------------------------------------------
which points to a place where set-cdr! is clearly defined:
...
(module main scheme/base
(require scheme/mpair
racket/undefined
(for-syntax scheme/base syntax/kerncase
"private/r5rs-trans.rkt")
(only-in mzscheme transcript-on transcript-off))
(provide (for-syntax syntax-rules ...
(rename-out [syntax-rules-only #%top]
[syntax-rules-only #%app]
[syntax-rules-only #%datum]))
(rename-out
[mcons cons]
[mcar car]
[mcdr cdr]
[set-mcar! set-car!] ;; --------------------------
[set-mcdr! set-cdr!] ;; <<<<<<<<======== LOOK HERE
[mpair? pair?] ;; --------------------------
[mmap map]
[mfor-each for-each])
= < > <= >= max min + - * /
abs gcd lcm exp log sin cos tan not eq?
call-with-current-continuation make-string
symbol->string string->symbol make-rectangular
exact->inexact inexact->exact number->string string->number
...
Here is a similar failure with #lang sicp
#lang sicp
(define foo '(a b))
(set-cdr! foo '(c))
foo
Error: struct:exn:fail:contract:variable
set-cdr!: undefined;
cannot reference an identifier before its definition
in module: "/home/rebcabin/.racket/7.2/pkgs/sicp/sicp/main.rkt"
----------------------------------------------------
pointing to code that only indirectly defines set-cdr!, but clearly in the
appropriate package:
....
#lang racket
(require racket/provide ;; --------------------------------------------
(prefix-in r5rs: r5rs) ;; <<<<<<<<======== PULL IN SET-CDR! ETC. HERE?
(rename-in racket [random racket:random])) ;; ------------------------
(provide (filtered-out (λ (name) (regexp-replace #px"^r5rs:" name ""))
(except-out (all-from-out r5rs) r5rs:#%module-begin))
(rename-out [module-begin #%module-begin]))
(define-syntax (define+provide stx)
(syntax-case stx ()
[(_ (id . args) . body) #'(begin
(provide id)
(define (id . args) . body))]
[(_ id expr) #'(begin
(provide id)
(define id expr))]))
...
I dig deeper into
Implement SICP evaluator using Racket
and find
(require (only-in (combine-in rnrs/base-6
rnrs/mutable-pairs-6)
set-car!
set-cdr!))
(define foo '(a b))
(set-cdr! foo '(c))
foo
yielding
Error: struct:exn:fail:contract
set-mcdr!: contract violation
expected: mpair?
given: '(a b)
argument position: 1st
other arguments...:
'(c)
This error implies that the problem really is the quoted list. I don't have an
easy way to make the big quoted list in eceval into an mlist or chain of
mcons. I tried it and it's very verbose and error prone, plus I think the code
that loads eceval scans and patches that list, so it uses other list operations.
I had to revert after going south in a bad way.
Perhaps I missed some way to automate the transformation, a macro, but that's a
deeper rabbit hole (my scheme macro-fu is too old).
So I'm stuck. Nothing easy or recommended works. I'd like to either (1) know a scheme implementation that will run this code (2) some way I can implement set-car! and set-cdr! in racket (3) some other kind of work around (4) or maybe I just made a stupid mistake that one of you kind people will easily fix.
I tried (by either running racket directly or running it via DrRacket)
#lang sicp
(define foo '(a b))
(set-cdr! foo '(c))
foo
and it outputs (a c).
This also works:
#lang r5rs
(define foo '(a b))
(set-cdr! foo '(c))
(display foo)
To answer your question regarding the implementation of SICP (which I am currently maintaining), you are correct that (prefix-in r5rs: r5rs) will import set-cdr!.
Update: I just installed Geiser and now experienced the same problem that you did when I C-c C-b. However, C-c C-a works as expected.
Personally, I would use racket-mode instead of Geiser.
Mit-scheme will load the eceval compiler from the code drop mentioned in the question. On Ubuntu, mit-scheme loads with sudo apt-install mit-scheme. The geiser package of emacs finds mit via run-mit. The problem is solved.
I am trying to write a small scheme-like language in python, in order to try to better understand scheme.
The problem is that I am stuck on syntax objects. I cannot implement them because I do not really understand what they are for and how they work.
To try to understand them, I played around a bit with syntax objects in DrRacket.
From what I've been able to find, evaluating #'(+ 2 3) is no different from evaluating '(+ 2 3), except in the case that there is a lexical + variable shadowing the one in the top-level namespace, in which case (eval '(+ 2 3)) still returns 5, but (eval #'(+ 2 3)) just throws an error.
For example:
(define (top-sym)
'(+ 2 3))
(define (top-stx)
#'(+ 2 3))
(define (shadow-sym)
(define + *)
'(+ 2 3))
(define (shadow-stx)
(define + *)
#'(+ 2 3))
(eval (top-sym)), (eval (top-stx)), and (eval (shadow-sym)) all return 5, while (eval (shadow-stx)) throws an error. None of them return 6.
If I didn't know better, I would think that the only thing that's special about syntax objects (aside from the trivial fact that they store the location of the code for better error reporting) is that they throw an error under certain circumstances where their symbol counterparts would have returned a potentially unwanted value.
If the story were that simple, there would be no real advantage to using syntax objects over regular lists and symbols.
So my question is: What am I missing about syntax objects that makes them so special?
Syntax objects are the repository for lexical context for the underlying Racket compiler. Concretely, when we enter program like:
#lang racket/base
(* 3 4)
The compiler receives a syntax object representing the entire content of that program. Here's an example to let us see what that syntax object looks like:
#lang racket/base
(define example-program
(open-input-string
"
#lang racket/base
(* 3 4)
"))
(read-accept-reader #t)
(define thingy (read-syntax 'the-test-program example-program))
(print thingy) (newline)
(syntax? thingy)
Note that the * in the program has a compile-time representation as a syntax object within thingy. And at the moment, the * in thingy has no idea where it comes from: it has no binding information yet. It's during the process of expansion, during compilation, that the compiler associates * as a reference to the * of #lang racket/base.
We can see this more easily if we interact with things at compile time. (Note: I am deliberately avoiding talking about eval because I want to avoid mixing up discussion of what happens during compile-time vs. run-time.)
Here is an example to let us inspect more of what these syntax objects do:
#lang racket/base
(require (for-syntax racket/base))
;; This macro is only meant to let us see what the compiler is dealing with
;; at compile time.
(define-syntax (at-compile-time stx)
(syntax-case stx ()
[(_ expr)
(let ()
(define the-expr #'expr)
(printf "I see the expression is: ~s\n" the-expr)
;; Ultimately, as a macro, we must return back a rewrite of
;; the input. Let's just return the expr:
the-expr)]))
(at-compile-time (* 3 4))
We'll use a macro here, at-compile-time, to let us inspect the state of things during compilation. If you run this program in DrRacket, you will see that DrRacket first compiles the program, and then runs it. As it compiles the program, when it sees uses of at-compile-time, the compiler will invoke our macro.
So at compile-time, we'll see something like:
I see the expression is: #<syntax:20:17 (* 3 4)>
Let's revise the program a little bit, and see if we can inspect the identifier-binding of identifiers:
#lang racket/base
(require (for-syntax racket/base))
(define-syntax (at-compile-time stx)
(syntax-case stx ()
[(_ expr)
(let ()
(define the-expr #'expr)
(printf "I see the expression is: ~s\n" the-expr)
(when (identifier? the-expr)
(printf "The identifier binding is: ~s\n" (identifier-binding the-expr)))
the-expr)]))
((at-compile-time *) 3 4)
(let ([* +])
((at-compile-time *) 3 4))
If we run this program in DrRacket, we'll see the following output:
I see the expression is: #<syntax:21:18 *>
The identifier binding is: (#<module-path-index> * #<module-path-index> * 0 0 0)
I see the expression is: #<syntax:24:20 *>
The identifier binding is: lexical
12
7
(By the way: why do we see the output from at-compile-time up front? Because compilation is done entirely before runtime! If we pre-compile the program and save the bytecode by using raco make, we would not see the compiler being invoked when we run the program.)
By the time the compiler reaches the uses of at-compile-time, it knows to associate the appropriate lexical binding information to identifiers. When we inspect the identifier-binding in the first case, the compiler knows that it's associated to a particular module (in this case, #lang racket/base, which is what that module-path-index business is about). But in the second case, it knows that it's a lexical binding: the compiler already walked through the (let ([* +]) ...), and so it knows that uses of * refer back to the binding set up by the let.
The Racket compiler uses syntax objects to communicate that kind of binding information to clients, such as our macros.
Trying to use eval to inspect this sort of stuff is fraught with issues: the binding information in the syntax objects might not be relevant, because by the time we evaluate the syntax objects, their bindings might refer to things that don't exist! That's fundamentally the reason you were seeing errors in your experiments.
Still, here is one example that shows the difference between s-expressions and syntax objects:
#lang racket/base
(module mod1 racket/base
(provide x)
(define x #'(* 3 4)))
(module mod2 racket/base
(define * +) ;; Override!
(provide x)
(define x #'(* 3 4)))
;;;;;;;;;;;;;;;;;;;;;;;;;;;
(require (prefix-in m1: (submod "." mod1))
(prefix-in m2: (submod "." mod2)))
(displayln m1:x)
(displayln (syntax->datum m1:x))
(eval m1:x)
(displayln m2:x)
(displayln (syntax->datum m2:x))
(eval m2:x)
This example is carefully constructed so that the contents of the syntax objects refer only to module-bound things, which will exist at the time we use eval. If we were to change the example slightly,
(module broken-mod2 racket/base
(provide x)
(define x
(let ([* +])
#'(* 3 4))))
then things break horribly when we try to eval the x that comes out of broken-mod2, since the syntax object is referring to a lexical binding that doesn't exist by the time we eval. eval is a difficult beast.
I've been reading through SICP (Structure and Interpration of Computer Programs) and was really excited to discover this wonderful special form: "make-environment", which they demonstrate to use in combination with eval as a way of writing modular code (excerpt from section 4.3 on "packages"):
(define scientific-library
(make-environment
...
(define (square-root x)
...)))
They then demonstrate how it works with
((eval 'square-root scientific-library) 4)
In their example, they then go on to demonstrate exactly the usage that I would want - an elegant, minimalist way of doing the "OO" style in scheme... They "cons" together a "type", which is actually what was returned by the "make-environment" special form (i.e. the vtable), and an arg ("the state")...
I was so excited because this is exactly what I've been looking for as a way to do polymorphic dispatch "by symbol" in Scheme without having to write lots of explicit code or macros.
i.e. I want to create an "object" that has, say, two functions, that I call in different contexts... but I don't want to refer to them by "car" and "cdr", I want to both declare and evaluate them by their symbolic names.
Anyway, when I read this I couldn't wait to get home and try it.
Imagine my disappointment then when I experienced the following in both PLT Scheme and Chez Scheme:
> (make-environment (define x 3))
Error: invalid context for definition (define x 3).
> (make-environment)
Error: variable make-environment is not bound.
What happened to "make-environment" as referenced in SICP? It all seemed so elegant, and exactly what I want, yet it doesn't seem to be supported in any modern Scheme interpreters?
What's the rationale? Is it simply that "make-environment" has a different name?
More information found later
I took at look at the online version:
https://mitp-content-server.mit.edu/books/content/sectbyfn/books_pres_0/6515/sicp.zip/full-text/book/book-Z-H-28.html#%_sec_4.3
I was reading was the first edition of SICP. The second edition appears to have replaced the discussion on packages with a section on non-deterministic programming and the "amp" operator.
After more digging around I discovered this informative thread on newsnet:
"The R5RS EVAL and environment specifiers are a compromise between
those who profoundly dislike first-class environments and want a
restricted EVAL, and those who can not accept/understand EVAL without
a second argument that is an environment."
Also, found this "work-around":
(define-syntax make-environment
(syntax-rules ()
((_ definition ...)
(let ((environment (scheme-report-environment 5)))
(eval '(begin definition
...)
environment)
environment))))
(define arctic
(make-environment
(define animal 'polarbaer)))
(taken from this)
However, I ended up adopting a "message passing" style kinda of like the first guy suggested - I return an alist of functions, and have a generic "send" method for invoking a particular function by name... i.e something like this
(define multiply
(list
(cons 'differentiate (...))
(cons 'evaluate (lambda (args) (apply * args)))))
(define lookup
(lambda (name dict)
(cdr (assoc name dict))))
; Lookup the method on the object and invoke it
(define send
(lambda (method arg args)
((lookup method arg) args)))
((send 'evaluate multiply) args)
I've been reading further and am aware that there's all of CLOS if I really wanted to adopt a fully OO style - but I think even above is somewhat overkill.
They wrote it like that because MIT Scheme does, in fact, have first-class environments, and presumably that's what the writers were planning to teach their class with (since the book was written at MIT).
Check out http://groups.csail.mit.edu/mac/projects/scheme/
However, I've noticed that MIT Scheme, while still somewhat actively developed, lacks many of the features that a really modern Scheme would have, like a foreign function interface or GUI support. You probably wouldn't want to use it for a serious software development project, at least not by itself.
Scheme has no first-class environments because of performance reasons. When Scheme was created, it wasn't the fastest language around due to nifty stuff like first-class functions, continuations, etc. Adding first-class environments would have crippled the performance even further. So it was a trade-off made in the early Scheme days.
Would a classical dispatcher function work? I think this is similar to what you're looking for.
(define (scientific-library f)
(define (scientific-square-root x) (some-scientific-square-root x))
(cond ((eq? f 'square-root) scientific-square-root)
(else (error "no such function" f))))
(define (fast-library f)
(define (fast-square-root x) (some-fast-square-root x))
(cond ((eq? f 'square-root) fast-square-root)
(else (error "no such function" f))))
((scientific-library 'square-root) 23)
((fast-library 'square-root) 23)
You could even combine the example scientific and fast libraries into one big dispatch method:
(define (library l f)
(define (scientific-library f)
...)
(define (fast-library f)
...)
(cond ((eq? l 'scientific) (scientific-library f))
((eq? l 'fast) (fast-library f))
(else (error "no such library" l))))
(library 'fast 'square-root)