The book Lisp in Small Pieces demonstrates a transformation from Scheme into continuation passing style (chapter 5.9.1, for those who have access to the book). The transformation represents continuations by lambda forms and call/cc is supposed to become equivalent to a simple (lambda (k f) (f k k)).
I do not understand how this can work because there is no distinction between application of functions and continuations.
Here is a version of the transformation stripped from everything except application (the full version can be found in this gist):
(define (cps e)
(if (pair? e)
(case (car e)
; ...
(else (cps-application e)))
(lambda (k) (k `,e))))
(define (cps-application e)
(lambda (k)
((cps-terms e)
(lambda (t*)
(let ((d (gensym)))
`(,(car t*) (lambda (,d) ,(k d))
. ,(cdr t*)))))))
(define (cps-terms e*)
(if (pair? e*)
(lambda (k)
((cps (car e*))
(lambda (a)
((cps-terms (cdr e*))
(lambda (a*)
(k (cons a a*)))))))
(lambda (k) (k '()))))
Now consider the CPS example from Wikipedia:
(define (f return)
(return 2)
3)
Above transformation would convert the application in the function body (return 2) to something like (return (lambda (g13) ...) 2). A continuation is passed as the first argument and the value 2 as the second argument. This would be fine if return was an ordinary function. However, return is supposed to be a continuation, which only takes a single argument.
I don't see how the pieces fit together. How can the transformation represent continuations as lambda forms but not give special consideration to their application?
I do not understand how this can work because there is no distinction between application of functions and continuations.
Implementing continuations without CPS requires approaches at the virtual machine level, such as using "spaghetti stacks": allocating lexical variables in heap-allocated frames that are subject to garbage collection. Capturing a continuation then just means obtaining an environment pointer which refers to a lexical frame in the spaghetti stack.
CPS builds a de facto spaghetti stack out of closures. A closure captures lexical bindings into an object with an indefinite lifetime. Under CPS, all closures capture the hidden variable k. That k serves the role of the parent frame pointer in the spaghetti stack; it chains the closures together.
Because the whole program is consistently CPS-transformed, there is a k parameter everywhere which points to a dynamically linked chain of closed-over environments that amounts to a de facto stack where execution can be restored.
The one missing piece of the puzzle is that CPS depends on tail calls. Tail calls ensure that we are not using the real stack; everything interesting is in the closed-over environments.
(However, even tail calls are not strictly required, as Henry Baker's approach, embodied in Chicken Scheme, teaches us. Our CPS-transformed code can use real calls that consume stack, but never return. Every once in a while we can move the reachable environment frames (and all contingent objects) from the stack into the heap, and rewind the stack pointer.)
Now consider the CPS example from Wikipedia:
Ah, but that's not a CPS example; that's an example of application code that uses continuations that are available somehow via call/cc.
It becomes CPS if either we transform it to CPS by hand, or use a compiler which does that mechanically.
However, return is supposed to be a continuation, which only takes a single argument.
Thus, return only takes a single argument because we're looking at application source code that hasn't been CPS-transformed.
The application-level continuations take one argument.
The CPS-implementation-level continuations will have the hidden k argument, like all functions.
The k parameter is analogous to a piece of machine context, like a stack or frame pointer. When using a conventional language, and call print("hello"), you don't ask, how come there is only one argument? Doesn't print have to receive the stack pointer so it knows where the parameters are? Of course when the print is compiled, the compiled code has a way of conveying that context from one function to another, invisible to the high level language.
In the case of CPS in Scheme, it's easy to get confused because the source and target language are both Scheme.
Related
I am trying to understand continuation passing style conversion.
I am trying to build a Scheme to C compiler and I want to use continuation passing style. Whether or not continuation passing style is the right way to do this can you guys explain me the conversion rules?
For example, here is a presentation by Marc Feeley, the creator of Gambit Scheme.
I will summarize the continuation passing style rules he gives, but note: I don't understand them. In particular I don't understand what C is.
Here is the notation:
[E]
C
which denotes the continuation passing style conversion of the expression E in the continuation C.
So, for example, one conversion rule is this one:
[c]
C
==>
(C c) ;; application of C to c
which denotes the CPS conversion of the constant c in the continuation C.
What is C? I am having trouble understanding what C is. It is like magic.
Another rule is:
[(if E1 E2 E3)]
C
==>
[E1]
(lambda (r1)
(if r1 [E2] [E3]))
C C
where E1 gets passed to r1.
But what is C?
Can you guys please explain?
Thanks.
If you scroll higher up in the article, to page 7, you will find definitions of what a continuation is, which is necessary to understand the rules for converting to continuation-passing style. An example given is
> (sqrt (+ (read) 1))
and it notes that the continuation for (read) is
a computation that takes a value, adds 1 to it, computes
its square-root, prints the result and goes to the next REPL interaction.
So the continuation C for an expression E is "whatever happens to the value of this expression". This is reiterated on page 20, with the example
(let ((square (lambda (x) (* x x))))
(write (+ (square 10) 1)))
where the continuation of (square 10) is
(lambda (r) (write (+ r 1)))
So as you are recursively translating the program to CPS style, the continuation C will grow as you get deeper into the expression. Note that each of the translation rules [E]|C results in a smaller un-translated E, perhaps empty if E was simple enough, and a larger translated C.
When you convert a code in CPS you practically introduce a strict discipline in evaluation.
When you write (+ x y z), it is unclear the order in which you evaluate each of +, x, y, z. If the language you write in explicitly defines an order, you know what happens. But if the language does not insert an order, you can define the order you wish by writing in/converting the code into CPS, in the example I proposed you would write:
(eval + (lambda (+)
(eval x (lambda(x)
(eval y (lambda (y)
(eval z (lambda (z)
(+ x y z))))
if you want a left-right evaluation.
If you write your code in CPS, this is like writing code in assembler, as each piece of code can be associated with an instruction that has a corresponding in very low programming. If you convert some code in CPS, you need to uniquely rename variables to avoid collisions. At the time the C language was created, I think it was not clearly defined the CPS transform; this is why the inline specifier rejects recursive calls. It is possible to convert a recursive function into a goto-loop by rewriting the C code and using the CPS transform, but standard C does not want to.
The ways to convert the code into CPS are many. In Mit-scheme for example, the input code is not explicitly rewritten in CPS form, but the evaluation process uses a combination of goto statements and trampoline calls to simulate it (this is a way you won's learn in school about, but it is used in practice).
The recursive CPS code can be converted directly into iterative loops (this is why scheme->C translators do the conversion first) to solve the tail recursion. The first edition of EPL of Dan Friedman details it. There is also an article of Friedman on this. If you cannot find it, I will try to find it for you.
Can any case of using call/cc be rewritten equivalently without using it?
For example
In (g (call/cc f)), is the purpose of f to evaluate the value of
some expression, so that g can be applied to the value?
Is (g (call/cc f)) always able to be rewritten equivalently
without call/cc e.g. (g expression)?
In ((call/cc f) arg), is the purpose of f to evaluate the
definition of some function g, so that function g can be
applied to the value of arg?
Is ((call/cc f) arg) always able to be rewritten equivalently
without call/cc e.g. (g arg)?
If the answers are yes, why do we need to use call/cc?
I am trying to understand the purpose of using call/cc, by contrasting it to not using it.
The key to the direct answer here is the notion of "Turing equivalence". That is, essentially all of the commonly used programming languages (C, Java, Scheme, Haskell, Lambda Calculus etc. etc.) are equivalent in the sense that for any program in one of these languages, there is a corresponding program in each of the other languages which has the same meaning.
Beyond this, though, some of these equivalences may be "nice" and some may be really horrible. This suggests that we reframe the question: which features can be rewritten in a "nice" way into languages without that feature, and which cannot?
A formal treatment of this comes from Matthias Felleisen, in his 1991 paper "On the Expressive Power of Programming Languages" (https://www.sciencedirect.com/science/article/pii/016764239190036W), which introduces a notion of macro expressibility, pointing out that some features can be rewritten in a local way, and some require global rewrites.
The answer to your original question is obviously yes. Scheme is Turing-complete, with or without call/cc, so even without call/cc, you can still compute anything that is computable.
Why "it is more convenient than writing the equivalent expression using lambda"?
The classic paper On the Expressive Power of Programming Languages by Matthias Felleisen gives one answer to this question. Pretty much, to rewrite a program with call/cc to one without it, you might potentially need to transform your whole program (global transformation). This is to contrast some other constructs that only need a local transformation (i.e., can be written as macro) to remove them.
The key is: If your program is written in continuation passing style, you don't need call/cc. If not, good luck.
I whole-heartedly recommend:
Daniel P. Friedman. "Applications of Continuations: Invited Tutorial". 1988 Principles of Programming Languages (POPL88). January 1988
https://cs.indiana.edu/~dfried/appcont.pdf
If you enjoy reading that paper, then check out:
https://github.com/scheme-live/bibliography/blob/master/page6.md
Of course anything that is written with call/cc can be written without it, because everything in Scheme is ultimately written using lambda. You use call/cc because it is more convenient than writing the equivalent expression using lambda.
There are two senses to this question: an uninteresting one and an interesting one:
The uninteresting one. Is there some computation that you can do with call/cc that you can't do in a language which does not have it?
No, there isn't: call/cc doesn't make a language properly more powerful: it is famously the case that a language with only λ and function application is equivalent to a universal Turing machine, and thus there is no (known...) more powerful computational system.
But that's kind of uninteresting from the point of view of programming-language design: subject to the normal constraints on memory &c, pretty much all programming languages are equivalent to UTMs, but people still prefer to use languages which don't involve punching holes in paper tape if they can.
The interesting one. Is it the case that call/cc makes some desirable features of a programming language easier to express?
The answer to this is yes, it does. I'll just give a couple of examples. Let's say you want to have some kind of non-local exit feature in your language, so some deeply-nested bit of program can just say 'to hell with this I want out', without having to climb back out through some great layer of functions. This is trivial with call/cc: the continuation procedure is the escape procedure. You can wrap it in some syntax if you want it to be nicer:
(define-syntax with-escape
(syntax-rules ()
[(_ (e) form ...)
(call/cc (λ (e) form ...))]))
(with-escape (e)
... code in here, and can call e to escape, and return some values ...)
Can you implement this without call/cc? Well, yes, but not without either relying on some other special construct (say block and return-from in CL), or without turning the language inside out in some way.
And you can build on things like this to implement all sorts of non-local escapes.
Or, well, let's say you want GO TO (the following example is Racket):
(define (test n)
(define m 0)
(define start (call/cc (λ (c) c)))
(printf "here ~A~%" m)
(set! m (+ m 1))
(when (< m n)
(start start)))
Or, with some syntax around this:
(define-syntax-rule (label place)
(define place (call/cc identity)))
(define (go place)
(place place))
(define (horrid n)
(define m 0)
(label start)
(printf "here ~A~%" m)
(set! m (+ m 1))
(when (< m n)
(go start)))
So, OK, this perhaps is not a desirable feature of a programming language. But, well, Scheme doesn't have GO TO right, and yet, here, it does.
So, yes, call/cc (especially when combined with macros) makes a lot of desirable features of a programming language possible to express. Other languages have all these special-purpose, limited hacks, Scheme has this universal thing from which all these special-purpose hacks can be built.
The problem is that call/cc doesn't stop with the good special-purpose hacks: you can also build all the awful horrors that used to blight programming languages out of it. call/cc is like having access to an elder god: it's really convenient if you want dread power, but you'd better be careful what comes with it when you call, because it may well be an unspeakable horror from beyond spacetime.
An easy use of call/cc is as a bail out. eg.
;; (1 2) => (2 4)
;; #f if one element is not a number
(define (double-numbers lst)
(call/cc
(lambda (exit)
(let helper ((lst lst))
(cond ((null? lst) '())
((not (number? (car lst))) (exit #f))
(else (cons (* 2 (car lst)) (helper (cdr lst)))))))))
So to understand this. If we are doing (double-numbers '(1 2 r)) the result is #f, but the helper has done (cons 1 (cons 2 (exit #f)))
Without call/cc we see the continuation would be whatever called double-numbers since it actually return normally from it. Here is an example without call/cc:
;; (1 2) => (2 4)
;; #f if one element is not a number
(define (double-numbers lst)
(define (helper& lst cont)
(cond ((null? lst) (cont '()))
((not (number? (car lst))) #f) ; bail out, not using cont
(else (helper& (cdr lst)
(lambda (result)
(cont (cons (* 2 (car lst)) result)))))))
(helper& lst values)) ; values works as an identity procedure
I imagine it gets harder pretty quick. Eg. my generator implementation. The generator relies on having access to continuations to mix the generator code with where it's used, but without call/cc you'll need to do CPS in both the generator, the generated generator and the code that uses it.
There is something I really don't understand about the encapsulation and the closure. I believe that the encapsulation is what can not be changed unless it is changed by the code. But I can't really understand when I am asked to explain how the closure and the encapsulation are applied on a code.
For example :
(define new-cercle #f)
(let ((n 0))
(set! new-cercle
(lambda (rayon)
(begin
(set! n (+ n 1))
(lambda (msg)
(cond ((eq? msg ’circonference)
(* 2 3.14 rayon))
((eq? msg ’surface)
(* 3.14 rayon rayon))
((eq? msg ’nb-cercles)
n)))))))
The n is encapsulated, right? So the question is: explain how the encapsulation and the closure are applied on this code.
Another thing I do not understand is why the let has to be above the lambda here? Why when I put it below the lambda, the function doesn't work well and there is no accumulator ?
(define acc
(let ((n 1))
(lambda (x)
(set! n (* n x))
n)))
I hope someone would explain to me this in an easy way because when I google'd it, honestly I didn't understand anything with the complicated examples most topics have.
Encapsulation the name of a pattern involving any situation in which some related items are put together into a container and then travel with that container, and are referenced through some access mechanism on that container. The items could be run-time values, or compile-time identifiers or whatever. An object made up of multiple fields encapsulates fields: a cons cell encapsulates car and cdr. A class encapsulates slots. In some object systems, methods also. Compilation units encapsulate their global definitions such as functions and variables.
The popular use of "encapsulate" in OOP refers to defining a class as a unit which contains the definition of data, together with the methods which operate on it: the code and data are one "capsule". (The Common Lisp object system isn't like this: methods are not encapsulated in classes.)
A closure is something else, something very specific: it is a body of program code, together with its lexical environment, reified into an object of function type. The body of the closure, when invoked, has visibility to two groups of names: the closure's function parameters, and the surrounding names in the lexical scope where the closure is created. A closure is an example of encapsulation: it encapsulates the body of code together with the lexical scope. The only means of access into the capsule is through the function: the function is like a "method", and the elements of the captured lexical environment are like "slots" in an object.
(By combining code and data, Lisp closures resemble the popular notion of encapsulation more than do Lisp class objects.)
About that funny word: in computer science, to "reify" some aspect of a program is to take something which is not a first class object, and somehow turn it into one.
Almost any recognizable concept which is applicable to the understanding of a program can potentially be reified. (Someone clever just has to come up with a sensible proposal about how.)
For instance, the entire future computation at a given point of execution can be reified, and the resulting object is called a continuation (and undelimited continuation, more precisely).
When the continuation operator captures a future computation, that future becomes hypothetical: it doesn't actually happen (doesn't execute). Instead, an alternative future executes in which the continuation is returned to the operator's caller, or passed into a function which the caller designates. The code which now has this continuation in its grasp can use it to explicitly invoke the original, captured future, as if it were a function. Or choose not to do that. In other words, program control flow (execute this block or don't execute this, or execute it several times) has become a function object (call this function or don't call it, or call it several times).
Objects are another example of reification: the reification of modules. Old-fashioned programs are divided into modules which have global functions and global variables. This "module" structure is a concept that we can recognize in a program and usefully apply in describing such programs. It is susceptible to reification: we can imagine, what if we had a run-time object which is "module", having all those same attributes: namely containing functions and data? And, presto: object-based programming is born, with advantages like multiple instantiation of the same module, possible because the variables are no longer global.
About cercle and rayon:
Firstly, new-cercle behaves like a constructor for objects: it is a global function that can be called from anywhere. It maintains a count of how many objects have been constructed. Only that function can access the counter, so it is encapsulated. (Actually not only that function can access it, but also the closures representing the circle instances!) This is an example of module-like encapsulation. It simulates modules, like in the language Modula-2 and similar, such as C language translation units with static variables at file scope.
When we call new-cercle we must supply an argument for the rayon parameter. An object is produced and returned. That object happens to be a function produced as a lexical closure. This closure had captured the rayon parameter, thereby encapsulating this value: the object knows its own radius. We can call new-cercle repeatedly, and obtain different instances of circles, each carrying its own rayon. This rayon is not externally visible; it is packaged inside the closure, and is only visible to that function.
We gain access into the rayon-container indirectly, through a "message" API on the container. We can call the function with the message symbol surface, and it replies by returning the surface area. None of the currently available messages reveals rayon directly, but we could provide an accessor message for that, and even a message to change the radius. There is even a message to access the shared variable n, the count of circles, which behaves like a class variable in an object system (static slot): any instance of a circle can report how many circles have been constructed. (Note that this count doens't inform us how many circles currently exist: it doesn't decrement when a circle becomes garbage and is reclaimed: there is no finalization).
In any case, we clearly have a container whose contents are not accessible except through an interface. That container binds together code and data, so it is not only encapsulation, but arguably encapsulation in the popular OOP sense.
You're perhaps having difficulties because in the trivial case, some differences go away. For example both (let ((n 1)) (lambda (x) n)) and (lambda (x) (let ((n 1)) n) give you basically the same function.
In your example
(define acc (let ((n 1))
(lambda (x) (set! n (* n x)) n)))
the ordering of let and lambda is significant. If you interchange them to (lambda (x) (let ((n 1)) ... then every time you call this function n will again be bound to 1. Instead you want there to be some location n that starts out with value 1 and can be modified by your function and does not go away when your function is done, which is what you get when you have (let ((n 1)) (lambda (x) (set! n ....
The function constructed by the inner lambda captures the use of the outer n and holds on to its location for as long as it itself lives. It also encapsulates n as nothing else can refer to it but this function. We also say that the function is closed by the surrounding binding of n, and that the function is a closure (of n).
Reading about lexical scope might help you as well.
I kinda understand how to convert elementary functions such as arithmetics to continuation-passing style in Scheme.
But what if the function involves recursion?
For example,
(define funname
(lambda (arg0 arg1)
(and (some procedure)
(funname (- arg0 1) arg1))))
Please give me advices.
Thank you in advance.
One place that has a good explanation on continuations and CPS is Krishnamurthi's PLAI book. The relevant part (VII) doesn't depend on other parts of the book so you can jump right in there. There is specifically an extended example of converting code to CPS manually, and tackling recursive functions (the first part of chapter 17).
In addition, I wrote an extended version of that text for my class, which has more examples and more details on the subject -- you might find that useful too. In addition to the PLAI text, I cover some common uses of continuations like implementing generators, the ambiguous operator and more. (But note that PLAI continues with a discussion of implementation strategies, which my text doesn't cover.)
(define (func x y k)
(some-procedure
(lambda (ret)
(if ret
(- x 1
(lambda (ret)
(func ret y k)))
(k #f))))
You are lacking a base case, which is why the only explicit call to the continuation is (k #f). If you have a base case, then you'd pass the base case return value to the continuation, also. For example:
(define (func x y k)
(zero? x
(lambda (ret)
(if ret
(k y)
(some-procedure
(lambda (ret)
(if ret
(- x 1
(lambda (ret)
(func ret y k)))
(k #f))))))))
This partly duplicates Chris Jester-Young's answer, but well, I hope I can explain it better :-).
In CPS, the difference you're seeking is between these two things (roughly):
You can invoke a procedure, and pass it the continuation you were passed. That's the equivalent of a direct-style optimized tail call.
Or, you can invoke a procedure, and pass in as its continuation a new procedure that does something with the "return value," passing in your original continuation. This is the equivalent of a direct-style stack call.
The latter is what the lambdas in Chris's example are doing. Basically, evaluating a lambda creates a closure—and these closures are used to do the same job that stack frames do in the execution of a a direct-style program. In place of the return address in a stack frame, the closure contains a binding for a continuation function, and the code for the closure invokes this.
I want to filter out all elements of list 'a from list 'b and return the filtered 'b. This is my function:
(defun filter (a b)
"Filters out all items in a from b"
(if (= 0 (length a)) b
(filter (remove (first a) a) (remove (first a) b))))
I'm new to lisp and don't know how 'remove does its thing, what kind of time will this filter run in?
There are two ways to find out:
you could test it with data
you could analyze your source code
Let's look at the source code.
lists are built of linked cons cells
length needs to walk once through a list
for EVERY recursive call of FILTER you compute the length of a. BAD!
(Use ENDP instead.)
REMOVE needs to walk once through a list
for every recursive call you compute REMOVE twice: BAD!
(Instead of using REMOVE on a, recurse with the REST.)
the call to FILTER will not necessarily be an optimized tail call.
In some implementations it might, in some you need to tell the compiler
that you want to optimize for tail calls, in some implementations
no tail call optimization is available. If not, then you get a stack
overflow on long enough lists.
(Use looping constructs like DO, DOLIST, DOTIMES, LOOP, REDUCE, MAPC, MAPL, MAPCAR, MAPLIST, MAPCAN, or MAPCON instead of recursion, when applicable.)
Summary: that's very naive code with poor performance.
Common Lisp provides this built in: SET-DIFFERENCE should do what you want.
http://www.lispworks.com/documentation/HyperSpec/Body/f_set_di.htm#set-difference
Common Lisp does not support tail-call optimization (as per the standard) and you might just run out of memory with an abysmal call-stack (depending on the implementation).
I would not write this function, becuase, as Rainer Joswig says, the standard already provides SET-DIFFERENCE. Nonetheless, if I had to provide an implementation of the function, this is the one I would use:
(defun filter (a b)
(let ((table (make-hash-table)))
(map 'nil (lambda (e) (setf (gethash e table) t)) a)
(remove-if (lambda (e) (gethash e table)) b)))
Doing it this way provides a couple of advantages, the most important one being that it only traverses b once; using a hash table to keep track of what elements are in a is likely to perform much better if a is long.
Also, using the generic sequence functions like MAP and REMOVE-IF mean that this function can be used with strings and vectors as well as lists, which is an advantage even over the standard SET-DIFFERENCE function. The main downside of this approach is if you want extend the function with a :TEST argument that allows the user to provide an equality predicate other than the default EQL, since CL hash-tables only work with a small number of pre-defined equality predicates (EQ, EQL, EQUAL and EQUALP to be precise).
(defun filter (a b)
"Filters out all items in a from b"
(if (not (consp a)) b
(filter (rest a) (rest b))))