I have see the following code... The first call of (next-num) returns 1, and the second returns 2.
(define next-num
(let ((num 0))
(lambda () (set! num (+ num 1)) num)))
(next-num) ; 1
(next-num) ; 2
What I can not understand is... num is created by let inside next-num, it is kind of a local variable... How does scheme know that each time next-num is called, the value of num is not erased by let ((num 0)); How does scheme know that it is always the same num that we modify whenever next-num is called?
It seems that num is both local and static... How can we define a local variable, but not static?
This is "lexical closure" and you're right that num, the "closed-over variable" is similar to a static variable, in C for example: it's only visible to code within the let form (its "lexical scope"), but it persists across the whole program run, rather than being re-initialized with each call to the function.
I think the part you're confused on is this: "num is created by let inside next-num, it is kind of a local variable". This isn't true because the let block isn't part of the next-num function: it's actually an expression which creates and returns the function which is then bound to next-num. (This is very different, e.g., from C, where functions can only be created at compile-time and by defining them at top-level. In Scheme, functions are values like integers or lists, which any expression can return).
Here's another way to write (almost) the same thing which makes it clearer that the define is just associating next-num to the value of a function-returning expression:
(define next-num #f) ; dummy value
(let ((num 0))
(set! next-num
(lambda () (set! num (+ num 1)) num)))
It's important to note the difference between
(define (some-var args ...) expression expression ...)
which makes some-var a function which executes all the expressions when called, and
(define some-var expression)
which binds some-var to the value of expression, evaluated then and there. Strictly speaking, the former version is unnecessary, because it's equivalent to
(define some-var
(lambda (args ...) expression expression ...))
Your code is almost the same as this, with the addition of the lexically scoped variable, num, around the lambda form.
Finally, here's a key difference between closed-over variables and static variables, which makes closures much more powerful. If you had written the following instead:
(define make-next-num
(lambda (num)
(lambda () (set! num (+ num 1)) num)))
then each call to make-next-num would create an anonymous function with a new, distinct num variable, which is private to that function:
(define f (make-next-num 7))
(define g (make-next-num 2))
(f) ; => 8
(g) ; => 3
(f) ; => 9
This is a really cool and powerful trick which accounts for a lot of the power of languages with lexical closures.
Edited to add: You ask how Scheme "knows" which num to modify when next-num is called. In outline, if not in implementation, this is actually pretty simple. Every expression in Scheme is evaluated in the context of an environment (a lookup table) of variable bindings, which are associations of names to places which can hold values. Each evaluation of a let form or a function call creates a new environment by extending the current environment with new bindings. To arrange to have lambda forms behave as closures, the implementation represents them as a structure consisting of the function itself plus the environment in which it was defined. Calls to that function are then evaluated by extending the binding environment in which the function was defined -- not the environment in which it was called.
Older Lisps (including Emacs Lisp until recently) had lambda, but not lexical scope, so although you could create anonymous functions, calls to them would be evaluated in the calling environment rather than the definition environment, and so there were no closures. I believe Scheme was the first language to get this right. Sussman and Steele's original Lambda Papers on the implementation of Scheme make great mind-expanding reading for anyone who wants to understand scoping, among many other things.
Related
Is there normal "set" function (not special form) in Scheme language or some way to implement it?
I'd like to write code something like:
(map (lambda (var)
(set var 0))
'(a b c))
which could assign a value (here it is '0') to variables from list (here they are 'a', 'b' and 'c').
No. And to see why there isn't consider something like this:
(define (mutant var val)
(let ((x 1))
(set var val)
x))
Now, what should (mutant 'x 3) return? If it should return 3, then:
set can't be a function since it needs access to the lexical environment of mutant;
any kind of reasonable compilation of this function is not possible.
If you want set to be a function then catastrophe follows. Consider this definition:
(define (mutant-horror f)
(let ([x 3])
(f)
x))
Now, you would think that this can be optimised to this:
(define (mutant-horror f)
(f)
3)
But it can't. Because you might call it like this:
(mutant-horror (λ () (set 'x 3)))
or, more generally, you might call it with a function which, somewhere eventually in some function called indirectly from it might end up saying (set 'x 3).
This means that no binding can ever be optimised at all, which is a disaster. It's also at least very close to meaning lexical scope is not possible: if as well as set, a function called get exists, which retrieves the binding of a symbol, then you have, essentially, dynamic scope. That in turn makes things like tail-call elimination at least difficult and probably impossible (in fact set probably does this on its own).
Reasons like this are why even very old Lisps, where things like set did exist and did superficially work, actually made special exemptions for compiled code, where set didn't work (see for instance the Lisp 1.5 programmer's manual (PDF link), appendix D. This divergence between the semantics of compiled and interpreted code is one of the things that later Lisps and Lisp-related languages such as CL and Scheme did away with.
If instead you want something like Common Lisp's semantics, where the equivalent thing
(defun mutant (var val)
(let ((x 1))
(set var val)
x))
Would return 1 (unless x was a globally (see below) special variable, in which case it might return something else) and as a side-effect modify the value cell of whatever symbol was named by var (which might be x), then, well, Scheme has no notion of that at all, and that's a good thing on the whole.
Note that a modified version of the function will also work for locally special variables:
(defun mutant/local-special (a b)
(let ((x 1))
(declare (special x))
(set a b)
x))
But in this case you always know there's a special binding happening because you can always see the declaration.
When you write something like
(map (lambda (var)
(set var 0))
'(a b c))
my first thought was that you try to accumulate unordered sets of the form ( (a 0) (b 0) (c 0) ).
You cannot implement your own setter for any of the internal data structures that are provided by the language as this would mean to write a scheme function to modify some data structures that is implemented in C. For the data structures implemented in C you need to provide setters written in C -- supposing the lower language is C.
If you want to implement your own setter you either
-- check how the data structure in implmented, and if it's implemented in scheme you will undestand how to modify it
-- define your own data structure using already existing data structures and define setters for it.
A setter that mutates the data structure contains a ! at the end of its name, such as set!, append!, etc.
A function call just evaluates a series of instructions in an extended environment, the environment in particular is that of definition extended with its function parameters, this is prety much the case in any language...
If you do this:
(define (my-set var val)
(set! var val))
you will bind the value of val to var, but only within the scope of the current call on my-set. The reason you cannot write such function has to do with the nature of scheme itself, var is a pointer to whatever you pass in the function, but set! will make this pointer point to something else (still within the scope of my-set). my-set could work if we had some sort of mechanism of using actual pointers, as some low lever languages allow. But scheme does not...
Note that scheme goes very well with the functional programming style as well as recursion, so if you have a need for a function as you described, you are probably "doing something wrong"... :)
You can, however, do this:
(define my-list (list 1 2 3))
(define (my-set a-list a-value)
(set-car! my-list a-value))
> (my-set my-list 4)
> my-list
(4 2 3)
this works since a-list is a pointer to a cons-cell, set-car! will modify the contents of a cons-cell, but not affect the pointer to which.
I'm having a hard time understanding the syntax of let vs some of the other statements. For example, a "normal" statement has one parentheses:
(+ 2 2)
$2 = 4
Yet the let statement has two:
(let ((x 2)) (+ x 2))
$3 = 4
Why is this so? I find it quite confusing to remember how many parentheses to put around various items.
Firstly, note that let syntax contains two parts, both of which can have zero or more elements. It binds zero or more variables, and evaluates zero or more forms.
All such Lisp forms create a problem: if the elements are represented as a flat list, there is an ambiguity: we don't know where one list ends and the other begins!
(let <var0> <var1> ... <form0> <form1> ...)
For instance, suppose we had this:
(let (a 1) (b 2) (print a) (list b))
What is (print a): is that the variable print being bound to a? Or is it form0 to be evaluated?
Therefore, Lisp constructs like this are almost always designed in such a way that one of the two lists is a single object, or possibly both. In other words: one of these possibilities:
(let <var0> <var1> ... (<form0> <form1> ...))
(let (<var0> <var1> ...) (<form0> <form1> ...))
(let (<var0> <var1> ...) <form0> <form1> ...)
Traditional Lisp has followed the third idea above in the design of let. That idea has the benefit that the pieces of the form are easily and efficiently accessed in an interpreter, compiler or any code that processes code. Given an object L representing let syntax, the variables are easily retrieved as (cadr L) and the body forms as (cddr L).
Now, within this design choice, there is still a bit of design freedom. The variables could follow a structure similar to a property list:
(let (a 1 b 2 c 3) ...)
or they could be enclosed:
(let ((a 1) (b 2) (c 3)) ...)
The second form is traditional. In the Arc dialect of Lisp designed Paul Graham, the former syntax appears.
The traditional form has more parentheses. However, it allows the initialization forms to be omitted: So that is to say if the initial value of a variable is desired to be nil, instead of writing (a nil), you can just write a:
;; These two are equivalent:
(let ((a nil) (b nil) (c)) ...)
(let (a b c) ...)
This is a useful shorthand in the context of a traditional Lisp which uses the symbol nil for the Boolean false and for the empty list. We have compactly defined three variables that are either empty lists or false Booleans by default.
Basically, we can regard the traditional let as being primarily designed to bind a simple list of variables as in (let (a b c) ...) which default to nil. Then, this syntax is extended to support initial values, by optionally replacing a variable var with a (var init) pair, where init is an expression evaluated to specify its initial value.
In any case, thanks to macros, you can have any binding syntax you want. In more than one program I have seen a let1 macro which binds just one variable, and has no parentheses. It is used like this:
(let1 x 2 (+ x 2)) -> 4
In Common Lisp, we can define let1 very easily like this:
(defmacro let1 (var init &rest body)
`(let ((,var ,init)) ,#body))
If we restrict let1 to have a one-form body, we can then write the expression with obsessively few parentheses;
(let1 x 2 + x 2) -> 4
That one is:
(defmacro let1 (var init &rest form)
`(let ((,var ,init)) (,#form)))
Remember that let allows you to bind multiple variables. Each variable binding is of the form (variable value), and you collect all the bindings into a list. So the general form looks like
(let ((var1 value1)
(var2 value2)
(var3 value3)
...)
body)
That's why there are two parentheses around x 2 -- the inner parentheses are for that specific binding, the outer parentheses are for the list of all bindings. It's only confusing because you're only binding one variable, it becomes clearer with multiple variables.
When I evaluate the following expression every time I get the value 10.
(((lambda (x) (lambda () (set! x (+ x 10)) x)) 0))
However I just modify by abstracting the above procedure with a name and call foo every time the value increments by 10!!
(define foo ((lambda (x) (lambda () (set! x (+ x 10)) x)) 0))
Can anybody please explain this?
The function you are calling is a counter that returns a number 10 higher every time it's called.
In the first case, every time, you are creating a new function and then immediately calling it once and then discarding the function. So every time, you are calling a new instance of this counter for the first time, so it should return 10.
In the second case, you create the function once and assign it to a variable and call that same function repeatedly. Since you are calling the same function, it should return 10, 20, ...
newacct is correct, but I would like to go into (a lot) more detail, since this is something that just blew my mind pretty recently.
I'm going to use the terms 'environment' and 'scope' pretty loosely and to mean essentially the same thing. Remember that scheme is a lexical scope language.
When scheme evaluates an expression it will look in its current environment for the values of any variables in the expression. If it doesn't find any in the current environment, it will look in the parent environment. If the value is not in the parent environment then it will look in the next level up and so on until it reaches the top (global) level where it will either find the value or throw an "unbound variable" error.
Anytime you call define you associate a symbol with a value on that environments symbol table. So if you call define on the top-level an entry will be added to the global symbol table. If you call define in the body of a procedure, then an entry will be added to the symbol table of that procedure.
A good way to think about calling define on a procedure is that you are creating an entry in the symbol table that consists of the parameters, body, and environment of that procedure. For example the procedure square would have an entry something like this:
(define a 3)
(define (square x)
(* x x))
GLOBAL
=================
a-|-3
|
square-|-{x}
| {(* x x)}
| {GLOBAL} ---> All the things defined on the global table
Then if I were to call (square a) the interpreter would first look in the environment in which square is defined and it would find that a is associated with the value 3. Then x -> 3 within the body of square and the procedure returns 9. Cool, makes sense.
Things get a little screwier when we start defining helper procedures within procedures, but all you really need to remember is that if it can't find anything associated with a symbol in the current environment, it will move up levels of scope until it does. Also, it will always stop on the first 'match'. So if there is a local x it will prefer it over the global x (rather it will use the local x without ever looking for a global one).
Next, remember that define just adds names to the symbol table, but set! is a mutator that actually changes the values with which a symbol is associated.
So (define b "blah") puts an entry in the symbol table. b => "blah". Nothing crazy. set! will change the actual value:
(set! b "foo")
b => "foo"
but set! can't add anything to the table. (set! c "bar") => UNBOUND VARIABLE C.
This is the most important difference: set! acts like any other procedure in that if it doesn't find the variable in the current scope, it will check progressively higher levels until it finds a match (or throws an error), but define always adds a binding to the scope in which it is called.
Alright, so you understand the difference between define and set!. Good. Now on to the question.
The expression (((lambda (x) (lambda () (set! x (+ x 10)) x)) 0)), as newacct pointed out, is going to return the same value each time because you are calling a new procedure each time. However if you name it you can keep track of the environment created by calling the procedure.
(define foo <--- associated name on the symbol table
(lambda (x) <--- scope where x is defined
(lambda () \
(set! x (+ x 10)) |--- body
x)) /
0) <--- initial value of x
So the inner lambda exists inside the environment created by the first one where the symbol x exists at an initial value of 0. Then set! looks for an entry in the symbol table for x and finds one in the next level up. Once it finds the entry it changes it, in this case adding 10 to whatever value it finds there. The really cool part is that since you associated the whole thing to a name in the global symbol table, that environment continues to exist after each call! This is why we can do cool things like implement message passing objects to keep track of and manipulate data!
Also, the let special form was created for this purpose, and may be a more intuitive way to structure this. It would look like this:
(define foo <--- associated name
(let ((x 0)) <--- scope where x is defined & initial x value
(lambda () \
(set! x (+ x 10)) |--- body
x))) /
For example,
I want to check if an element is in a list. The algorithm is straightforward, let's do it in C++
bool element_of( const std::vector<int>& lst, int elem ) {
for( int i( 0 ), ie = lst.size(); i < ie; ++i )
if( elem == lst[i] )
return true;
return false;
}
Since Scheme don't let me use single if statement, I can't do something similar to the C++ code above. Then I came up with a temporary variable, namely result. result will have initial value of #f, next I recursively call the function to check the next item in the list i.e. cdr lst ... So my question is, does the variable which created with let restore its initial value each time it enters a new function call or its value stays the same until the last call?
On the other hand, using fold, my solution was,
(define (element-of x lst)
(fold (lambda (elem result)
(if (eq? elem x) (or result #t) result))
#f
lst))
Thanks,
Each Let call creates a new set of variables in the environment that the main body of the Let is being evaluted in. The Let syntax is a "syntactic sugar" for a lambda being evaluated with arguments passed to it that have been evaluted. For instance
(let ((a (func object))
(b (func object2)))
(cons a b))
is the same as writing
((lambda (a b) (cons a b)) (func object) (func object2))
So you can see that in the Let syntax, the arguments are first evaluated, and then the body is evaluated, and the definitions of a and b are utilized in the local environment scope. So if you recursively call Let, each time you enter the body of the Let call, you are evaluating the body in a new environment (because the body is inside a newly defined lambda), and the definition of the arguments defined in the local Let scope will be different (they are actually new variables in a nested environment setup by the new lambda, not simply variables that have been mutated or "re-defined" like you would find in a C++ loop).
Another way of saying this is that you're variables will be like the local scope variables in a C++ recursive function ... for each function's stack-frame, the locally scoped variables will have their own definition, and their own memory location ... they are not mutated variables like you might see in a loop that re-uses the same memory variables in the local scope.
Hope this helps,
Jason
let always reinitialises variables; it's evident, since you must always provide new binding values. e.g.,
(let ((a 42))
...)
Inside the ..., a starts out as 42, always. It doesn't "retain" values from previous invocations.
By the way, I think you meant to write (or result (equal? elem x)) rather than (if (eq? elem x) (or result #t) result). :-)
(or result (equal? elem x)) translates to the following C++ code:
return result || elem == x;
(assuming that the == operator has been overloaded to perform what equal? does, of course.) The benefit of this is that if result is already true, no further comparisons are performed.
I'm actually reading the book for fun but it might be considered homework. At any event, I don't feel comfortable with local state variables at all with this language... Take for example this code:
(define flip
(let ((count 0))
(lambda ()
(if (= 0 count)
(begin (set! count 1) count)
(begin (set! count 0) count)))))
why does this code alternate between 1 and 0? count is given the value of 0 every time this function is called! A python equivalent would be:
class Flip:
def __init__(self):
pass
def __call__(self):
count = 0
if count == 0:
count = 1
return count
else:
count = 0
return count
This returns the same thing every time. I'm confused...
I have a little experience with writing compilers for functional languages, so perhaps a short description of how that function is stored/represented in memory is in order. Every function can roughly be thought of as a pair (E,F) where E is the set of free variables, and F is the "code" of the function itself. When you call the function, it takes the values in E and substitutes those in for the variables in F, and then executes the code using those values.
So, where your example is concerned, you have defined the variable "flip" to be the function returned by your let expression. That function is the stuff inside your lambda. Because "count" is defined outside the lambda, it's a free variable, so it's stored in the function's environment. Then, every time you call (flip), the interpreter goes to the code in the lambda, sees that it needs to look up the value of "count" in the environment, does that, changes it, and returns. That's why each time you call it, the value stored in "count" persists.
If you want count to be zero every time you call flip, put the let expression inside the lambda, so it's a bound variable instead of a free variable.
The lambda is a closure. It's a function that references a free variable (count), which, not being locally defined or one of the parameters, is bound to the closest enclosing lexical environment.
The function being called is the lambda, not "flip". Flip is just a name you've given to the lambda that's returned out of the (let ...) expression.
As for the Python, I don't know the language, but it looks like count should be a member of the Flip object, not a variable local to call.
Because your flip function actually returns a function (which is defined inside lambda)
Each time you call the returned function it modifies its environment.
If you think about it the let creates the environment (and initializes count to 0) only once - when the lambda function is returned to you.
In a sense lambda creates a function object for you which uses environment, whose last frame was initialized in let with a single variable count. Each time you call your function it modifies its environment.
If you call flip a second time it returns another function object with different environment. (count initialized to 0) You can then toggle the two functors independently.
If you want to undestand fully how it works you should read about environmantal model.
it's more like
class Flip:
def __init__(self):
self.count = 0
def __call__(self):
if self.count == 0:
self.count = 1
return self.count
else:
self.count = 0
return self.count
Update with more explanation:
The function in Scheme is a closure that "closes" around the free variable count, which is defined in the scope outside it. The way that count is defined in a let with just the function as the body, means that the function is the only thing that can access it -- making count effectively a kind of private mutable state that is attached to the function.
This is the way "objects" are traditionally created in Scheme in SICP -- to have a let define a bunch of variables (the instance variables, initialized to their initial values), and in the body define a bunch of functions which are "methods" that have shared access to the instance variables. That is why it is natural here to use a Python class to represent what is going on, with count being an instance variable.
A more literal translation into Python 3.x would be something like this (note that it is only approximate as Python doesn't have a let (limited-scope local variable declaration) syntax, and Python's lambdas can't be used because they don't take statements):
count = 0
def flip():
nonlocal count
if count == 0:
count = 1
return count
else:
count = 0
return count
# pretend count isn't in scope after this
The problem with the original code is that it has a strong influence of the imperative style. A more idiomatic solution will be:
(define (flip)
(let ((flag #t))
(lambda ()
(set! flag (not flag))
(if flag 1 0))))
To answer the question in you comment ooboo, you want a function that returns a function
(define make-flipper
(lambda ()
(let ((count 0))
(lambda ()
(let ((v count))
(set! count (- 1 count))
v)))))
;; test it
(let ((flip-1 (make-flipper)))
(format #t "~s~%" (flip-1))
(format #t "~s~%" (flip-1))
(format #t "~s~%" (flip-1))
(let ((flip-2 (make-flipper)))
(format #t "~s~%" (flip-2))
(format #t "~s~%" (flip-2))
(format #t "~s~%" (flip-2))))
You can trivially change the set! line so that it becomes a counter, not a flipper (more useful).