When I evaluate the following expression every time I get the value 10.
(((lambda (x) (lambda () (set! x (+ x 10)) x)) 0))
However I just modify by abstracting the above procedure with a name and call foo every time the value increments by 10!!
(define foo ((lambda (x) (lambda () (set! x (+ x 10)) x)) 0))
Can anybody please explain this?
The function you are calling is a counter that returns a number 10 higher every time it's called.
In the first case, every time, you are creating a new function and then immediately calling it once and then discarding the function. So every time, you are calling a new instance of this counter for the first time, so it should return 10.
In the second case, you create the function once and assign it to a variable and call that same function repeatedly. Since you are calling the same function, it should return 10, 20, ...
newacct is correct, but I would like to go into (a lot) more detail, since this is something that just blew my mind pretty recently.
I'm going to use the terms 'environment' and 'scope' pretty loosely and to mean essentially the same thing. Remember that scheme is a lexical scope language.
When scheme evaluates an expression it will look in its current environment for the values of any variables in the expression. If it doesn't find any in the current environment, it will look in the parent environment. If the value is not in the parent environment then it will look in the next level up and so on until it reaches the top (global) level where it will either find the value or throw an "unbound variable" error.
Anytime you call define you associate a symbol with a value on that environments symbol table. So if you call define on the top-level an entry will be added to the global symbol table. If you call define in the body of a procedure, then an entry will be added to the symbol table of that procedure.
A good way to think about calling define on a procedure is that you are creating an entry in the symbol table that consists of the parameters, body, and environment of that procedure. For example the procedure square would have an entry something like this:
(define a 3)
(define (square x)
(* x x))
GLOBAL
=================
a-|-3
|
square-|-{x}
| {(* x x)}
| {GLOBAL} ---> All the things defined on the global table
Then if I were to call (square a) the interpreter would first look in the environment in which square is defined and it would find that a is associated with the value 3. Then x -> 3 within the body of square and the procedure returns 9. Cool, makes sense.
Things get a little screwier when we start defining helper procedures within procedures, but all you really need to remember is that if it can't find anything associated with a symbol in the current environment, it will move up levels of scope until it does. Also, it will always stop on the first 'match'. So if there is a local x it will prefer it over the global x (rather it will use the local x without ever looking for a global one).
Next, remember that define just adds names to the symbol table, but set! is a mutator that actually changes the values with which a symbol is associated.
So (define b "blah") puts an entry in the symbol table. b => "blah". Nothing crazy. set! will change the actual value:
(set! b "foo")
b => "foo"
but set! can't add anything to the table. (set! c "bar") => UNBOUND VARIABLE C.
This is the most important difference: set! acts like any other procedure in that if it doesn't find the variable in the current scope, it will check progressively higher levels until it finds a match (or throws an error), but define always adds a binding to the scope in which it is called.
Alright, so you understand the difference between define and set!. Good. Now on to the question.
The expression (((lambda (x) (lambda () (set! x (+ x 10)) x)) 0)), as newacct pointed out, is going to return the same value each time because you are calling a new procedure each time. However if you name it you can keep track of the environment created by calling the procedure.
(define foo <--- associated name on the symbol table
(lambda (x) <--- scope where x is defined
(lambda () \
(set! x (+ x 10)) |--- body
x)) /
0) <--- initial value of x
So the inner lambda exists inside the environment created by the first one where the symbol x exists at an initial value of 0. Then set! looks for an entry in the symbol table for x and finds one in the next level up. Once it finds the entry it changes it, in this case adding 10 to whatever value it finds there. The really cool part is that since you associated the whole thing to a name in the global symbol table, that environment continues to exist after each call! This is why we can do cool things like implement message passing objects to keep track of and manipulate data!
Also, the let special form was created for this purpose, and may be a more intuitive way to structure this. It would look like this:
(define foo <--- associated name
(let ((x 0)) <--- scope where x is defined & initial x value
(lambda () \
(set! x (+ x 10)) |--- body
x))) /
Related
Is there normal "set" function (not special form) in Scheme language or some way to implement it?
I'd like to write code something like:
(map (lambda (var)
(set var 0))
'(a b c))
which could assign a value (here it is '0') to variables from list (here they are 'a', 'b' and 'c').
No. And to see why there isn't consider something like this:
(define (mutant var val)
(let ((x 1))
(set var val)
x))
Now, what should (mutant 'x 3) return? If it should return 3, then:
set can't be a function since it needs access to the lexical environment of mutant;
any kind of reasonable compilation of this function is not possible.
If you want set to be a function then catastrophe follows. Consider this definition:
(define (mutant-horror f)
(let ([x 3])
(f)
x))
Now, you would think that this can be optimised to this:
(define (mutant-horror f)
(f)
3)
But it can't. Because you might call it like this:
(mutant-horror (λ () (set 'x 3)))
or, more generally, you might call it with a function which, somewhere eventually in some function called indirectly from it might end up saying (set 'x 3).
This means that no binding can ever be optimised at all, which is a disaster. It's also at least very close to meaning lexical scope is not possible: if as well as set, a function called get exists, which retrieves the binding of a symbol, then you have, essentially, dynamic scope. That in turn makes things like tail-call elimination at least difficult and probably impossible (in fact set probably does this on its own).
Reasons like this are why even very old Lisps, where things like set did exist and did superficially work, actually made special exemptions for compiled code, where set didn't work (see for instance the Lisp 1.5 programmer's manual (PDF link), appendix D. This divergence between the semantics of compiled and interpreted code is one of the things that later Lisps and Lisp-related languages such as CL and Scheme did away with.
If instead you want something like Common Lisp's semantics, where the equivalent thing
(defun mutant (var val)
(let ((x 1))
(set var val)
x))
Would return 1 (unless x was a globally (see below) special variable, in which case it might return something else) and as a side-effect modify the value cell of whatever symbol was named by var (which might be x), then, well, Scheme has no notion of that at all, and that's a good thing on the whole.
Note that a modified version of the function will also work for locally special variables:
(defun mutant/local-special (a b)
(let ((x 1))
(declare (special x))
(set a b)
x))
But in this case you always know there's a special binding happening because you can always see the declaration.
When you write something like
(map (lambda (var)
(set var 0))
'(a b c))
my first thought was that you try to accumulate unordered sets of the form ( (a 0) (b 0) (c 0) ).
You cannot implement your own setter for any of the internal data structures that are provided by the language as this would mean to write a scheme function to modify some data structures that is implemented in C. For the data structures implemented in C you need to provide setters written in C -- supposing the lower language is C.
If you want to implement your own setter you either
-- check how the data structure in implmented, and if it's implemented in scheme you will undestand how to modify it
-- define your own data structure using already existing data structures and define setters for it.
A setter that mutates the data structure contains a ! at the end of its name, such as set!, append!, etc.
A function call just evaluates a series of instructions in an extended environment, the environment in particular is that of definition extended with its function parameters, this is prety much the case in any language...
If you do this:
(define (my-set var val)
(set! var val))
you will bind the value of val to var, but only within the scope of the current call on my-set. The reason you cannot write such function has to do with the nature of scheme itself, var is a pointer to whatever you pass in the function, but set! will make this pointer point to something else (still within the scope of my-set). my-set could work if we had some sort of mechanism of using actual pointers, as some low lever languages allow. But scheme does not...
Note that scheme goes very well with the functional programming style as well as recursion, so if you have a need for a function as you described, you are probably "doing something wrong"... :)
You can, however, do this:
(define my-list (list 1 2 3))
(define (my-set a-list a-value)
(set-car! my-list a-value))
> (my-set my-list 4)
> my-list
(4 2 3)
this works since a-list is a pointer to a cons-cell, set-car! will modify the contents of a cons-cell, but not affect the pointer to which.
I am a little confused on how the environment model of evaluation works, and hoping someone could explain.
SICP says:
The environment model specifies: To apply a procedure to arguments,
create a new environment containing a frame that binds the parameters
to the values of the arguments. The enclosing environment of this
frame is the environment specified by the procedure. Now, within this
new environment, evaluate the procedure body.
First example:
If I:
(define y 5)
in the global environment, then call
(f y)
where
(define (f x) (set! x 1))
We construct a new environment (e1). Within e1, x would be bound to the value of y (5). In the body, the value of x would now be 1. I found that y is still 5. I believe the reason for this is because x and y are located in different frames. That is, I completely replaced the value of x. I modified the frame where x is bound, not just its value. Is that correct?
Second example:
If we have in the global environment:
(define (cons x y)
(define (set-x! v) (set! x v))
(define (set-y! v) (set! y v))
(define (dispatch m)
(cond ((eq? m 'car) x)
((eq? m 'cdr) y)
((eq? m 'set-car!) set-x!)
((eq? m 'set-cdr!) set-y!)
(else (error "Undefined
operation: CONS" m))))
dispatch)
(define (set-car! z new-value)
((z 'set-car!) new-value)
z)
Now I say:
(define z2 (cons 1 2))
Suppose z2 has a value the dispatch procedure in an environment called e2, and I call:
(set-car! z2 3)
Set-car! creates a new environment e3. Within e3, the parameter z is bound to the value of z2 (the dispatch procedure in e2) just like in my first example. After the body is executed, z2 is now '(3 2). I think set-car! works the way it does is because I am changing the state of the object held by z (which is also referenced by z2 in global), but not replacing it. That is, I did not modify the frame where z is bound.
In this second example it appears that z2 in global and z in e3 are shared. I am not sure about my first example though. Based on the rules for applying procedures in the environment model, it appears x and y are shared although it is completely undetectable because 5 does not have local state.
Is everything I said correct? Did I misunderstood the quote?
To answer your first question: assuming that you meant to write (f y) in your first question rather than (f 5), the reason that y is not modified is that racket (like most languages) is a "call by value" language. That is, values are passed to procedure calls. In this case, then the argument y is evaluated to 5 before the call to f is made. Mutating the x binding does not affect the y binding.
To answer your second question: in your second example, there are shared environments. That is, z is a function that is closed over an environment (you called it e2). Each call to z creates a new environment that is linked to the existing e2 environment. Performing mutation on either x or y in this environment affects all future references to the e2 environment.
Summary: passing the value of a variable is different from passing a closure that contains that variable. If I say
(f y)
... the after the call is done, 'y' will still refer to the same value[*]. If I write
f (lambda (...) ... y ...)
(that is, passing a closure that has a reference to y, then y might be bound to a different value after the call to f.
If you find this confusing, you're not alone. The key is this: don't stop using closures. Instead, stop using mutation.
[*] if y is a mutable value, it may be mutated, but it will still be the "same" value. see note above about confusion.
TL;DR: simple values in Scheme are immutable, are copied in full when passed as arguments into functions. Compound values are mutable, are passed as a copy of a pointer, whereas the copied pointer points to the same memory location as the original pointer does.
What you're grappling with is known as "mutation". Simple values like 5 are immutable. There's no "set-int!" to change 5 to henceforth hold the value 42 in our program. And it is good that there isn't.
But a variable's value is mutable. A variable is a binding in a function invocation's frame, and it can be changed with set!. If we have
(define y 5)
(define (foo x) (set! x 42) (display (list x x)))
(foo 5)
--> foo is entered
foo invocation environment frame is created as { x : {int 5} }
x's binding's value is changed: the frame is now { x : {int 42} }
(42 42) is displayed
y still refers to 5 in the global environment
But if foo receives a value that is itself holding mutable references, which can be mutated, i.e. changed "in place", then though foo's frame itself doesn't change, the value to which a binding in it is referring can be.
(define y (cons 5 6)) ; Scheme's standard cons
--> a cons cell is created in memory, at {memory-address : 123}, as
{cons-cell {car : 5} {cdr : 6} }
(define (foo x) (set-car! x 42) (display (list x x)))
(foo y)
--> foo is entered
foo invocation environment frame is created as
{ x : {cons-cell-reference {memory-address : 123}} }
x's binding's value is *mutated*: the frame is still
{ x : {cons-cell-reference {memory-address : 123}} }
but the cons cell at {memory-address : 123} is now
{cons-cell {car : 42} {cdr : 6} }
((42 . 6) (42 . 6)) is displayed
y still refers to the same binding in the global environment
which still refers to the same memory location, which has now
been altered in-place: at {memory-address : 123} is now
{cons-cell {car : 42} {cdr : 6} }
In Scheme, cons is a primitive which creates mutable cons cells which can be altered in-place with set-car! and set-cdr!.
What these SICP exercises intend to show is that it is not necessary to have it as a primitive built-in procedure; that it could be implemented by a user, even if it weren't built-in in Scheme. Having set! is enough for that.
Another jargon for it is to speak of "boxed" values. If I pass 5 into some function, when that function returns I'm guaranteed to still have my 5, because it was passed by copying its value, setting the function invocation frame's binding to reference the copy of the value 5 (which is also just an integer 5 of course). This is what is referred to as "pass-by-value".
But if I "box" it and pass (list 5) in to some function, the value that is copied -- in Lisp -- is a pointer to this "box". This is referred to as "pass-by-pointer-value" or something.
If the function mutates that box with (set-car! ... 42), it is changed in-place and I will henceforth have 42 in that box, (list 42) -- under the same memory location as before. My environment frame's binding will be unaltered -- it will still reference the same object in memory -- but the value itself will have been changed, altered in place, mutated.
This works because a box is a compound datum. Whether I put a simple or compound value in it, the box itself (i.e. the mutable cons cell) is not simple, so will be passed by pointer value -- only the pointer will be copied, not what it points to.
x bound to the value of y means that x is a new binding which receives a copy of the same value that y contains. x and y are not aliases to a shared memory location.
Though due to issues of optimization, bindings are not exactly memory locations, you can model their behavior that way. That is to say, you can regard an environment to be a bag of storage locations named by symbols.
Educational Scheme-in-Scheme evaluators, in fact, use association lists for representing environments. Thus (let ((x 1) (y 2)) ...) creates an environment which simply looks like ((y . 1) (x . 2)). The storage locations are the cdr fields of the cons pairs in this list, and their labels are the symbols in the car fields. The cell itself is the binding; the symbol and location are bound together by virtue of being in the same cons structure.
If there is an outer environment surrounding this let, then these association pairs can just be pushed onto it with cons:
(let ((z 3))
;; env is now ((z . 3))
(let ((x 1) (y 2))
;; env is now ((y . 2) (x . 1) (z . 3))
The environment is just a stack of bindings that we push onto. When we capture a lexical closure, we just take the current pointer and stash it into the closure object.
(let ((z 3))
;; env is now ((z . 3))
(let ((x 1) (y 2))
;; env is now ((y . 2) (x . 1) (z . 3))
(lambda (a) (+ x y z a))
;; lambda is an object with these three pices:
;; - the environment ((y . 2) (x . 1) (z . 3))
;; - the code (+ x y z a)
;; - the parameter list (a)
)
;; after this let is done, the environment is again ((z . 3))
;; but the above closure maintains the captured one
)
So suppose we call that lambda with an argument 10. The lambda takes the parameter list (a) and binds it to the argument list to create a new environment:
((a . 1))
This new environment is not made in a vacuum; it is created as an extension to the captured environment. So, really:
((a . 1) (y . 2) (x . 1) (z . 3))
Now, in this effective environment, the body (+ x y z a) is executed.
Everything you need to understand about environments can be understood in reference to this cons pair model of bindings.
Assignment to a variable? That's just set-cdr! on a cons-based binding.
What is "extending an environment"? It's just pushing a cons-based binding onto the front.
What is "fresh binding" of a variable? That's just the allocation of a new cell with (cons variable-symbol value) and extending the environment with it by pushing it on.
What is "shadowing" of a variable? If an environment contains (... ((a . 2)) ...) and we push a new binding (a . 3) onto this environment, then this a is now visible, and (a . 2) is hidden, simply because the assoc function searches linearly and finds (a . 2) first! The inner-to-outer environment lookup is perfectly modeled by assoc. Inner bindings appear to the left of outer bindings, closer to the head of the list and are found first.
The semantics of sharing all follow from the semantics of these lists of cells. In the assoc list model, environment sharing occurs when two environment assoc lists share the same tail. For instance, each time we call our lambda above, a new (a . whatever) argument environment is created, but it extends the same captured environment tail. If the lambda changes a, that is not seen by the other invocations, but if it changes x, then the other invocations will see it. a is private to the lambda invocation, but x, y and z are external to the lambda, in its captured environment.
If you fall back on this assoc list model mentally, you will not go wrong as far as working out the behavior of environments, including arbitrarily complex situations.
Real implementations basically just optimize around this. for instance, a variable that is initialized from a constant like 42 and never assigned does not have to exist as an actual environment entry at all; the optimization called "constant propagation" can just replace occurrences of that variable with 42, as if it were a macro. Real implementations may use hash tables or other structures for the environment levels, not assoc lists. Real implementations may be compiled: lexical environments can be compiled according to various strategies such as "closure conversion". Basically, an entire lexical scope can be flattened into a single vector-like object. When a closure is made at run time, the entire vector is duplicated and initialized. Compiled code doesn't refer to variable symbols, but to offsets in the closure vector, which is substantially faster: no linear search through an assoc list is required.
I'm pretty new to Scheme and I'm currently working in DR Racket to learn it. So basically my problem is that I'm having trouble permanently changing global variable.
This is my change function:
(define (change b)
(set! b (- b 1)) b)
As you can note I return the value of a at the end of the function.
And this is the output on the console.
> (define a 4)
> (change a)
3
> a
4
As you can see, in the function it returned the value of 3 (4 - 1). But it seems as though the global variable didn't actually change when i call it after the function. How do I leave it changed? Thanks in advance.
First, Racket is NOT Scheme. They both are similar in this example, but they are very different languages. And calling them the same thing is similar to saying C and C++ are the same language.
Now, on to your actual reference.
The reason change doesn't appear to do anything is because you are passing the 'value' into b, rather than any actual variable.
If you want to pass the actual reference of a primitive like this what you need to do is to box it with the box function. Then you can use set-box! to do the actual mutation, and unbox to get the value in the box
Giving you the following function:
(define (change b)
(set-box! b (- (unbox b) 1)) b)
Now when you use the change function, the variable should update.
> (define a (box 4))
> (change a)
'#&3
> a
'#&3
> (unbox a)
3
(The '#& is just a compact way that the Racket repl says that the next value is in a box. Which is why we get 3 when you unbox it.)
There are plenty of other data structures to that will have the same effect, vectors, structs, etc. But for a single value box is the simplest.
You're not truly changing the global variable! you're just modifying a parameter, which for all practical purposes is acting as a local variable. If you want to modify the global one, don't pass it as a parameter:
(define a 4)
(define (change)
(set! a (- a 1))
a)
a
=> 4
(change)
=> 3
a
=> 3
I have see the following code... The first call of (next-num) returns 1, and the second returns 2.
(define next-num
(let ((num 0))
(lambda () (set! num (+ num 1)) num)))
(next-num) ; 1
(next-num) ; 2
What I can not understand is... num is created by let inside next-num, it is kind of a local variable... How does scheme know that each time next-num is called, the value of num is not erased by let ((num 0)); How does scheme know that it is always the same num that we modify whenever next-num is called?
It seems that num is both local and static... How can we define a local variable, but not static?
This is "lexical closure" and you're right that num, the "closed-over variable" is similar to a static variable, in C for example: it's only visible to code within the let form (its "lexical scope"), but it persists across the whole program run, rather than being re-initialized with each call to the function.
I think the part you're confused on is this: "num is created by let inside next-num, it is kind of a local variable". This isn't true because the let block isn't part of the next-num function: it's actually an expression which creates and returns the function which is then bound to next-num. (This is very different, e.g., from C, where functions can only be created at compile-time and by defining them at top-level. In Scheme, functions are values like integers or lists, which any expression can return).
Here's another way to write (almost) the same thing which makes it clearer that the define is just associating next-num to the value of a function-returning expression:
(define next-num #f) ; dummy value
(let ((num 0))
(set! next-num
(lambda () (set! num (+ num 1)) num)))
It's important to note the difference between
(define (some-var args ...) expression expression ...)
which makes some-var a function which executes all the expressions when called, and
(define some-var expression)
which binds some-var to the value of expression, evaluated then and there. Strictly speaking, the former version is unnecessary, because it's equivalent to
(define some-var
(lambda (args ...) expression expression ...))
Your code is almost the same as this, with the addition of the lexically scoped variable, num, around the lambda form.
Finally, here's a key difference between closed-over variables and static variables, which makes closures much more powerful. If you had written the following instead:
(define make-next-num
(lambda (num)
(lambda () (set! num (+ num 1)) num)))
then each call to make-next-num would create an anonymous function with a new, distinct num variable, which is private to that function:
(define f (make-next-num 7))
(define g (make-next-num 2))
(f) ; => 8
(g) ; => 3
(f) ; => 9
This is a really cool and powerful trick which accounts for a lot of the power of languages with lexical closures.
Edited to add: You ask how Scheme "knows" which num to modify when next-num is called. In outline, if not in implementation, this is actually pretty simple. Every expression in Scheme is evaluated in the context of an environment (a lookup table) of variable bindings, which are associations of names to places which can hold values. Each evaluation of a let form or a function call creates a new environment by extending the current environment with new bindings. To arrange to have lambda forms behave as closures, the implementation represents them as a structure consisting of the function itself plus the environment in which it was defined. Calls to that function are then evaluated by extending the binding environment in which the function was defined -- not the environment in which it was called.
Older Lisps (including Emacs Lisp until recently) had lambda, but not lexical scope, so although you could create anonymous functions, calls to them would be evaluated in the calling environment rather than the definition environment, and so there were no closures. I believe Scheme was the first language to get this right. Sussman and Steele's original Lambda Papers on the implementation of Scheme make great mind-expanding reading for anyone who wants to understand scoping, among many other things.
What's the point of using the set! assignment operator in scheme? Why not just rebind a variable to a new value using define?
> (define x 100)
> (define (value-of-x) x) ;; value-of-x closes over "x"
> x
100
> (value-of-x)
100
> (set! x (+ x 1))
> x
101
> (value-of-x)
101
> (define x (+ x 1))
> x
102
> (value-of-x)
102
>
Though both define and set! will redefine a value when in the same scope, they do two different things when the scope is different. Here's an example:
(define x 3)
(define (foo)
(define x 4)
x)
(define (bar)
(set! x 4)
x)
(foo) ; returns 4
x ; still 3
(bar) ; returns 4
x ; is now 4
As you can see, when we create a new lexical scope (such as when we define a function), any names defined within that scope mask the names that appear in the enclosing scope. This means that when we defined x to 4 in foo, we really created a new value for x that shadowed the old value. In bar, since foo does not exist in that scope, set! looks to the enclosing scope to find, and change, the value of x.
Also, as other people have said, you're only supposed to define a name once in a scope. Some implementations will let you get away with multiple defines, and some won't. Also, you're only supposed to use set! on a variable that's already been defined. Again, how strictly this rule is enforced depends on the implementation.
It is not usually permitted to define a variable more than once. Most REPLs allow it for convenience when you're trying things out, but if you try to do that in a Scheme program it will give you an error.
For example, in mzscheme, the program
#lang scheme
(define x 1)
(define x 2)
gives the error
test.ss:3:8: module: duplicate definition for identifier at: x in: (define-values (x) 2)
In addition, define has a different meaning when used inside of other contexts. The program
#lang scheme
(define x 1)
x
(let ()
(define x 2)
x)
x
has the output
1
2
1
This is because defines inside of certain constructs are actually treated as letrecs.
When you use lexical bindings you do not define them:
(let ((x 1))
(set! x (+ x 1))
x)
When you use define you create a new variable with the new value, while the old variable still exists with the old value; it is just hidden by the new one. On the command line you don't see the difference to set!, but define won't be usable for e.g. a loop counter in an imperative program.