I would like to know if there is a style guideline, published by ANSI or implementation authors or another influential authority, for Lisp functions which are implemented using recursive helper functions which take additional parameters that the person calling the function doesn't need to think about. Here is the simplest example I can think of. Which of these three, if any, is preferred in a standardized style guide for common lisp (if there is one)?
(defun factorial (n)
(defun tail-factorial (n m)
(if (= n 1)
m
(tail-factorial (- n 1) (* n m))))
(tail-factorial n 1))
This seems unpleasant because of the function declaration within the function declaration. I would use a lambda to avoid naming the helper function and convoluting things, but I am not clear how to recurse within a lambda expression by having it call itself. Even if there is a way to have an anonymous function call itself, it seems like it would get messy, especially if the helper needs a helper needs a helper ...
Another alternative is to declare the tail guy first (or after, but that makes sbcl complain):
(defun tail-factorial (n m)
(if (= n 1)
n
(tail-factorial (- n 1) (* n m))))
(defun factorial (n)
(tail-factorial n 1))
This seems unpleasant because somebody reading my code would see tail-factorial and might feel the need to understand it, even though it's just a helper function for factorial and won't be used elsewhere. In my own code, I keep writing functions analogous to this and I really struggle to come up with comments that will make me re-understand what I did months or years from now. Again, the situation gets really bad here when the helper needs a helper needs a ...
Another alternative uses optionals:
(defun factorial (n &optional (m 1))
(if (= n 1)
m
(factorial (- n 1) (* n m))))
This seems unpleasant because we expect only one argument for a factorial. Nobody outside this function would have a reason to call this with that second argument. I find it annoying to try to understand code with optional arguments I will never use.
Now, I'm clear that asking what you think is best is the kind of subjective conversation stackoverflow dislikes, so my objective question is whether or not there is some kind of standardized style guideline about which of these alternatives is preferred. I use SBCL and have not found a guideline for this sort of thing on their site, and I am not aware of such a guide published by ANSI or any other standardizing body, including other implementors.
Perhaps there is another alternative altogether, and I welcome your suggestions. I keep running into this situation of needing a little helper function (who needs a helper function, etc) and I want to get used to writing in the way most people will find clear. Somebody asked a similar question at Recursing in a lambda function , and several people recommended some favorite pet projects but nobody mentioned a style guideline.
Thanks in advance!
defun only makes global functions. If you use defun inside defun you are actually making a new global function at each time you call it. To make a local lexical function to only be used inside factorial the standard way is with labels
(defun factorial (n)
(labels ((tail-factorial (n m)
(if (= n 1)
m
(tail-factorial (- n 1) (* n m)))))
(tail-factorial n 1)))
Common Lisp has no guarantee to do tail call optimization so this might blow the stack. For serious work you would have used the loop macro to do this.
Optional arguments works, but you really shouldn't make internals optional if it's not the intention that the user would ever want to supply it. Then it's just leaking abstraction and making confusing documentation.
I think that some of the answers provide more general answers, but I do want to point out that some of the most common uses of tail-recursive helper functions are often very cleanly expressed using a do loop. With a do loop, you declare the variables, their initial values, a step form (the value for the next iteration), a termination condition, and a result form. (Some of those are optional, but we'll use all of them here.) In that sense, a do is more like Scheme's named let, except that you don't end up putting a recursive call in the body. In fact, many times, you end up not needing a body at all. For the factorial function, your code could be expressed as:
(defun factorial (n)
(do ((n n (1- n)) ; n starts as n, and is (1- n) on next iteration
(m 1 (* n m))) ; m starts at 1, and is (* m n) on next iteration
((= 1 n) m))) ; if (= 1 n), then return m, else go to next iteration
CL-USER> (factorial 11)
39916800
Speaking of named let, you can implement it pretty easily in terms of labels in Common Lisp:
(defmacro nlet (name bindings &body body)
`(labels ((,name ,(mapcar 'first bindings)
,#body))
(,name ,#(mapcar 'second bindings))))
CL-USER> (macroexpand-1 '(nlet factorial ((n n) (m 1))
(if (= n 1) m
(factorial (1- n) (* m n)))))
(LABELS ((FACTORIAL (N M)
(IF (= N 1)
M
(FACTORIAL (1- N) (* M N)))))
(FACTORIAL N 1))
That still has the issue that Common Lisp implementations don't have to support tail recursion, though, but it can be a useful utility if you're coming from a Scheme background. You could implement a nlet type macro that makes the recursive call go to the next iteration, but that wouldn't be quite the same, since you only want the "reuse stack frame" behavior in a tail position; in a non-tail position, you'd still want the proper return.
A very subtle bug in the implementation above
As Sylwester pointed out in the comments, there's a very subtle bug in the implementation of nlet above. Because the call to the local function happens within the lexical scope where it's defined, doing something like
(nlet frob ((f #'frob))
...)
makes the initializer #'foo a reference to the new local function. That's probably not what you'd want, and it's not what Scheme's named let does. You can get around this by returning the local function from the labels form and calling it outside of that scope:
(defmacro nlet (name bindings &body body)
`(funcall (labels ((,name ,(mapcar 'first bindings)
,#body))
#',name)
,#(mapcar 'second bindings)))
CL-USER> (macroexpand-1 '(nlet factorial ((n n) (m 1))
(if (= n 1) m
(factorial (1- n) (* m n)))))
(FUNCALL
(LABELS ((FACTORIAL (N M)
(IF (= N 1)
M
(FACTORIAL (1- N) (* M N)))))
#'FACTORIAL)
N 1)
(defmacro nlet (name bindings &body body)
`(funcall (labels ((,name ,(mapcar 'first bindings)
,#body))
#',name)
,#(mapcar 'second bindings)))
Good style is:
useful function name
documentation string
argument type checking
no sources of control stack overflows in portable code -> mostly avoid tail-recursive code
Example:
(defun factorial (n &aux (result 1))
"Calculates the factorial N! for N >= 0"
(check-type n (integer 0 *))
(loop for i from 1 to n
do (setf result (* result i)))
result)
Using:
CL-USER 94 > (apropos "factorial")
FACTORIAL (defined)
CL-USER 95 > (documentation 'factorial 'function)
"Calculates the factorial N! for N >= 0"
CL-USER 96 > (factorial 7)
5040
Related
write a function in lisp called number(N) that you have to use a nonnegative integer N, and produce the list of all integers from 1 up to and including N.
(defun numbers (N)
(if (<= N 0)
nil
(cons N nil)
(numbers (- N 1)))
I checked some questions, but most of them use loop and range, but this question doesn't allowed me to do this, so I have to use recursion instead:
here is my code, but this code keeps giving me warning:
; caught STYLE-WARNING:
; The variable N is defined but never used.
;
; compilation unit finished
; caught 1 ERROR condition
; caught 1 STYLE-WARNING condition
I think my algorithm is correct ,but because I am new to lisp, I still don't know how to write the function properly. It is grateful if anyone could gave me any help.
IF has generally a common syntax, but there are exceptions
Generally in Lisps like Common Lisp the if operator allows the following syntax:
IF test-form then-form [else-form]
This means that in Lisp usually zero or one else-form are allowed. An example is if in Common Lisp.
In Emacs Lisp multiple else-forms are allowed. Emacs Lisp has the following syntax:
IF test-form then-form else-form*
This means that in Emacs Lisp zero or more else-forms are allowed.
Thus: it's important to mention which language&dialect you are actually using.
Your code
a) Let's assume that you use Common Lisp with its IF syntax.
Your code:
(defun numbers (N)
(if (<= N 0)
nil
(cons N nil)
(numbers (- N 1)))
Your code has the problem, that there are more than one else clauses. You need to write a version which has a single else clause.
b) Let's assume that you use Emacs Lisp with its IF syntax with multiple else forms.
Your code:
(defun numbers (N)
(if (<= N 0)
nil
(cons N nil)
(numbers (- N 1)))
Here the (cons N nil) form is allowed, but has no effect. Its return value is not used and it has no side effect. You could delete it and it would make no difference. Again: you would need how to combine its effect with the form (numbers (- N 1)).
Syntax error: missing closing parenthesis
There is another problem in your code. The s-expressions are not complete -> a closing parenthesis is missing:
(defun numbers (N)
(if (<= N 0)
nil
(cons N nil)
(numbers (- N 1)))
As you can see a closing parenthesis is missing at the end.
Thus your code can not be read by Lisp.
There are two ways one generally can avoid this problem:
count the parentheses and set them accordingly
use the editor to count the parentheses
Most people prefer the latter.
The way to think about this is to think about what the algorithm should be:
To compute the numbers from 1 to n:
if n is less than 1 then there are no numbers, so this is the empty list;
otherwise we want a list which looks like (... n), where ... is all the numbers from 1 to n-1.
Note that we want the numbers in forward order: this is going to be critical.
Doing this is slightly difficult in Lisp because we want the number to be at the end of the list, and access to the ends of lists is hard.
Here is the start of a version which builds the list backwards (so this is not the right answer).
(defun numbers (n)
(if (< n 1)
'() ;the empty list
;; n 1 or more, so build a list which is (n . ...)
(cons n <some function involving n>)))
Well, OK, what function should we call recursively? Do we have a function which returns the list we want? Well, yes: it's numbers, with an argument which is one less than n!
(defun numbers (n)
(if (< n 1)
'()
(cons n (numbers (- n 1)))))
And this function works. But it gets the wrong answer: the list is backwards:
> (numbers 10)
(10 9 8 7 6 5 4 3 2 1)
There are two fixes to this problem: the first is to build the list forwards, using append. This version looks like this (remember append wants to append two lists: it doesn't append an element to the end of a list):
(defun numbers (n)
(if (< n 1)
'()
(append (numbers (- n 1)) (list n))))
This gets the right answer:
> (numbers 10)
(1 2 3 4 5 6 7 8 9 10)
but it's a terrible answer: append has to walk all the way down the list (lists in Lisp are chains of conses: there is no fast access to the end of a list), copying it as it goes, to append the new element. So this has absolutely terrible space & time complexity. Programs written like this are why 'Lisp is slow'.
A better approach is to build the list backwards and then reverse it.
(defun numbers (n)
(reverse (numbers-backwards n)))
(defun numbers-backwards (n)
(if (< n 1)
'()
(cons n (numbers-backwards (- n 1)))))
The problem with this, from the homework perspective, might be that using reverse is not allowed. That's OK, we can write it, recursively. The implementation is slightly fiddly, but this is going to help us below.
(defun reverse-list (l)
;; in real life reverse-list-accumulator would be a local function
(reverse-list-accumulator l '()))
(defun reverse-list-accumulator (l accum)
(if (null l)
accum
(reverse-list-accumulator (rest l) (cons (first l) accum))))
The way this works is that reverse-list calls this auxiliary function with an extra argument. The auxiliary function then checks the list, and if it's not empty it calls itself with the tail of the list and the head of the list consed onto the auxiliary argument. If it is empty, it returns the auxiliary argument. It's a little subtle but you can see that this in fact reverses the list.
So now we can write our function using only recursive functions we wrote:
(defun numbers (n)
(reverse-list (numbers-backwards n)))
But now there should be a moment of inspiration: why are we doing this whole
build-it-backwards-and-reverse-it thing? Why don't we just make numbers do the accumulator trick itself! Well, we can do that:
(defun numbers (n)
(numbers-accumulator n '()))
(defun numbers-accumulator (n accum)
(if (< n 1)
accum
(numbers-accumulator (- n 1) (cons n accum))))
And now we don't need to reverse the list, and for added value our
function is 'tail recursive' and will generally be compiled much more
efficiently.
A real-life version of numbers might look more like this, using a local function:
(defun numbers (n)
(labels ((numbers-accumulator (m accum)
(if (< m 1)
accum
(numbers-accumulator (- m 1) (cons m accum)))))
(numbers-accumulator n '())))
Here is a comparison between the version of numbers using append and the above function, on an argument small enough that the append version does not overflow the stack.
> (time (progn (numbers/append 2000) (values)))
Timing the evaluation of (progn (numbers/append 2000) (values))
User time = 0.024
System time = 0.001
Elapsed time = 0.017
Allocation = 32176304 bytes
97 Page faults
> (time (progn (numbers 2000) (values)))
Timing the evaluation of (progn (numbers 2000) (values))
User time = 0.000
System time = 0.000
Elapsed time = 0.001
Allocation = 32000 bytes
0 Page faults
You can see how terrible the append version is, and how good the other one is: this is a 64-bit Lisp, and conses are two words or 16 bytes: it has allocated precisely 2000 cons cells which is the minimum it could do.
This question already has an answer here:
Turning structural recursion into accumulative recursion in Racket
(1 answer)
Closed 4 years ago.
Is it possible to perform tail call optimization by the compiler/interpreter for the factorial function given below?
(define factorial
(lambda (x)
(if (= x 0)
1
(* x (factorial (- x 1))))))
I would like to get a brief explanation for the same.
From the comment by #rsm below I understand program should be written something like this:
(define fact
(lambda (x accumulator)
(if (<= x 1)
accumulator
(fact (- x 1) (* x accumulator)))))
(define factorial
(lambda (x)
(fact x 1)))
Or something like this:
(define factorial
(lambda (n)
(let fact ([i n] [a 1])
(if (= i 1)
a
(fact (- i 1) (* a i))))))
First, a terminological issue: you, as the programmer, cannot "do" tail call optimization. (This is one of the reasons why "tail call optimization" is a bad name for this property.) In this case, the "optimization" is done by the evaluator; specifically, a correct evaluator for Racket or Scheme or any properly tail-calling language makes a promise: it promises not to use unbounded memory on certain types of programs. Specifically, programs that make only "tail calls."
So, what you're really asking is how you can convert your program to one that makes only tail calls. The key here is to understand tail calls, and to convert your program to accumulator style. And, at this point, I'm going to defer to the excellent discussion that appears in Alexis King's answer.
It looks like you are trying to calculate x!.
You defined factorial as a lambda with input x (here it is in a readable pseudocode, not a prefix syntax as in Scheme):
factorial (x) = {
(x=0) -> 1 // end condition
(x<>0) -> x * fact(x-1) // 'fact' should be 'factorial' AFAIK
}
or in other words:
factorial(x) * factorial(x-1) * factorial((x-1)-1) ...factorial(0) => x!
the following is already tail recursive ( a recursion were the last call is the recursion):
factorial (x , sum(1)) = {
(x=0) -> sum // end condition
(x<>0) -> fact(x-1 , sum(x*sum)) // 'fact' should be 'factorial' AFAIK
}
back to code, should be something like:
(define factorial
(lambda (x , sum=1)
(if (= x 0)
sum
(fact (- x 1) (* x sum)))))
I feel that understanding this subtlety might help me to understand how scope works in Scheme.
So why does Scheme error out if you try to do something like:
(define (func n)
(define n (+ 1 n))
n)
It only errors out at runtime when calling the function.
The reason I find it strange is because Scheme does allow re-definitions, even inside functions. For example this gives no error and will always return the value 5 as expected:
(define (func n)
(define n 5)
n)
Additionally, Scheme also seems to support self-re-definition in global space. For instance:
(define a 5)
(define a (+ 1 a))
gives no error and results in "a" displaying "6" as expected.
So why does the same thing give a runtime error when it occurs inside a function (which does support re-definition)? In other words: Why does self-re-definition only not work when inside of a function?
global define
First off, top level programs are handled by a different part of the implementation than in a function and defining an already defined variable is not allowed.
(define a 10)
(define a 20) ; ERROR: duplicate definition for identifier
It might happen that it works in a REPL since it's common to redefine stuff, but when running programs this is absolutely forbidden. In R5RS and before what happened is underspecified and didn't care since be specification by violating it it no longer is a Scheme program and implementers are free to do whatever they want. The result is of course that a lot of underspecified stuff gets implementation specific behaviour which are not portable or stable.
Solution:
set! mutates bindings:
(define a 10)
(set! a 20)
define in a lambda (function, let, ...)
A define in a lambda is something completely different, handled by completely different parts of the implementation. It is handled by the macro/special form lambda so that it is rewritten to a letrec*
A letrec* or letrec is used for making functions recursive so the names need to be available at the time the expressions are evaluated. Because of that when you define n it has already shadowed the n you passed as argument. In addition from R6RS implementers are required to signal an error when a binding is evaluated that is not yet initialized and that is probably what happens. Before R6RS implementers were free to do whatever they wanted:
(define (func n)
(define n (+ n 1)) ; illegal since n hasn't got a value yet!
n)
This actually becomes:
(define func
(lambda (n)
(let ((n 'undefined-blow-up-if-evaluated))
(let ((tmpn (+ n 1)))
(set! n tmpn))
n)))
Now a compiler might see that it violates the spec at compile time, but many implementations doesn't know before it runs.
(func 5) ; ==> 42
Perfectly fine result in R5RS if the implementers have good taste in books.
The difference in the version you said works is that this does not violate the rule of evaluating n before the body:
(define (func n)
(define n 5)
n)
becomes:
(define func
(lambda (n)
(let ((n 'undefined-blow-up-if-evaluated))
(let ((tmpn 5)) ; n is never evaluated here!
(set! n tmpn))
n)))
Solutions
Use a non conflicting name
(define (func n)
(define new-n (+ n 1))
new-n)
Use let. It does not have its own binding when the expression gets evaluated:
(define (func n)
(let ((n (+ n 1)))
n))
I am trying to programmatically increment a purely alphabetical string in Scheme.
Like this "MA", then "MB" and when it reaches "MZ", it should become "MAA" and so on till "MZZ" and then it should become "MAAA" and so on.The "M" needs to be added as a prefix for the kind of work that I am doing.
I looked at this question: Incrementing alphabets and this is exactly what I want.
However, I have absolutely no idea from where to start. For starters I am not even sure how to handle ASCII in scheme. I am not expecting the whole code, but I would appreciate it if I got a few hints.
Here's my implementation. Note that you need to load SRFI 1, which provides unfold-right:
(define letters "ABCDEFGHIJKLMNOPQRSTUVWXYZ")
(define (number->letters num)
(unfold-right negative?
(lambda (i) (string-ref letters (remainder i 26)))
(lambda (i) (- (quotient i 26) 1))
num))
(define (number->tag num)
(list->string (cons #\M (number->letters num))))
Examples:
> (number->tag 0)
"MA"
> (number->tag 18277)
"MZZZ"
> (number->tag 18278)
"MAAAA"
The OP asked for an explanation of what the code does. So, with the understanding that the OP already understands the algorithm (since they linked to it already), what's basically left is the unfold operation.
Fold and unfold are a bit lengthy to explain and I don't want to derail this post by explaining them, but it's possible to "expand" the unfold into the equivalent loop (using the same variable names as the SRFI 1 reference implementation of unfold-right) to express what's going on:
(define (number->letters num)
(let lp ((seed num) (ans '()))
(if (negative? seed)
ans
(lp (- (quotient seed 26) 1)
(cons (string-ref letters (remainder seed 26)) ans)))))
Basically, it builds a list, from right to left, using (string-ref letters (remainder seed 26)) each iteration (where seed is num in the initial iteration). seed's value is then updated to (- (quotient seed 26) 1) for the next iteration. The list stops when (negative? seed) is true.
You might then ask why one would use an unfold instead of the loop. Basically, in functional programming, when a "concept" can be expressed in higher-level terms (e.g., for-each, map, filter, fold, or unfold), using those terms helps other programmers understand what the code does. It's a bit like "design patterns" (as commonly used in object-oriented programming), within a functional programming context. :-)
In an effort to find a simple example of CPS which doesn't give me a headache , I came across this Scheme code (Hand typed, so parens may not match) :
(define fact-cps
(lambda(n k)
(cond
((zero? n) (k 1))
(else
(fact-cps (- n 1)
(lambda(v)
(k (* v n))))))))
(define fact
(lambda(n)
(fact-cps n (lambda(v)v)))) ;; (for giggles try (lambda(v)(* v 2)))
(fact 5) => 120
Great, but Scheme isn't Common Lisp, so I took a shot at it:
(defun not-factorial-cps(n k v)
(declare (notinline not-factorial-cps)) ;; needed in clisp to show the trace
(cond
((zerop n) (k v))
((not-factorial-cps (1- n) ((lambda()(setq v (k (* v n))))) v))))
;; so not that simple...
(defun factorial(n)
(not-factorial-cps n (lambda(v)v) 1))
(setf (symbol-function 'k) (lambda(v)v))
(factorial 5) => 120
As you can see, I'm having some problems, so although this works, this has to be wrong. I think all I've accomplished is a convoluted way to do accumulator passing style. So other than going back to the drawing board with this, I had some questions: Where exactly in the Scheme example is the initial value for v coming from? Is it required that lambda expressions only be used? Wouldn't a named function accomplish more since you could maintain the state of each continuation in a data structure which can be manipulated as needed? Is there in particular style/way of continuation passing style in Common Lisp with or without all the macros? Thanks.
The problem with your code is that you call the anonymous function when recurring instead of passing the continuation like in the Scheme example. The Scheme code can easily be made into Common Lisp:
(defun fact-cps (n &optional (k #'values))
(if (zerop n)
(funcall k 1)
(fact-cps (- n 1)
(lambda (v)
(funcall k (* v n))))))
(fact-cps 10) ; ==> 3628800
Since the code didn't use several terms or the implicit progn i switched to if since I think it's slightly more readable. Other than that and the use of funcall because of the LISP-2 nature of Common Lisp it's the identical code to your Scheme version.
Here's an example of something you cannot do tail recursively without either mutation or CPS:
(defun fmapcar (fun lst &optional (k #'values))
(if (not lst)
(funcall k lst)
(let ((r (funcall fun (car lst))))
(fmapcar fun
(cdr lst)
(lambda (x)
(funcall k (cons r x)))))))
(fmapcar #'fact-cps '(0 1 2 3 4 5)) ; ==> (1 1 2 6 24 120)
EDIT
Where exactly in the Scheme example is the initial value for v coming
from?
For every recursion the function makes a function that calls the previous continuation with the value from this iteration with the value from the next iteration, which comes as an argument v. In my fmapcar if you do (fmapcar #'list '(1 2 3)) it turns into
;; base case calls the stacked lambdas with NIL as argument
((lambda (x) ; third iteration
((lambda (x) ; second iteration
((lambda (x) ; first iteration
(values (cons (list 1) x)))
(cons (list 2) x)))
(cons (list 3) x))
NIL)
Now, in the first iteration the continuation is values and we wrap that in a lambda together with consing the first element with the tail that is not computed yet. The next iteration we make another lambda where we call the previous continuation with this iterations consing with the tail that is not computed yet.. At the end we call this function with the empty list and it calls all the nested functions from end to the beginning making the resulting list in the correct order even though the iterations were in oposite order from how you cons a list together.
Is it required that lambda expressions only be used? Wouldn't a named
function accomplish more since you could maintain the state of each
continuation in a data structure which can be manipulated as needed?
I use a named function (values) to start it off, however every iteration of fact-cps has it's own free variable n and k which is unique for that iteration. That is the data structure used and for it to be a named function you'd need to use flet or labels in the very same scope as the anonymous lambda functions are made. Since you are applying previous continuation in your new closure you need to build a new one every time.
Is there in particular style/way of continuation passing style in
Common Lisp with or without all the macros?
It's the same except for the dual namespace. You need to either funcall or apply. Other than that you do it as in any other language.