Macros• You might recall from earlier that some of the statements

we use are not actually functions but macros– In CL, a function evaluates all of its parameters and applies

them to the function• if is a macro, not a function

– only one of the two clauses gets evaluated based on the condition

• let is a macro – the variables are not evaluated, only allocated and bound – the initialization code is evaluated but not the variables

• dolist and dotimes are not macros – the (var lis/value) is not a function call

• etc

– Macros are way of writing code that can be used to generate code rather than be invoked by code

– Macros are essential in order to “grow” the language• many programming languages have macro facilities, but few languages

use them as extensively as in Lisp

What is a Macro• A macro is a piece of code, much like a function

– but in the case of a macro, the code is not evaluated when entered into the REPL environment

– instead, it is evaluated on demand• The macro returns a form (code) rather than a value

– so that the code returned can be evaluated by someone else at a later time

• The macro can be defined so that it builds code– you can also pass it code to be used as it builds code– for instance, you might want to build a function by

appending pieces of code to a list and then returning the list when done

• So the macro allows us to postpone evaluation– defining a macro is somewhat like writing a function, you

use defmacro, include params and follow it by the code to be executed

Controlling Evaluation• The ’ is normally used to control evaluation

– We have used ’ to say “take literally, do not evaluate”• In a macro, we can continue to use the ’ but there is a

superior operator to use: ` (back quote)– The quote and back quote both say: “do not evaluate what

follows”• but the back quote permits the use of the , (comma) which means

“unsuppress the back quote”• the , will allow us to send a name to a macro definition and use that

name in place of the parameter’s name– Deciding when to use ` and , will be challenging

• if you try to use , outside of a ` you will get an error• you do not have to use , in your ` code but if not, then it is the same as

if you were using ’– Note that we could avoid using ` and , by using a combination

of list and ’ as shown on the next slide but this would get tedious

Back Quote and Comma Examples• `(the sum of 5 and 10 is (+ 5 10))

– since this is all in the scope of `, none of it is evaluated, instead it is returned verbatim:

• (the sum of 5 and 10 is (+ 5 10))• we get the same result if we use the normal quote

– if we place a , prior to (+ 5 10) we get• (the sum of 5 and 10 is 15)

• ` (like ’) says “do not evaluate, just return” whereas ` …,x... says “evaluate x right now”

• `(dolist (a lis) (print a))– returns (dolist (a lis) (print a))– assume that lis is the list ’(a b c d), then `(dolist (a ,lis) (print a)) returns

• (dolist (a (a b c d)) (print a))

• `(,x + ,y = ,(+ x y)) – if x = 3 and y = 4 then this returns (3 + 4 = 7)– Notice that we could achieve the same results by placing any of these lists

into a (list) function call and quoting each item in the list except those that have a ,

• (list ’the ’sum ’of ’5 ’and ’10 ’is (+ 5 10))• (list x ’+ y ’= (+ x y))

A More Useful Example• Consider that we want a function that will evaluate an expression

and return one of three values– First value if expression is negative– Second value if expression is zero– Third value if expression is positive

• This is equivalent to FORTRAN’s first if statement– if(expression) a, b, c

– We can’t use defun because all 3 expressions would be evaluated• We could use an if or cond statement though, but using a macro is easier

(defmacro if3 (expr val1 val2 val3) `(cond ((< ,expr 0) ,val1) ((= ,expr 0) ,val2) (t ,val3)))

If we call if3 as (if3 (- a b) ’+ ’0 ’-)this becomes (or unfolds into):

(cond ((< (- a b) 0) ’+) ((= (- a b) 0) ‘0) (t ’-)

You can see that (- a b) is substituted forexpr, this is called a macro-substitution

Multiple Assignment Example• Assume that you want to assign multiple variables to the same

expression– We can do this by (setf a expr b expr c expr …) but this is inefficient (we

are evaluated expr several times)– I could also do (let ((temp (expr))) (setf a temp b temp c temp) …) but this

can be tedious– What I want to do is use code like this (setf a b c … expr)

• setf however does not work properly, so instead I will define a macro

(defmacro assign2 (v1 v2 exp) `(let ((temp ,exp))

(setf ,v1 temp) (setf ,v2 temp) temp))

(defmacro assign-many (exp v1 &optional v2 v3 v4) `(let ((temp ,exp))

(setf ,v1 temp)(if (not (null ,v2)) (setf ,v2 temp))(if (not (null ,v3)) (setf ,v3 temp))(if (not (null ,v4)) (setf ,v4 temp)) temp))

Returns thevalue assigned

Up to 4 variablescan be assigned,only one is required

Notice the order, expcomes first

Variable Assignment Example• Here I want to choose which of two variables to assign to

the other– For instance, if a < b, then assign a to b otherwise assign b to a– Code: (if (< a b) (setf a b) (setf b a))– As a macro:

(defmacro assigntwoways (v1 v2 test) `(progn

(if ,test (setf ,v1 ,v2) (setf ,v2 ,v1)) ,v1))

Notice the use of progn here, it is needed because of the ,v1 at the endwithout `(progn, CL would think that the ` was only in front of the if statement meaning that the ,v1 has a , outside of a ` scope

Since this code unfolds into an if statement which executes a setf, v1 or v2will change – as opposed to writing a defun statement where we would haveto pass back v1 and/or v2 to be stored again

Writing a Macro• So far it doesn’t seem difficult to write a macro but

– how do you know when to use ` and , – how do you know what code to use– how do you know when to use progn or let

• Start by writing the code that you want to have produced in a given setting– (cond ((< (- x y) 0) x) ((= (- x y) 0) y) (t (+ x y)))

• Now make it into a function with parameters to replace the various items (the statements x, y, (+ x y) and the expression (- x y)) with variables

• Since the variables are not to be evaluated by the macro, but substituted when the macro is called, insert a , before them

• Since we have , in our code, we have to ` -- find a location where ` will cover the scope of all parameters that have a ,

• Replace defun with defmacro

Example• I want to write a do loop that will iterate while the loop

variable is positive– (do ((x init step)) ((<= x 0)) code)

• I want this code to be generated whenever anyone wants to use it by passing the variable, the init value, the step function, and the code

(defun while-positive (x init step code) (do ((x init step)) ((<= x 0)) code))

The defun has a flaw, x in the loop is not the same as thex passed as a parameter, I need macro substitution:

(defmacro while-positive (x init step code) `(do ((,x ,init ,step)) ((<= ,x 0)) ,code))

Call this with (while-positive y 10 (- y 2) (print y))

Using &Body and ,@• So far, the body of our construct (the executable code

inside the loop or if statement) has been a single statement– this approach does not work if we have lists of executable

statements• for instance, (while-positive x (+ x 1) (setf a (+ a x)) (print a)) does not

work because there are two items where the macro expects the single item code

– to accommodate multiple statements as the body, we could use &rest (all of the following code is placed into a single list of functions)

– instead, we use a variation called &body • the only difference is that &body pretty prints better than &rest

• We supply a single parameter after this which is then the collection of all items passed– inside of the ` scope, we can apply the body by

doing ,@parameter

Examples Using `,@ vs ’

Backquote Syntax Equivalent List-Building Code Result

`(a (+ 1 2) c) (list 'a '(+ 1 2) 'c) (a (+ 1 2) c)

`(a ,(+ 1 2) c) (list 'a (+ 1 2) 'c) (a 3 c)

`(a (list 1 2) c) (list 'a '(list 1 2) 'c) (a (list 1 2) c)

`(a ,(list 1 2) c) (list 'a (list 1 2) 'c) (a (1 2) c)

`(a ,@(list 1 2) c) (append (list 'a) (list 1 2) (list 'c)) (a 1 2 c)

Code Examples• A number of the operations we have been using are

macros by using ,@body:– (defmacro when (condition &body body) `(if ,condition (progn

,@body)))– (defmacro unless (condition &body body) `(if (not ,condition)

(progn ,@body)))– (defmacro dotimes ((var1 var2) &body body) `(do ((,var1 0

(+ ,var1 1))) ((= ,var1 ,var2)) ,@body))• Notice the extra layer of ( ) in the parameters to fit the dotimes form

(dotimes (a 10) …)

– (defmacro dolist ((var1 lis) &body body) `(do ((,var1 (car ,lis) (car ,lis))) ((null ,lis)) (setf ,lis (cdr ,lis)) ,@body))

• the @ is used in conjunction with , to say “evaluate the following as a group”

• Many implements of CL allow you to use &rest instead of &body but use &body just to play safe, it also helps when pretty-printing

Destructuring• Notice in some of the previous examples that we

wrapped our params in an extra set of ( )– When binding the formal parameters (those in the macro

header) to the actual parameters (those in the macro call), CL does something called destructuring – it finds the matching parameters based on embeddedness

• (defmacro dotimes ((var1 var2) &body body) `(do ((,var1 0 (+ ,var1 1))) ((= ,var1 ,var2)) ,@body))

• (dotimes (x 10) …) – var1 gets x, var2 gets 10

• Destructuring can be done to any arbitrary nestedness– consider some examples

• (defmacro m1 ((a b) (c d) (e f)) …) called with (m1 (1 2) (3 4) (5 6))• (defmacro m2 (a (b c) ((d) e)) …) called with (m2 1 (2 3) ((4) 5))• (defmacro m3 (a (b &optional c) (&optional d)) …) called with (m3 1

(2) ( )) or (m3 (1 (2 3) ( )) or (m3 (1 (2) (3))

More Macro Examples• Consider that we want to write a counter controlled loop

that iterates over the sequence of prime numbers– (do ((i 2 (+ i 1))) ((= i n)) (if (prime i) (b i)))

• b is the body of code to execute on each prime

• This becomes the macro– (defmacro doprimes ((var limit) &body b) `(do ((,var 2 (+ ,var

1))) ((= ,var ,limit)) (if (prime ,var) ,@b))

• Now consider the same thing over a sequence generated by some function f rather than prime– (defmacro dosequence ((var limit f) &body b) `(do ((,var 2

(+ ,var 1))) ((= ,var ,limit)) (if (funcall ,f ,var) ,@b)))• could be called by (dosequence (a 100 #’evenp) (print a))

– (defmacro dosequence ((var limit f) &body b) `(do ((,var 2 (+ ,var 1))) ((= ,var ,limit)) (if (funcall #’,f ,var) ,@b)))

• could be called by (dosequence (a 100 evenp) (print a))

Building Code From A List• Let’s make a simple switch statement in CL

– In Java/C, we do switch (var) { case val1 : statement; break; case val2 : statement; break; …}

– Here, we want to be able to say (switch x val1 statement1 val2 statement2 …)

• we don’t need to add the extra syntax of “case” and :, ; • to simplify matters, we will assume that each statement is a single

operation, and to make it even easier, merely a return value (say a number)

• we will also simplify the syntax by permitting the values and return values to be stated outside of a list

• (switch x 1 1 2 20 3 300 4 -1) – x = 1 returns 1, x = 2 returns 20, x = 3 returns 300, x = 4 returns -1

• Problem: we can’t just use ,@body here because we need to divide the body up into separate statements that say ((= x 1) 1) ((= x 2) 20) ((= x 3) 300) etc– So rather than relying on ,@body, we will build our body

through a do loop

Building Our Macro• It seems like we could write any function to build a cond

statement for us:– (let ((temp ’(cond)))– (do … (setf temp (append temp ’((= x 1) 1))

• However, we need to be able to insert the variable name provided by the caller as well as the unique list of values/return values– So a macro it is!

• In the macro though, we have to differentiate whether we are appending to a local variable, or manipulating a parameter– The former is done outside of the ` scope, the latter is done within the `

scope

(defmacro switch (var &body lis)(let ((temp '(cond)))(do ((i 0 (+ i 2))) ((>= i (length lis)))

(setf temp (append temp `(((= ,var ,(nth i lis)) ,(nth (+ i 1) lis))))))

temp))

Do lis elements in pairs, (nth i lis) beingthe value, used in (= var val) and thenext element being the return value

Notice this is the only part of the macro that has ` scope

An Alternative Approach

(defmacro switch (var &body b)(let ((temp '(cond)) (count 0) temp2 temp3 item) (do ( ) ((null b)) (setf item (car b)) (setf b (cdr b)) (if (evenp count) (setf temp2 `(= ,var ,item)) (progn (setf temp3 (append (list temp2) `(,item))) (setf temp (append temp (list temp3))))) (setf count (+ 1 count))) temp))

• The previous approach was concise but perhaps not as easy to understand as it could be– Here we more clearly build each switch argument as two separate lists,

one storing (= var item) and the other being an executable statement– We then tack each together into a list and then append it to the growing

cond statementWhen called with(switch x 0 (print ’a) 1 (print ’b) 2 (print ’c))unfolds into

(cond ((= x 0) (print ’a)) ((= x 1) (print ’b)) ((= x 2) (print ’c)))

Improving Our Macro• What if we want to permit a default?

– Since the default case should be the last in our list, we will exit our loop once we find the default

– Notice the use of `t – this permits a default case to be selected when default is found in the list

• we could alternatively use `default if we want the programmer to use the word default when calling the macro

– We could invoke this macro with (switch2 x 1 10 2 20 t 30)• if x were 2, we would get 20, if x were 5, we would get 30

(defmacro switch2 (var &body lis) (let ((temp '(cond))) (do ((i 0 (+ i 2))) ((>= i (length lis)))

(if (equal (nth i lis) `t) (setf temp (append temp `((t ,(nth (+ i 1) lis))))) (setf temp (append temp `(((= ,var ,(nth i lis)) ,(nth (+ i 1) lis)))))))

temp))

Macro Expanding• When you define a macro, you are returning a new form which can

be called directly – From the REPL interface– From another function

• But the macro is actually performing substitution by taking what you defined and substituting parameters– This is done through macro-expansion– What is returned is then executed

• You can view the intermediate form without execution by using macroexpand-1– (macroexpand-1 ’(macroname params))

• For example, I want to see the code generated by doprimes passing it the parameters (n 12) and (print n):– (macroexpand-1 '(doprimes (n 12) (print n)))– (DO ((N 2 (+ N 1))) ((= N 12)) (IF (ISPRIME N) (PRINT N)))– T

• Notice that macroexpand-1 prints out the macro as expanded by the parameters, and then returns T (or nil if there is a failure)

Using Macros to Define Functions• To this point, our macros have expanded into function

calls and the return of the function call is what the macro call returns– consider switch which expands to a cond statement that is then

executed• We can also use macros to generate defun statements

– this lets us generate functions to be called later• Consider a database using a list of structures

– we could define search functions for all of our slots, but this could be time consuming

– or we could define a macro that generates various search functions for us

• How?– we need to have access to all of the structures’ slots and their

types, for each slot, we generate one or more defun statements from our macro

Starting The Macro• Consider that we have a student structure which stores

the slots: name, major, hours, gpa of types string, string, integer, number– We want to define search functions similar to the following:

– However, we don’t want to have to write them all• So we want a macro which will generate defun statements• This macro will need to know the names of the slots

– in this way, we can access the structure’s slot as we did above with (student-hours a)

• And the type of each slot– so that we know which comparison functions to generate (an integer will

probably require <, >, and =, but a string might include others such as contains, does-not-contain,

(defun search-hours> (db target)(dolist (a db)

(if (> (student-hours db) target) (print a))))

Partial Macro Solution(defmacro generate-a-search (struct-name slot-name slot-type) (let ((function-name (intern (concatenate 'string

(symbol-name struct-name) "-" (symbol-name slot-name) ">")))

(compare-type (if (or (equal slot-type 'number) (equal slot-type 'integer)) #'> #'String>))

(accessor-name (intern (concatenate 'string (symbol-name struct-name) "-" (symbol-name slot-name)))))

`(defun ,function-name (db target) (dolist (a db) (if (apply ,compare-type (,accessor-name a) target)

(print a))))))

Now, we call this for each slot-name/slot-type in the structureExample: (generate-a-search STUDENT MAJOR STRING)We have to expand on this if we want to go beyond simple >, <, = searches

Continued• Let’s go beyond the search producing macro

– We would like to pass a structural definition to a macro, it can produce for us an entire database

• The struct definition• Functions for search• A function to print the structure in a formatted way• A function to test two structures for equality (equal if they have the same slot

values)• A function to input a new structure

– Etc

– The structural definition should include the slot names, but here we might also want to include each slot’s type

• we can decide to what search functions to produce (e.g., a search based on whether a string contains a substring rather than simply string<, string>, string=)

• we can use the proper form of equality (=, equal, char=, string=, etc)• we can generate a proper format statement for an output function

– On the next slide, we have a macro that produces the defstruct and an equal’s function

• Notice the weird use of `,variable– This became necessary in order to pass parameters properly

Example Continued(defmacro generate-struct (name slots types) (let (slot type temp1 temp2 fname (s1 (gensym)) (s2 (gensym)) andlist eq-func) (setf fname (intern (concatenate 'string (symbol-name `,name “-equals”)))) (setf andlist '(and)) (dotimes (i (length slots)) (setf slot (nth i slots)) (setf type (nth i types)) (if (or (equal type 'integer) (equal type 'number)) (setf eq-func '=)

(if (equal type 'character) (setf eq-func 'char=) (setf eq-func 'string=))) (setf temp1 (list (intern (concatenate 'string (symbol-name `,name) "-"

(symbol-name `,slot))) `,s1)) (setf temp2 (list (intern (concatenate 'string (symbol-name `,name) "-"

(symbol-name `,slot))) `,s2)) (setf andlist (append andlist (list `(,eq-func ,temp1 ,temp2)))))

;; andlist becomes (and (equal (slot-name1 s1) (slot-name1 s2)) …) for each slot`(progn (defstruct ,name ,@slots) ;; produce the defstruct (defun ,fname (,s1 ,s2) ,andlist)))) ;; produce the equal function

A Problem• Consider the following macro definition:

– (defmacro foo (a b) `(let ((sum 0)) (do ((,a 1 (+ 1 ,a))) ((= ,a ,b)) (setf sum (+ sum ,a))) sum))

• This macro merely constructs a do loop that iterates from 1 to b, summing up the values and returning the sum – obviously we don’t need a macro for this, but it illustrates an important point

• No big deal right?– If we were to do (foo a 10) then it sums up 1-9 and returns 45

as we would expect– What do you suppose will happen if we do (foo sum 10)?

• Let’s macro expand this (foo sum 10) and find out:– (LET ((SUM 0)) (DO ((SUM 1 (+ 1 SUM))) ((= SUM 10))

(SETF SUM (+ SUM SUM))) SUM)– Notice our terminating condition is not what we had expected

– instead of stopping once the loop variable reaches 10 because our loop variable’s name is replaced with sum

– And the problem here is that (= sum 10) is never true (sum takes on the values 1, 3, 7, 15, 31, …

The Problem: Same Named Items• The problem with our last example was that a parameter

that we passed to the macro to substitute for ,a happened to be the same name as a local variable– What are the odds of that happening?– Probably not very high, but if it does happen, it produces

behavior that we don’t want

• The solution?– Renaming the parameter isn’t an adequate solution because we

can’t control what the other programmers might do • and since we can’t predict what they will do, we need to play safe

– Renaming the local variable would work if we could come up with a name that we are guaranteed that they will never ever use

• how about Slartibartfast??? Ok, someone out there might be compelled to name a variable this!

The Solution: Gensym• Gensym is a CL function that generates a unique

symbol, one that is not used elsewhere in the system– These symbols will differ from user defined symbols

because they have #:G generated followed by a unique number as in #:G708

– Using gensym several times in a row will generate several symbols with consecutive integer values (so that the next would be #:G709)

• This does not seem very usable – what good is it to generate a seemingly random symbol?– Its use is this: we will name any local variable using

gensym so that the name of the local variable is unique

(let ((sum (gensym))) … now for the remainder of the macro, refer to ,sum instead of sum, sum is the parameter passed to the macro, ,sum is a symbol like #:G708 – there is no way that a programmer can pass #:G708 it is disallowed, therefore we are guaranteed a unique name

Another Example• In this example, there doesn’t seem to be a problem since

we are not declaring local variables– (defmacro cube (n) `(* ,n ,n ,n))– This works fine when called with (cube 2) or (cube x) where x

was set to some value previously or even (cube (- x y)) where x and y were both previously set

• But what happens if we call cube as (cube (incf x))?– Let’s macroexpand this:

• (macroexpand-1 ’(cube (incf x))) (cube (incf x) (incf x) (incf x))– Recall that incf is destructive

• the value of x changes every time it is called• so if we originally set x to 2, then we get (cube 3 4 5) 60!

• To get around this, lets again use gensym– (defmacro cube (x) (let ((name (gensym))) `(setf ,name x)

(* ,name ,name ,name)))

Revised Macro

• Here we have pulled the let statement out to appear prior to the `– This allows us to generate a symbol for sum, and then reference that

generated symbol so that we don’t have to take the chance that the programmer called foo with sum

• If we were to macro-expand (foo sum 10) now we get:– (PROGN (SETF #:G829 0) (DO ((SUM 1 (+ SUM 1))) ((= SUM 10))

(SETF #:G829 (+ #:G829 SUM))) #:G829)– Notice how #:G829 has been generated to replace our local variable so that

our terminating condition no longer is affected by that variable

(defmacro foo (a b) (let ((sum (gensym)))

`(progn (setf ,sum 0) (do ((,a 1 (+ ,a 1))) ((= ,a ,b)) (setf ,sum (+ ,sum ,a)))

,sum)))

Do We Need A Macro?• Consider this macro:

– Do we really need to make it into a macro? Could we not just use defun?

– Yes, just use defun – so under what circumstance(s) do we want to use defmacro?

• Both defun and defmacro generate new pieces of code but defmacro gives us the ability to control when things are evaluated so that we can make use of macro substitution– If at the time I am writing the code, I do not know how the

code will be used by another programmer, I should use defmacro

– Alternatively, if I have a routine that depends on another function (like dosequence or doprimes), I could write a function which is passed the function to apply using funcall

(defmacro is-greater (x y) `(> ,x ,y))