Using Common Lisp in Emacs
Using Common Lisp in Emacs [LWN.net]


This article brought to you by LWN subscribers

Subscribers to LWN.net made this article — and everything that
       surrounds it — possible.  If you appreciate our content, please
       buy a subscription and make the next
       set of articles possible.



Lisp
is one of the oldest programming languages still in use today, but it has 
evolved in multiple directions over its more than 60-year history.  Two of
the more prominent descendants, Common Lisp and Emacs Lisp (or Elisp),
are fairly closely related at some level, but there is still something of a
divide between them.  Some recent discussion in the emacs-devel mailing
list have shown that some elements from Common Lisp are not completely
welcome in 
Elisp—at least in the code that is maintained by the Emacs project itself. 



The discussion goes back to at least mid-September when the subject of
keyword
arguments, which are used extensively in Common Lisp, came
up in 
another context; at that time, Richard Stallman pointed out
that, while there is some amount of Common Lisp compatibility available for
Elisp, it is "not supposed to
be used a lot". Alfred M. Szmidt concurred,
noting that useful pieces of Common Lisp can be adopted, but: "There is
no need to make 
Emacs Lisp complicated for the sake of compatibility with Common Lisp."
Meanwhile, Emacs maintainer  Eli Zaretskii quantified the
situation: "466 out of 1637 Lisp files in Emacs require cl-lib (some of them only
during compilation, i.e. they use only the macros)."  The Elisp "cl-lib"
library (formerly "cl", which is deprecated, as Emacs regularly tells me)
provides various compatibility macros and functions for those who need or
want to use 
them in Emacs.



The conversation mostly dropped for a month, but Stallman picked it up
again in mid-October; he was trying to clarify how many uses of cl-lib in
Emacs were problematic.  Using cl-lib for its macros, which only happens
at compile-time, is not really much of a problem in his view, but run-time
uses of the library may be more so. As he said in 2003, Stallman dislikes Common Lisp; he does "not have a very high
opinion of many of the decisions that were made in Common Lisp",
keyword arguments in particular.



There are, Zaretskii reported, 226 run-time
uses, which Emanuel Berg converted to
percentages: 28% of Elisp files use cl-lib, 15% at compile time, and 14% at
run time.  Stallman said that he is most concerned with anything that might
cause cl-lib to be loaded when Emacs is started without any customization
(as when using the -Q command-line option). 



Alan Mackenzie tried to
track down how many uses of cl- symbols there were in each of
the Elisp 
files,  suggesting that it would be a huge chore to remove all of them
"because there are enough contributors who 
think that cl-lib is just an ordinary part of Emacs to be used freely
without restraint", though he does not count himself among them.
Zaretskii questioned the numbers,
but said that some uses of cl-lib are likely unavoidable:

We cannot possibly expect people to contribute code if we force them
not to use the macros they are used to.  If cl-lib is not loaded as
result, that is good enough for us, I think.




Berg wondered why it
mattered whether cl-lib was not being loaded under some circumstances;
"I think it is pretty clear that 
cl-lib is used quite broadly, probably because people find
it useful."  Zaretskii replied that it matters
because the project decided not to load cl-lib in "vanilla
Emacs" because of "bloat,
unnecessary namespace pollution, etc."  While cl-lib is not loaded for
"emacs -Q", Berg noted that it is
likely loaded soon after Emacs starts, since the debugger, native
compilation, and many other Emacs packages rely on cl-lib.  Zaretskii was
unfazed by that:

What matters to us is that this unnecessary stuff is not loaded in
"emacs -Q", i.e. is not dumped and preloaded into the bare
uncustomized Emacs session.  This lets users start from the leanest
possible Emacs, then load what they need, without being "punished" by
loads of stuff they never need or want.

For Emacs to preload unnecessary stuff is unclean, and we try to avoid
that uncleanliness.



There were arguments made that, these days, cl-lib was just an expected
part of the Emacs environment, so perhaps it should be considered for
preloading as well.  As might be guessed, that did not really go far.  In
fact, the current long thread is an offshoot of one where switching Emacs
to use Common Lisp was suggested as a way to reduce the C footprint of the
editor.  That went over even worse, unsurprisingly.



The main objection is that developers have to learn the Common Lisp
functions and such in order to understand code that uses them.  While
Zaretskii sees no
problem with developers using cl-lib in their own code or in Emacs
extensions, he does not want to see it used in the vanilla Elisp code that
gets preloaded when Emacs is started.  Some see that as an ideological
move, but Zaretskii said that it is simply a maintainability choice that
the Emacs maintainers have made.



Stallman would like to
go further, however; he sees the use of cl-lib by the debugger and
native compilation 
extensions as problems to be solved:

Let's fix the debugger, and native compilation, not
to load cl-lib. 

It's not so bad if you load a specialized package that serves your code
and it uses cl-lib.  But the debugger can be loaded to debug anything,
even programs you have nothing to do with.  It's rude for the debugger
to load cl-lib.

Likewise for native compilation.  You might recompile anything.




Zaretskii, however, does
not see those uses as problems at all (for 
example: "The debugger uses cl-lib for good reasons.").  He
said that he does not see a reason to "waste our resources" on an
effort of that sort.  Stefan Kangas concurred:
"let's focus our efforts on fixing real problems".



Stallman did
a bit of digging into some of the entries in Mackenzie's list, finding
a false positive for byte-run.el, but also that the abbrev
minor mode uses the cl-pushnew
macro defined in cl-lib. "A generally useful minor mode as abbrev.el
should not load cl-lib."  He suggested that the definition of
cl-pushnew should move 
elsewhere so that it does not cause cl-lib to get loaded.  Then the other
entries 
on the list should be investigated and fixed as well.  After that:

Once we fix them all, how about if we add a regression test
to verify that various packages anyone might load at any time
do not use cl-lib.  With that, bugs like this would get caught
right away.




Zaretskii pointed out
that cl-pushnew is simply a thin wrapper around another function
that is defined in cl-lib, so moving it would not stop the loading of the
library. Meanwhile, a regression test would be tricky to write "since
our test suite 
infrastructure, ert.el, itself uses cl-lib extensively".  Stallman came up
with a different way to write the code that uses cl-pushnew,
but it was more verbose (and repetitive)—opinions varied on whether it was
truly an 
improvement.



But these days, Zaretskii said, it is pretty much a
requirement for Emacs maintainers to "be familiar with the cl-lib and cl-macs
functionalities" and that it is not rocket science to do so.  Mackenzie strongly disagreed that it
was straightforward to pick up that knowledge:

I simply don't have the brain power to
memorize such a morass of arbitrary, poorly named, poorly documented
stuff. [...] I frequently
spend, perhaps, half of my debugging time trying to understand what some
cl-* does rather than concentrating on the problem to be debugged.




It is this added complexity that Stallman was trying to avoid when he
designed Elisp, 
Bob Rogers said;
"he sees cl-lib.el as a trojan horse that is
changing Emacs Lisp".  Stallman agreed with
that, noting that he was under the impression that cl-lib was simply meant
as an optional extension, which is fine, but that he objects to it becoming
essential, which is where things are heading.

There is no sharp line between the one and the other, and no simple
test to determine which of those two our current situation actually
is.  Rather, there is a spectrum that runs from "CL [Common Lisp] support is
available but you can ignore it" to "the CL constructs are an
essential and unavoidable part of Emacs Lisp."

The specific practical questions I've asked are efforts to evaluate
where we are now along that spectrum.  Of course, the answer to that
isn't precise either.  But I was very surprised to learn how far
Emacs has gone towards the latter end.




There are other libraries that add complexity to Elisp, however, so João
Távora wondered
what was special about cl-lib in this regard:

Then why this laser-focus on cl-lib.el?  Why not criticize the use of
seq.el and map.el and pcase.el all of which "add many functions, which
have many details"?  Why are these libraries seemingly exempt from this
discussion?




He followed
that up with some further analysis of the difference in the way cl-lib
is treated; there are other additions to Elisp that have been adopted along
the way, some of them to the point of being preloaded (such as the
seq.el 
library for sequence
functions) these days.



Part of the difference is that Stallman truly
dislikes the prevalence of keyword arguments in Common Lisp, which is
something that he does not want to propagate into Elisp.  Some discussion of
adding an Elisp pushnew to replace cl-pushnew made it
clear that he did not
want an Elisp version to support the latter's keyword 
arguments, thus it would not be a drop-in replacement.  There are, Stallman
said,
some places where keyword arguments might make sense in Elisp, such 
as for sort,
but that simpler functions and macros should not add them.



The proposed change to the abbrev minor mode, which would avoid
cl-pushnew, is not something that Zaretskii sees as needed, though
he would not object if "the result is clean, tested, and is not horribly
complicated".  There is "nothing wrong with loading cl-lib when its
facilities are put to a 
good use".  Stallman disagreed
and said that he has "decided to start fixing some of those files not to
use the 
run-time CL features"



To an outsider, the push to avoid cl-lib
(and Common Lisp, in general) in Emacs seems a bit strange and the "rules"
governing its use seem fairly arbitrary—not even the creator and the
current maintainers can agree.  There is obviously a balance to be struck,
for maintainability and for reducing the amount of Lisp constructs that
Emacs hackers need to know, but one has to wonder if that balance has
tipped too far in either direction.  Stallman (and some others) clearly
seem to think it has, 
but Common Lisp has a fair share of Lisp hackers as well—many using Emacs
as their development environment—so the situation is a little puzzling, at
least to this neophyte (Common) Lisp hacker.

            