1.1 Datastar and Common Lisp for web development
================================================
2025-08-17
So you're living under the gun
Circumstances have you on the run
A doctor frowns, you feel bad
Take this, you've just been had!
Don't you lose it, now listen to us
Everything's going to be all right
Take a break, take some time
Everything's going to be all right
Don't you lose it, remember to take
Time out for fun.
-- _Devo, Time Out for Fun, 1982_
Doing front-end development with state in the back-end, using
whichever language one prefers, and specifically Common Lisp, while
being reactive and scalable: if doing this is wrong, I don't want to be
right, and Datastar seems almost too good to be true.
1.1.1 The demo
--------------
Let's start by going directly to what this is all about: Datastar (1) is
an 'hypermedia framework' that uses server-side rendering along with a
(very light) front-end library. There are several SDKs, including
Clojure, but I wanted to see if it worked with Common Lisp, and long
story short, it does.
This small demo is the outcome of my curiosity: a minimal example of
a dynamic website made with Datastar and a Common Lisp backend (2),
based on the 'click this button' example in the Datastart guide.
(Codeberg repository (3))
[Datastar and Common Lisp ]
Datastar and Common Lisp (4)
Read the Datastar guide (5) for more information, this is nothing
more than an example of how it works when using Common Lisp, with a
couple of functions/macros that can be thought as an initial SDK.
1.1.2 What and why
------------------
As I work on See Budotree (and there's a lot of new ideas floating
around to make it bigger and better), I've continued my descent into
'frontend development'. I've started with vanilla JavaScript, and it
was fun enough, and then I started looking into the more established
ecosystem. I never had to actually worry about it before (that I wrote
a Texinfo blogging solution from scratch: Texiblog is a good indication
of of this...), but I tried things with an open mind, like I often do...
So much so that I now have hundreds of lines of code using React, MUI,
Vite, and some other things I can't remember, and truth be told that
from a user perspective, it looks OK.
[The Budō Lineage Tree React version]
The Budō Lineage Tree React version
It's not fun though.
I've had more fun in the couple of hours I've spent in this small
demo with Datastar then when doing the above. This is largely my fault
and a consequence of my own limitations and preferences, but even the
filesystem organisation bothers me, everything seems to be optimised for
complexity, and the urge to just ask some LLM to do most of the stuff
ends up being overwhelming - and that's the first step to completely
lose the plot and start a read-only project.
Yes, I know there are alternatives. Yes, I've tried them. No, it
didn't make a difference.
1.1.3 Datastar and hypermedia
-----------------------------
One of the good things of having everything in the client is not even
technical: it's cost. I can share my static content and leverage the
browser. This is very compelling and has been tremendously useful.
However, as I try to add more features, it starts to be insufficient,
and once that happens, a new world of additional options needs to be
navigated: edgeless functions, SSR, application servers. There's
nothing wrong with that, of course, but it does open a door for
reevaluating my options.
This is how I found Datastar. With the assumption that I would need
to have a back-end, why not try to actually enjoy the process? I had
stumbled on Alpine.js before (and Preact, and Solid.js, and...), and had
read a bit on HTMX, but I got to Datastar since I found it while
searching for web development with Common Lisp: the creator of Datastar
mentioned Lisp (6) a couple of times, and catching that was enough to
pick my interest. (7) This video on real-time hypermedia (8) by
Delaney is a very interesting watch and sets some of the major
principles in which Datastar rests.
Datastar already has a Clojure SDK. Clojure is not new to me:
Clojure-MQTT, but if I really want to have fun, well, it's not Common
Lisp. That Datastar had Clojure support (and from what I gathered, a
significant one) was however not irrelevant, since Clojure is a Lisp (9)
1.1.4 What I did, and what was used
-----------------------------------
This demo implements the button with SSE events, and in reality the
Datastar specific parts are the smallest parts of it: we just need to
send SSE events, which means that the bulk of the work (although still
quite simple) is done at the web server level, specifying 'send an
event-stream reply with the appropriate format, headers, and using this
format as payload'.
For that, I've used this macro, which actually does a bit more than
it should (the ‘let’ sets Spinneret options to keep the HTML output in a
single line, something that is not related with SSE directly).
(defmacro with-sse-response ((stream-var) &body body)
"Set up SSE headers and bind STREAM-VAR to a UTF-8 SSE stream. Based on
https://github.com/edicl/hunchentoot/issues/175#issuecomment-585040829,
but without detaching the socket."
(let ((*html-style* :tree)
(*print-pretty* nil)
(*always-quote* t))
(setf (content-type*) "text/event-stream; charset=utf-8")
(setf (header-out "Cache-Control") "no-cache")
(setf (header-out "Access-Control-Allow-Origin") "*")
(let* ((raw-stream (send-headers))
(,stream-var (flex:make-flexi-stream raw-stream :external-format :utf-8)))
,@body)))
The main components used are:
“Hunchentoot (
https://edicl.github.io/hunchentoot/)”
One of the most popular Common Lisp webservers. There are options,
but I wanted to keep this as simple as possible, so Im not using
Clack or any additional layer.
“Spinneret (10)”
An HTML generator that created human-readable HTML from lists.
“css-lite (11)”
Does for CSS what Spinneret does for HTML.
“SBCL (12)”
Common Lisp compiler; one of several possible choices since I'm not
using anything that depends on non-CL libraries.
The code itself is in a single file (not counting the package
definitions) on purpose, I personally find it easier to learn new things
when I don't have to visit lots of different files. For a more robust
usage I will almost surely split things in different ways, with handlers
separated from webserver start, with the HTML and CSS in different
files, etc.
Deployment-wise, I'm not expecting issues with this, I've tested it
with thousands of concurrent users and it didn't made a dent. There's
something to be said about the real needs of websites and the idea that
most things need a 'service' that 'scales': serving HTML is not that
hard for a server to do.
1.1.5 How it works
------------------
1.1.6 Where to go from here
---------------------------
By itself, this example doesn't do a lot, but for me it opened the door
to a different way to create a reactive website. I'll likely add more
and more as I explore it, and as I do I've found that there's a lot of
things that I can go 'back to basics' on: I've had some difficulties in
completely understanding the entire CSS stack used, especially when
things like Tailwind and then used to build more 'semantic' frameworks,
different things piled on top of each other. The realities of
reactive/mobile-first design is one of the reasons behind the apparent
complexity, but using Datastar has allowed me to peel at least one of
the layers.
Give it a try: even if you're not using Common Lisp, it is a polished
framework, done by some pretty smart people that have poured their
experience and opinion on it, and for me that's a big advantage.
---------- Footnotes ----------
(1) Datastar (
https://data-star.dev/)
(2) a minimal example of a dynamic website made with Datastar and a
Common Lisp backend (
https://datastar.interlaye.red/)
(3) Codeberg repository (
https://codeberg.org/fsm/cl-datastar-demo)
(4) Datastar and Common Lisp (
https://datastar.interlaye.red/)
(5) Datastar guide (
https://data-star.dev/guide/getting_started)
(6) mentioned Lisp
(
https://x.com/DelaneyGillilan/status/1935673128384110756)
(7) This is an important point because my choice has not been
primarily motivated by benchmarks or anything of the sort, but more due
to having landed on it while looking for a different way to do things,
having read the documentation, seeing a couple of videos, and feeling it
made sense. The author, Delaney Gillilan, is outspoken about why he
thinks Datastar makes sense, and why the framework works the way it
does, and so far it's been pretty convincing.
(8) video on real-time hypermedia
(
https://www.youtube.com/watch?v=0K71AyAF6E4)
(9) There's an entire discussion about what this means that I will
sidestep. Clojure has been very useful in opening up the Java
ecosystem, enabling the use of a member of the Lisp family instead of
Java itself, and that is not an insignificant benefit.
(10) Spinneret (
https://github.com/ruricolist/spinneret)
(11) css-lite (
https://github.com/paddymul/css-lite)
(12) SBCL (
https://www.sbcl.org/)
