---
layout: ../Site.layout.js
---
# C++ and embeddable common lisp habitat example ~ base64 encoding

[Habitat being jeremy_list's HaikuOS secure scuttlebutt C++ gossip protocol client](https://codeberg.org/jeremylist/habitat). Of course, I use ANSI CL and not C++, and openbsd or maybe debian and not haikuos. So let's see if we can just make the C++ bits into lisp functions in that magical [embeddable common lisp](https://ecl.common-lisp.dev/) way.

Here we continue directly from [making the conventional BSD port of `ecl` embeddable common lisp `--with-cxx`](/fundamental/cxx-ecl-openbsd-port). I'm aware that at least the [gentoo linux distribution has a C++ ecl port](https://packages.gentoo.org/packages/dev-lisp/ecl) (in gentoo, called an `ebuild`).

## Set up common lisp / eev / emacs

`#P"~/ecl-habitat-test/eat-base64-habitat.ecl.lisp"` was my working file. This is the adaptation of eev's `M-x eeit` generated infrastructure I have found myself using.

```
#|
• (setq inferior-lisp-program "ecl")
• (slime)
• (setq eepitch-buffer-name "*slime-repl ECL*")

(compile-file "~/ecl-habitat-test/eat-base64-habitat.ecl.lisp" :load t)
|#
```

`compile-file` in embeddable common lisp provides a very nice low-level-ffi, sffi into either C or C++ (whatever `ecl` had been built with (`--with-cxx` or not)).

NOTE all of ecl's `sffi` stuff is `compile-file`-only. So we have to extract the `ffi` of this article into a lisp file and call `compile-file` .. `:load t` as indicated above. The interpreter doesn't know how to compile C (it only interprets lisp).

## Taking C++ `base64::encode1` from habitat's source

I eventually came to terms with the fact that habitat used a nonportable HaikuOS C++ string library. Instead, I pulled out the base64 encoding source C++ namespace fragment. I promoted the enums and integer-like types to ints, though ecl's sffi directly supports all these common C/C++ types, but they are a bit extraneous to this minimal ecl C++ sffi example.

(Edit: Simple fix mentioned everywhere else right after publishing) Actually, from what I can tell, at least my extracted version of b64 encoding here doesn't work, but the C++ embeddable common lisp works which is our focus. [jeremy_list](https://thunix.net/~jeremylist/) will take a look once this article is up and point out where I went wrong here.

```
(ffi:clines "
namespace base64 {

enum Variant {
 STANDARD,
 URL,
};

int
encode1(int byte, int variant)
{
 if (byte <= 25) {
   return byte + 'A';
 } else if (byte <= 51) {
   return byte - 26 + 'a';
 } else if (byte <= 61) {
   return byte - 52 + '0';
 } else if (byte == 62) {
   if (variant == STANDARD)
     return '+';
   else
     return '-';
 } else {
   if (variant == STANDARD)
     return '/';
   else
     return '_';
 }
}

}
")
```

## A lisp defun whose body is that C++ function

`ffi:defentry` ([sffi docs](https://ecl.common-lisp.dev/static/manual/Foreign-Function-Interface.html#SFFI-Reference)) attaches a C/C++ function as body to a common lisp function. This is just what the form is like.

```
(ffi:defentry c-encode1
   (:int :int)
 (:int |base64::encode1|))
```

`ffi:defentry`'s arguements are

1. name (so the lisp function will be named `c-encode1`)
1. C/C++ argument types (I just promoted them to ints - if you want to [you can get the underlying types and enum right](https://ecl.common-lisp.dev/static/manual/Foreign-Function-Interface.html#Primitive-Types))
1. `(return-type c-or-c++-function-name)`

## A common lisp stab at b64 encoding using that

In one big common lisp loop sorry. I just tried to literally match jeremy_list's code rather than refer to openbsd's b64encode myself. So I am not sure what translation mistake has crept in: Anyway, the function 'works' for what it is it happens to do.

```
(defun b64encode (bytes &optional (variant 0))
 (loop
   :with mask6 := #b111111

   :for n :from 0 :by 4
   :for byte-1 :in bytes :by #'cdddr
   :for byte-2 :in (cdr bytes) :by #'cdddr ; I forgot this cdr
   :for byte-3 :in (cddr bytes) :by #'cdddr ; and cddr originally here.

   :for result-1
     := (c-encode1 (ash byte-1 -2) variant)
   :for partial-1
     := (logand (ash byte-1 4)
                mask6)

   :for result-2
     := (c-encode1 (logior partial-1
                           (ash byte-2 -4))
                   variant)
   :for partial-2
     := (logand (ash byte-2 2)
                mask6)

   :for result-3
     := (c-encode1 (logior (ash byte-3 -6)
                           partial-2)
                   variant)
   :for result-4
     := (c-encode1 (logand byte-3 mask6)
                   variant)

   :nconcing
   (list
    result-1 result-2 result-3 result-4)
     :into b64s
   :finally
      (return
        (coerce
         (mapcar 'code-char b64s)
         'string))))
```

## Seeing that working ("working")

```
#|
• (setq inferior-lisp-program "ecl")
• (slime)
• (setq eepitch-buffer-name "*slime-repl ECL*")
(load "eat-base64-habitat.ecl.lisp")

"abcdefghijkl"
;; "abcdefghijkl"
(coerce * 'list)
;; (#\a #\b #\c #\d #\e #\f #\g #\h #\i #\j #\k #\l)
(mapcar 'char-code *)
;; (97 98 99 100 101 102 103 104 105 106 107 108)
(b64encode *)
;; YWJjZGVmZ2hpamts ; now works.
;; "YWFhZGRkZ2dnampq" ; When I forgot those cdrs.
```

Well, this (Edit: Now works, simple typo above) seem[ed] wrong viz openbsd [b64encode(1)](https://man.openbsd.org/b64encode):

```
• (eepitch-shell)
printf "abcdefghijkl" | b64encode lowercase
;; " | b64encode lowercase
;; begin-base64 644 lowercase
;; YWJjZGVmZ2hpamts
;; ====
|#
```
Edit: Now it's working. Forgot three `cdr`s.

Formerly: Though it seems oddly close doesn't it. But it's difficult for me to guess what has gone wrong.

Still, the C++ program code was made available in common lisp. So that has gone right.

# Conclusions

We saw C++ embeddable common lisp compile and connect common lisp to jeremy_list's original C++ namespace using its `sffi` (which is `ecl`; not the external `libffi` - I have met people who just guess only libffi exists), and used it to write a portable lisp version of the base64 encoding rather than a nonportable haikuos C++ one. So we can straightforwardly salvage other projects as starting points; especially C++ programs and libs.

A simple sanity check with openbsd shows something like 2/4 bytes seem to agree with openbsd's b64encode implementation. Edit: This turned out to be a simple fix.

Threeish future implications being ecl inhabiting a C++ program to serve [McCLIM application-frames](https://mcclim.common-lisp.dev/main.html) into the running C++ program ; relatedly, starting a swank server inside a pre-existing C++ program and [slime-connecting](https://slime.common-lisp.dev/doc/html/) into the running program from emacs ; lastly, and it's been on my todo-list a while - using the [enzyme library](https://enzyme.mit.edu/) for automatic differentiation and friends with the previous, especially together with [my Leonardo system](/lispgames/LCKR-object-oriented-simulation-simulation/).

# Fin.

[Talk on the Mastodon as always please](https://gamerplus.org/@screwlisp/114963441276577775).

Oh! One hour from right now 8am Zulu Sunday I'm going to have a peertube live with [Kasper Galkowski](https://itch.io/profile/kgal) about common lisp indie gamedev with opengl, which is at least an ffi adjacent topic. Watch the Mastodon, again please.

[Edrx](https://anggtwu.net/) has been working on emacs eev-mode [kitten](https://kitten.small-web.org/) and has one [for me personally](https://anggtwu.net/elisp/screwlispkitten.el.html) I need to work through - this will shape what these articles are like going forward.

You are viewing proxied material from gopher.someodd.zip. The copyright of proxied material belongs to its original authors. Any comments or complaints in relation to proxied material should be directed to the original authors of the content concerned. Please see the disclaimer for more details.