Continuous Integration of Emacs Packages with CircleCI
I was very inspired by Damien Cassou's great presentation during EmacsConf 2019 to write this post and I encourage you to check it out if you haven't already. In short, when writing packages for Emacs, it is best practice to run several quality tools on them, like syntax and documentation checkers, or even ERT Tests. But once these packages are public and pull requests start coming in, it is a huge time saver to have these same tools ran automatically and provide feedback to contributors. That's right, we're talking about Continuous Integration for Emacs packages.
There are many CI/CD services out there that can help us out. Damien Cassou shows examples with Travis, Gitlab and Drone, but I wanted to focus on CircleCI. It's far from perfect but their free tier is decent, their markdown badge is pretty, and I wrote a CircleCI Magit extension to show the build status in Emacs (ie. an extension for an Emacs extension).
We want a few things to happen automatically:
- install package dependencies from Elpa/Melpa/Org
- run ERT tests
- byte compile
- lint package
To keep things simple, I will use my package kubel as the guinea pig here.
Leaving out the actual commands, we can start with a simple draft of the .circleci/config.yml that uses the docker image for Emacs 27.1 as our base:
version: 2.1
jobs:
build:
docker:
- image: silex/emacs:27.1
working_directory: /kubel
steps:
- run: apt update && apt install -y git ssh
- checkout
- run:
name: Install packages
command: TBD
- run:
name: ERT tests
command: TBD
- run:
name: Compile
command: TBD
- run:
name: Lint
command: TBD
Note that the silex/emacs image doesn't come with git and ssh installed, and that we need to apt install them first thing as they are required to run the code checkout step.
Let's translate the desired run steps into actual Emacs Lisp Code for Kubel:
- my package dependencies are transient, dash, yaml-mode and s
(let ((my-packages '(transient dash yaml-mode s)))
(add-to-list 'package-archives '("melpa" . "https://melpa.org/packages/") t)
(package-initialize)
(package-refresh-contents)
(package-install 'package-lint) ;; we'll need it later
(dolist (pkg my-packages)
(package-install pkg)))
- run ERT tests in
test/kubel-test.el (load-file "kubel.el") ;; load code first
(load-file "test/kubel-test.el")
(ert-run-tests-batch-and-exit)
- simple byte compile with warnings not counting as errors
(setq byte-compile-error-on-warn nil)
(batch-byte-compile)
- run
package-lint, this time with warnings counting as errors (require 'package-lint)
(setq package-lint-batch-fail-on-warnings t)
(package-lint-batch-and-exit)
This is probably the biggest hurdle: how to run Emacs Lisp Code from the command line in an elegant manner? There are quite a few solutions out there to help with this: Damien Cassou's fancy Makefile, collections of bash script, and plenty of others. However I am not completely convinced by any of them as I find the syntax always a bit clunky and hard to read. I am currently settled with putting all the Emacs Lisp code into a single make.el file, and call the individual step functions via the --funcall option in the CLI.
Here is the full make.el which I find quite readable:
(require 'package)
(defconst make-packages
'(transient dash yaml-mode s))
(defun make-init ()
(add-to-list 'package-archives '("melpa" . "https://melpa.org/packages/") t)
(package-initialize))
(defun make-install-packages ()
(make-init)
(package-refresh-contents)
(package-install 'package-lint)
(dolist (pkg make-packages)
(package-install pkg)))
(defun make-ert ()
(make-init)
(load-file "/kubel/kubel.el")
(load-file "/kubel/test/kubel-test.el")
(ert-run-tests-batch-and-exit))
(defun make-compile ()
(make-init)
(setq byte-compile-error-on-warn nil)
(batch-byte-compile))
(defun make-lint ()
(make-init)
(require 'package-lint)
(setq package-lint-batch-fail-on-warnings t)
(package-lint-batch-and-exit))
(provide 'make)
;;; make.el ends here
And from there I can call my step functions directly:
emacs -Q --batch -l make.el --funcall make-install-packages
emacs -Q --batch -l make.el --funcall make-ert
emacs -Q --batch -l make.el --funcall make-compile kubel.el
emacs -Q --batch -l make.el --funcall make-lint kubel.el
I placed my make.el file in the .circleci folder to keep things organized:
version: 2.1
jobs:
build:
docker:
- image: silex/emacs:27.1
working_directory: /kubel
steps:
- run: apt update && apt install -y git ssh make
- checkout
- run:
name: Install packages
command: |
emacs -Q --batch -l .circleci/make.el --funcall make-install-packages
- run:
name: ERT tests
command: |
emacs -Q --batch -l .circleci/make.el --funcall make-ert
- run:
name: Compile
command: |
emacs -Q --batch -l .circleci/make.el --funcall make-compile kubel.el
- run:
name: Lint
command: |
emacs -Q --batch -l .circleci/make.el --funcall make-lint kubel.el
This will nicely run all our steps in order on Emacs 27.1 on every commit.
Yes!
Running tests and linter and whatnot is very nice, but I think there's an even bigger benefit we can reap here from spinning up a fully isolated Emacs instance. We can answer questions which are often harder to investigate locally:
- how can we make sure we truly only depend on the packages we say we depend on?
- how can we make sure our package actually works on the all the Emacs versions we say we support?
We already answer the first question thanks to the dockerized Emacs and the controlled external package installation. For the second question, we can parallelize the build steps to run on multiple Emacs versions all at once:
version: 2.1
steps: &steps
working_directory: /kubel
steps:
- run: apt update && apt install -y git ssh make
- checkout
- run:
name: Install packages
command: |
emacs -Q --batch -l .circleci/make.el --funcall make-install-packages
- run:
name: ERT tests
command: |
emacs -Q --batch -l .circleci/make.el --funcall make-ert
- run:
name: Compile
command: |
emacs -Q --batch -l .circleci/make.el --funcall make-compile kubel.el
- run:
name: Lint
command: |
emacs -Q --batch -l .circleci/make.el --funcall make-lint kubel.el
jobs:
emacs-27:
docker:
- image: silex/emacs:27.1
<<: *steps
emacs-26:
docker:
- image: silex/emacs:26.3
<<: *steps
emacs-25:
docker:
- image: silex/emacs:25.3
<<: *steps
workflows:
version: 2
build:
jobs:
- emacs-25
- emacs-26
- emacs-27
Why not use docker and a trendy CI/CD system to run integration tasks for extensions of a Babylonian software?
I encourage you to checkout: