The first iteration of the Emacs User Survey is currently on track to open up Monday October 19th 2020.

A huge thanks to everyone who gave feedback and suggestions, and particularly u/yyoncho who had put together a great list of questions on r/emacs.

1Why

For the community to understand itself better.

2Where

https://emacssurvey.org

3When

The survey will be open from Monday October 19th to Monday November 30th.

4How

Two ways to submit the survey:

• online form
• filling out a template and emailing it back

5Who

Questions or comments can go to contact@emacssurvey.org

6What you can do to help

• fill it out!
• most importantly, share it with your Emacs friends!
  • blog about it
  • email your colleagues who also use Emacs
  • etc

I was very inspired by Damien Cassou's great presentation during EmacsConf 2019 to write this post and I encourage you to check it out if you haven't already. In short, when writing packages for Emacs, it is best practice to run several quality tools on them, like syntax and documentation checkers, or even ERT Tests. But once these packages are public and pull requests start coming in, it is a huge time saver to have these same tools ran automatically and provide feedback to contributors. That's right, we're talking about Continuous Integration for Emacs packages.

1Why CircleCI

There are many CI/CD services out there that can help us out. Damien Cassou shows examples with Travis, Gitlab and Drone, but I wanted to focus on CircleCI. It's far from perfect but their free tier is decent, their markdown badge is pretty, and I wrote a CircleCI Magit extension to show the build status in Emacs (ie. an extension for an Emacs extension).

2Run Steps

We want a few things to happen automatically:

• install package dependencies from Elpa/Melpa/Org
• run ERT tests
• byte compile
• lint package

To keep things simple, I will use my package kubel as the guinea pig here.

3CircleCI Config Skeleton

Leaving out the actual commands, we can start with a simple draft of the .circleci/config.yml that uses the docker image for Emacs 27.1 as our base:

  version: 2.1

  jobs:
    build:
      docker:
        - image: silex/emacs:27.1
      working_directory: /kubel
      steps:
        - run: apt update && apt install -y git ssh
        - checkout
        - run:
            name: Install packages
            command: TBD

        - run:
            name: ERT tests
            command: TBD

        - run:
            name: Compile
            command: TBD

        - run:
            name: Lint
            command: TBD

Note that the silex/emacs image doesn't come with git and ssh installed, and that we need to apt install them first thing as they are required to run the code checkout step.

4Emacs Lisp Code

Let's translate the desired run steps into actual Emacs Lisp Code for Kubel:

• my package dependencies are transient, dash, yaml-mode and s

      (let ((my-packages '(transient dash yaml-mode s)))
        (add-to-list 'package-archives '("melpa" . "https://melpa.org/packages/") t)
        (package-initialize)
        (package-refresh-contents)
        (package-install 'package-lint) ;; we'll need it later
        (dolist (pkg my-packages)
          (package-install pkg)))
  

• run ERT tests in test/kubel-test.el

      (load-file "kubel.el") ;; load code first
      (load-file "test/kubel-test.el")
      (ert-run-tests-batch-and-exit)
  

• simple byte compile with warnings not counting as errors

      (setq byte-compile-error-on-warn nil)
      (batch-byte-compile)
  

• run package-lint, this time with warnings counting as errors

      (require 'package-lint)
      (setq package-lint-batch-fail-on-warnings t)
      (package-lint-batch-and-exit)
  

5How to trigger from CLI

This is probably the biggest hurdle: how to run Emacs Lisp Code from the command line in an elegant manner? There are quite a few solutions out there to help with this: Damien Cassou's fancy Makefile, collections of bash script, and plenty of others. However I am not completely convinced by any of them as I find the syntax always a bit clunky and hard to read. I am currently settled with putting all the Emacs Lisp code into a single make.el file, and call the individual step functions via the --funcall option in the CLI.

Here is the full make.el which I find quite readable:

  (require 'package)

  (defconst make-packages
    '(transient dash yaml-mode s))

  (defun make-init ()
    (add-to-list 'package-archives '("melpa" . "https://melpa.org/packages/") t)
    (package-initialize))

  (defun make-install-packages ()
    (make-init)
    (package-refresh-contents)
    (package-install 'package-lint)
    (dolist (pkg make-packages)
      (package-install pkg)))

  (defun make-ert ()
    (make-init)
    (load-file "/kubel/kubel.el")
    (load-file "/kubel/test/kubel-test.el")
    (ert-run-tests-batch-and-exit))

  (defun make-compile ()
    (make-init)
    (setq byte-compile-error-on-warn nil)
    (batch-byte-compile))

  (defun make-lint ()
    (make-init)
    (require 'package-lint)
    (setq package-lint-batch-fail-on-warnings t)
    (package-lint-batch-and-exit))

  (provide 'make)
  ;;; make.el ends here

And from there I can call my step functions directly:

  emacs -Q --batch -l make.el --funcall make-install-packages
  emacs -Q --batch -l make.el --funcall make-ert
  emacs -Q --batch -l make.el --funcall make-compile kubel.el
  emacs -Q --batch -l make.el --funcall make-lint kubel.el

6Putting It All Together

I placed my make.el file in the .circleci folder to keep things organized:

  version: 2.1

  jobs:
    build:
      docker:
        - image: silex/emacs:27.1
      working_directory: /kubel
      steps:
        - run: apt update && apt install -y git ssh make
        - checkout
        - run:
            name: Install packages
            command: |
              emacs -Q --batch -l .circleci/make.el --funcall make-install-packages

        - run:
            name: ERT tests
            command: |
              emacs -Q --batch -l .circleci/make.el --funcall make-ert

        - run:
            name: Compile
            command: |
              emacs -Q --batch -l .circleci/make.el --funcall make-compile kubel.el

        - run:
            name: Lint
            command: |
              emacs -Q --batch -l .circleci/make.el --funcall make-lint kubel.el

This will nicely run all our steps in order on Emacs 27.1 on every commit.

7Can We Do Better?

Yes!

Running tests and linter and whatnot is very nice, but I think there's an even bigger benefit we can reap here from spinning up a fully isolated Emacs instance. We can answer questions which are often harder to investigate locally:

• how can we make sure we truly only depend on the packages we say we depend on?
• how can we make sure our package actually works on the all the Emacs versions we say we support?

We already answer the first question thanks to the dockerized Emacs and the controlled external package installation. For the second question, we can parallelize the build steps to run on multiple Emacs versions all at once:

  version: 2.1

  steps: &steps
    working_directory: /kubel
    steps:
      - run: apt update && apt install -y git ssh make
      - checkout
      - run:
          name: Install packages
          command: |
            emacs -Q --batch -l .circleci/make.el --funcall make-install-packages

      - run:
          name: ERT tests
          command: |
            emacs -Q --batch -l .circleci/make.el --funcall make-ert

      - run:
          name: Compile
          command: |
            emacs -Q --batch -l .circleci/make.el --funcall make-compile kubel.el

      - run:
          name: Lint
          command: |
            emacs -Q --batch -l .circleci/make.el --funcall make-lint kubel.el

  jobs:
    emacs-27:
      docker:
        - image: silex/emacs:27.1
      <<: *steps
    emacs-26:
      docker:
        - image: silex/emacs:26.3
      <<: *steps
    emacs-25:
      docker:
        - image: silex/emacs:25.3
      <<: *steps

  workflows:
    version: 2
    build:
      jobs:
        - emacs-25
        - emacs-26
        - emacs-27

8Final Words & Further Reading

Why not use docker and a trendy CI/CD system to run integration tasks for extensions of a Babylonian software?

I encourage you to checkout:

• Damien Cassou's full presentation
• Silex' Dockerized Emacs
• CircleCI Doc

This is the written version of a lightning talk I recently gave at the NYC Emacs Meetup, which is a great meetup that I cannot recommend enough.

1Why

Emacs lisp in particular is a good candidate for automated testing because it is ancient and quirky, provides no static types, and is often the result of many individual contributors. And to make things super easy, Emacs is shipped with ert, the Emacs Lisp Regression Testing library to write and run tests.

2Simple example

    (defun silly-division (a b)
      "Silly divide A by B."
      (if (equal b 1)
          a
        (/ a b)))

1. divide a by b
2. if b is 1, spare ourselves the computation and return a

2.1Simple test

     (require 'ert)

     (ert-deftest silly-test-division ()
       (should (equal 4 (silly-division 8 2))))

1. import the ert library
2. create a test named silly-test-division
3. make sure that in our world 8/2 = 4

For practical reasons, I would write my tests in a file named silly-test.el next to silly.el.

2.2How to run

     (ert 'silly-test-division)

Once ran, you will be in the debugging editor where you can:

• "TAB" to move around
• "." to jump to code
• "b" for backtrace
• "l" to show all should statements

2.3Testing multiple cases

     (ert-deftest silly-test-division ()
       (should (equal 4 (silly-division 8 2)))
       (should (equal 2 (silly-division 10 5)))
       (should (equal 10 (silly-division 10 1)))
       (should (equal 2 (silly-division 8 4))))

     (ert 'silly-test-division)

Remember that the "l" key in the tests results buffer will show all the should statements individually:

2.4Testing for error

     (ert-deftest silly-test-division-by-zero ()
       (should-error (silly-division 8 0)
                     :type 'arith-error))

     (ert 'silly-test-division-by-zero)

2.5Testing for failure

Rather than fix our function to perform floating point division, let's write a test for this bug:

     (ert-deftest silly-test-float-division-bug ()
       :expected-result :failed
       (should (equal .5 (silly-division 1 2))))

     (ert 'silly-test-float-division-bug)

3Trickier function

    (defun silly-temp-writer (str)
      (with-temp-buffer
        (insert (format "L %s L" str))
        (write-file "/tmp/silly-write.txt")))

1. take a string
2. format it "L %s L"
3. write that string to "/tmp/silly-write.txt"
4. don't return anything

How can we reliably test this? And what are we actually trying to test? I would argue that we want to make sure that what ends up being written to the file what we expect.

3.1Naive method

     (ert-deftest silly-test-temp-writer ()
       (silly-temp-writer "my-string")
       (should (equal "L my-string L"
                      (with-temp-buffer
                        (insert-file-contents "/tmp/silly-write.txt")
                        (string-trim (buffer-string))))))

     (ert 'silly-test-temp-writer)

1. call silly-temp-write-function with string "my-string"
2. read the content of "/tmp/silly-write.txt"
3. remove the new line
4. compare it to expected result "Lmy-stringL"

However, we have a few issues here:

• side effects in the test, we are leaving a file on the machine after running the test
• no isolation, if another process deletes the file mid-test, we could have a false negative
• testing more than we should, we are not just testing our logic but also the write-file function
• test complexity, our test is convoluted and hard to read

3.2Better approach with mocking

     (require 'cl-macs)

     (ert-deftest silly-test-temp-writer ()
       (cl-letf (((symbol-function 'write-file)
                  (lambda (filename)
                    (should (equal "/tmp/silly-write.txt" filename))
                    (should (equal "L my-string L" (buffer-string))))))
         (silly-temp-writer "my-string")))

     (ert 'silly-test-temp-writer)

1. Define a mock write-file function
   • check that we write in the correct location
   • check that the content is formatted properly
2. Temporarily replace the real write-file with our mock
3. Call silly-temp-writer

We can observe that now most of our issues from the naive test are gone:

• Not actually writing to the system or leaving state
• Test more and closer to the intended behavior
• Not testing something we didn't intend to (ie. the write-file function)

NB: In the past I used to do this with the flet function but apparently it is obsolete since Emacs 24.3. As a replacement, I found that cl-left from the cl-macs library did the job pretty well.

4Running all tests at once

    (ert "silly-test-*")

And you can take it even further by running the tests from a bash script or a docker command, perfect for your CI pipeline:

     docker run -it --rm  -v $(pwd):/silly silex/emacs emacs -batch -l ert -l /silly/silly.el -l /silly/silly-test.el -f ert-run-tests-batch-and-exit

Which will output:

    Running 4 tests (2020-07-08 14:47:49+0000)
       passed  1/4  silly-test-division
       passed  2/4  silly-test-division-by-zero
       failed  3/4  silly-test-float-division-bug
       passed  4/4  silly-test-temp-writer

    Ran 4 tests, 4 results as expected (2020-07-08 14:47:49+0000)
    1 expected failures

5Visualizing coverage

1. M-x testcover-start
2. Select the silly.el file
3. Run tests

          (ert "silly-test-*")
   

4. M-x testcover-mark-all and select silly.el
5. See results:
   • red is not evaluated at all
   • brown is always evaluated with the same value

For example, in this case, we can see that I only have one test case for my silly-division with b equal to 1 and returning a directly.

6Best practices

• ask yourself what you want to test
• start by making tests fail, there's nothing better to insure you are taking the code path you think you are taking
• write clean test with no side effect, and if you must have side effect, run a cleanup function afterwards
• descriptive test names can really help figure out what is broken
• good tests means good debugging

7More Resources

• ERT Manual
• Emacs Manuel on Test Coverage

I work with websockets a lot and as painful as they can be sometimes, their versatility easily makes up for it. If you are not familiar, a websocket is basically a two way connection between a client and server. You would typically encounter one in a web chat applications, or any use case when you would want the server to send data to the browser without the browser requesting it first.

And of course there is an amazing Emacs extension for it on Elpa thanks to Andrew Hyatt. It's a bit lacking in explicit documentation, but the functional tests for it provide the main ideas on how to get started.

1Emacs as a websocket client

Probably the first use case: how to open a websocket connection from Emacs.

Here is a basic example making use of the websocket.org echo test which echoes back any string sent.

  (require 'websocket)

  (setq my-websocket
        (websocket-open "wss://echo.websocket.org"
                        :on-message (lambda (_websocket frame)
                                      (message "ws frame: %S" (websocket-frame-text frame)))
                        :on-close (lambda (_websocket) (message "websocket closed"))))

  (websocket-send-text my-websocket "hello from emacs")

  (websocket-close my-websocket)

1. we load the websocket extension
2. we open a new websocket and name it my-websocket
   1. we provide a function to call when we receive a message from the server: print it out
   2. we provide a function to call when the websocket is closed: message "websocket closed"
3. we send "hello from emacs" through the websocket
4. we close the websocket

Note that the on-message function is given two arguments:

1. the websocket object itself, that we are ignoring here
2. the frame data from which we can extract text with the websocket-frame-text function

In terms of output, we see

ws frame: "hello from emacs"
websocket closed

1. the echo server sending back our original message
2. the websocket being closed

2Emacs as a websocket server

Now onto the more exotic stuff: how to turn Emacs into a websocket server.

2.1Basic Setup

Let's start with a basic setup:

  (setq my-websocket-server
        (websocket-server
         3000
         :host 'local
         :on-message (lambda (_websocket frame)
                       (message "received message through websocket"))
         :on-open (lambda (_websocket)
                    (message "websocket opened"))
         :on-close (lambda (_websocket)
                     (message "websocket closed"))))

1. we start a websocket server and call it my-websocket-server
2. the server is running on localhost port 3000
3. when the server receives a message, we print "received message through websocket"
4. when a client connects to the server, we print "websocket opened"
5. when a client closes the websocket, we print "websocket closed"

Now to test this code, we could use the sample from the earlier section, but instead let's use some Javascript code that we will enhance later on. We can paste this in the browser console:

  const ws = new WebSocket("ws://localhost:3000");

  ws.onmessage = function(event) {
    console.log(event.data);
  }

  ws.send("hi");
  ws.close();

1. we establish a connection to localhost:3000
2. we register a function to log any frame data coming from the server
3. we send "hi" to the server and see "hi" echoed back in the console
4. we close the connection

From the Emacs perspective, we see

websocket opened
received message through websocket
websocket closed

And just for the sake of completeness, we should close our server:

  (websocket-server-close my-websocket-server)

2.2Automatic Refresh on Save

Let's do a real use case: trigger the browser to refresh a page when we save some edits. This is a good example because it is the classic case of having a server (Emacs) needing to send a message to the client (the browser) without having client requesting it first. In other words, we don't want the browser to poll Emacs every X seconds asking if it should refresh, we want Emacs to tell the browser to refresh.

We start with a very simple HTML document that we name "test.html".

  <html>
    <body>
      <h1>Hello world</h1>
    </body>
    <script>
     const ws = new WebSocket("ws://localhost:3000");
     ws.onmessage = function(frame) {
       location.reload();
     }
    </script>
  </html>

All it shows is "Hello world" in big font but actually:

1. we open a websocket to localhost:3000
2. on every message coming from that websocket, we trigger a page reload

Now on the Emacs side, we need to do define the function that we want to call when "test.html" is saved

  (defvar opened-websocket nil)

  (defun websocket-on-save ()
    (when (and opened-websocket
               (equal "test.html" (buffer-name (current-buffer))))
      (websocket-send-text opened-websocket "refresh")))

  (add-hook 'after-save-hook #'websocket-on-save)

1. Define a global object for our websocket and initialize it as nil
2. Define the websocket-on-save function which
   1. if our websocket is not nil
   2. and we are currently editing "test.html"
   3. we send the string "refresh" through our websocket
3. Have websocket-on-save be called after every buffer save

Now let's start the server again (if you encounter an "Address already in use error" you might have forgotten to stop the server in the previous example)

  (setq my-websocket-server
        (websocket-server
         3000
         :host 'local
         :on-open (lambda (ws) (setq opened-websocket ws))
         :on-close (lambda (_websocket) (setq opened-websocket nil))))

1. When a connection is made, we assign it to our global websocket object
2. When a connection is closed, we reset our global to nil

With all that hooked up, we can make some changes to the "test.html" file and see them appear without refreshing!

And let's not forget to clean up by removing the hook and closing the server:

  (remove-hook 'after-save-hook #'websocket-on-save)
  (websocket-server-close my-websocket-server)

Full disclaimer first: I am not a statistician nor well versed in statistics. But I was recently very interested in knowing more about MELPA and how it was used. I did a bit of research, wrote a little data massaging script, compiled data here, and went on to gather a few basic statistics about MELPA.

1First glance

At the time of writing this and when the data massaging script was ran, MELPA had

• 4,548 packages
• 1811 package owners
• 180,821,044 total downloads

2Timeline

The first thing I wanted to know was how many packages were added by year. MELPA started in 2012 and since then its biggest year was 2013 with 755 new packages accepted into the repo. Since then, it has been on the decline.

3Sources

No surprises here but over 95% of packages are hosted on github.com.

4Downloads

The basic stats are:

• average of 39,908 downloads for a package
• but a median of 1,831 downloads
• with standard deviation of 148,129 downloads
• the package with the fewest downloads was just added and sits at 7 downloads
• the packages with the most downloads is at 2,693,349 downloads

This translates into a very long tail distribution with a huge quantity of packages with "few" downloads and outliers with a very large number of downloads.

Not very useful. There are definitely fancy ways of representing long tail data but it will have to be for another time.

More interesting to compare is downloads to months since published.

We can see that a lot of packages stay at a relatively low number of downloads over time meanwhile newer packages can reach a large number of downloads quickly.

Without thinking much, I compared the length of a package name to its download count.

It seems that packages with shorter names get more downloads. My two cents theory is that they are evocative enough or a library with catchy name (ie. dash). On the other hand, packages with very long names can be extensions of extensions like (company-XXX) with more of a niche aspect to begin with.

5Owners

Earlier I mentioned 1811 unique package owners. I was interested in seeing how many packages does one owner have. Obviously it averages to 2.45 packages per owner, but if I break down into buckets of 1, 2, 3, 4, and 5 and over packages I get this:

Which tells us that about 63% of owners published only 1 package to MELPA.

6Further steps

I had quite a bit of fun looking at these numbers. I could foresee going back to the data with different questions. It would also be neat to get information about the code repositories themselves and interact with the github API to get things like total contributors per packages and number of commits.

And who knows, maybe an interactive page to see charts in more details.

I have tried to include Hyperbole more and more into my workflow, and to match with the upcoming release of Hyperbole 7.0.8, I wanted to share a few tricks I like.

This is far from covering all of Hyperbole's capabilities, but its the easiest ones to start with.

The number one killer feature: "implicit buttons", or "do what I mean here".

By default, put your cursor on something and hit M-Return

1Jump to any file

2Run a key sequence

You can execute any key sequence from its text representation

3Jump to error

I use that one in my daily workflow. If I get a compiler error, I can directly jump to the line causing a problem.

4Action buttons

This is brand new, you can call any emacs lisp function or display a variable value with the <EXPRESSION> syntax.

5More cool buttons

• External processes

        "!/usr/local/bin/redis-cli"
  

• git, github, gitlab links

        gh#abrochard/emacs-todoist/2a63ce7
  

• social media

        twitter@abrochard
  

6How to install latest

I like to work off the git sources.

6.1Clone and make

  git clone https://git.savannah.gnu.org/git/hyperbole.git
  cd hyperbole
  make src

6.2Load

  (add-to-list 'load-path "PATH-TO-HYPERBOLE-FOLDER")
  (load "hyperbole-autoloads")
  (load "hyperbole")

7What to do next

Check out the demo with {C-h h d d}