Interprocess Communication (IPC) In Emacs for Typing Sounds

• mpv installed
  
  • mpv will essentially act as a daemon that we can write text to and return control quickly to emacs
    
• *nix operating system
  
  • (this might work on windows, untested; windows works a bit differently with pipes)
    
• An audio file
  
• (optional) emacs selectric-mode
  

Emacs is mainly single threaded¹. So even though emacs has (play-sound), this will block, locking up emacs for the duration of the sound.

Much of parallelism in emacs boils down to forking a process and sending commands to it.

That's what selectric-mode does. It calls

(start-process "*Messages*" nil "aplay" sound-file-name)

And hey, it works. Play a sound a file from a different process and the user is back in control in emacs.

But forking processes is expensive. It costs both time, and memory. What's more–we're just playing the same sound. Do we really need to start a process again and again?

My computer couldn't handle selectric very well when I first used it years ago. I felt this when I was using a selectric-mode on a pentium core processor….It's runs okay now, but my computer still gets very hot. No one has worked on the performance of it and the author has not responded to issues.

Looking up playing audio online with emacs shows people mostly using aplay or play however.

Couldn't we just decouple the caller (emacs) from the receiver (thing that plays audio), and keep the receiver alive?

Before diving into the daemon approach, I wanted to try threads.

So instead of (start-process ...) on the main thread, we can instead move that off to another thread. This frees up the command loop² from starting the process.

(use-package selectric-mode
  :config
  (defvar jwow/typewriter-enabled nil)
  (defun jwow/threaded-sound () (make-thread #'selectric-move))
  (defun jwow/toggle-typewriter ()
    (interactive)
    (if (setq jwow/typewriter-enabled (not jwow/typewriter-enabled))
        (add-hook 'post-command-hook #'jwow/threaded-sound)
      (remove-hook 'post-command-hook #'jwow/threaded-sound)))

  ;; Now bind the key to a keymap, since I use xfk I bound it like so
  ;; (define-key xah-fly-leader-key-map  (kbd "; b") #'jwow/toggle-typewriter)
)

M-x jwow/toggle-typewriter to toggle.

This solves the blocking issue. I was able to type continuously, holding down a key for 30 seconds without lag.

But the CPU was still high. See Performance comparison.

That's because starting a process each time was expensive. What would help was a long running process with the audio already loaded in memory. Something we could control, like mpv.

The idea is taken from wasamasa's chip8.el: use mpv as a daemon and feed it messages from emacs. Emacs regains control and does other logic while the audio plays.

I first heard about this approach from his emacsconf talk. I tried the code, but it doesn't work anymore because mpv dropped the --input-file command option.

Eventually I stumbled onto other flags, like --input-ipc-server. And the mpv package, empv.el, takes a similar approach in creating the socket now. In my case it's a unix socket file.

All that's left is for emacs to send messages through the socket. Of course, emacs has a native API for that: Network Processes.

Once we have this process that acts as a wrapper for the socket, we simply talk to the process with (process-send-string process-reference string). Would it work?

For this section, I just looked at the graph from gnome system monitor. I also didn't type for the same duration in each of the three approaches as there was enough data.

Without any sound

Using plain selectric mode:

Using threads:

Using mpv:

Although the screenshots are from different days, and therefore slightly different state on my computer, these results tend to hold regardless. I was happy enough with my approach, but I won't pretend it's perfect.

• In 2024-12-24, I got excited by another general solution. Join me there.
  
• mpv has many options and you can drive it completely by ipc (I think!). Try grabbing data from the mpv process too!
  
• empv.el drives mpv using ipc as well
  
• If the process you want to communicate with registers itself with D-Bus, you can try emacs's D-Bus API. To debug and find message signatures, I've had good luck with the D-Spy tool. Things are a bit cryptic since many programs have lacking documentation on how they are controlled via dbus.
  
• In the example here, I only changed the execution flow when when a user was typing. I didn't change how the move sound worked in the typewriter. You can actually try other commands to drive mpv remotely here: https://github.com/mpv-player/mpv/blob/master/DOCS/man/input.rst
  
  • It'd be interesting to try loadfile <url> [<flags> [<index> [<options>]]] to load other files into mpv and cycle through them. Would it be performant?
    
• I tried out midio which may do what you're looking for. From my understanding, you send midi commands to fluidsynth, which is kind of similar to what we're doing here.
  

Copy and paste into a scratch buffer and M-x eval-buffer!

;; I'm using selectric mode, but you can try out the code without it. 
;; Just make sure you have an audio file you can use
;; And call (process-send-string jwow/socket-process "seek -10; cycle pause\n")
(use-package selectric-mode)
(selectric-mode 1)
(setq jwow/mpv-audio-file (let ((selectric-audio-file
                                 (expand-file-name
                                  (format "%s/../selectric-move.wav"
                                          (find-library-name "selectric-mode")))))
                            (message selectric-audio-file)))
(setq jwow/unix-socket-file (expand-file-name "~/ipc-socket"))

;; This is the process
(defvar mpv-process)
(defun jwow/start-mpv-with-socket (socket-file audio-file)
  (setq mpv-process
        ;; Note that if you have a config, that might have different settings than how the process is started The
        ;; options are more verbose here because I have defaults that I'm overriding
        (start-process "mpv" "*mpv*" "mpv"
                       "--no-video" "--loop-playlist=no" "--pause"
                       ;; Otherwise the *mpv* buffer gets really full 
                       "--no-terminal"
                       "--keep-open" (format "--input-ipc-server=%s" socket-file)
                       audio-file)))

(jwow/start-mpv-with-socket jwow/unix-socket-file jwow/mpv-audio-file)
;; This must be run after the mpv process has started
;; This is because we connect to that process
(setq jwow/socket-process 
      (make-network-process
       :name "socket-client"
       :buffer "*socket-output*"
       ;; Ensure that we are talking through a unix socket using 'local
       :family 'local
       ;; Usually this is a protocol name but 
       :service jwow/unix-socket-file))

(defun selectric-type-sound ()
  ;; The \n is important, otherwise mpv will not recognize the command. Commands apparently must be terminated with a new line.
  (process-send-string jwow/socket-process "seek -10; cycle pause\n"))