A Ruby REPL workflow with Neovim and neoterm

Clojure’s REPL workflow has spoiled me, so I’ve arranged a crude approximation of it for Ruby using Neovim, the neoterm plugin and a tweak to pry.

These things make it possible to very quickly evaluate Ruby code in a console from inside a Neovim session, which is a good thing because

  • it makes it easy to set up state in a REPL
  • while collecting data or building complicated queries you can keep all the steps safe in a buffer, then go back and re-run them without having to sit at the console tapping ‘up’ like a madman
  • everything happens in a Neovim buffer so you can easily search, copy and paste. You can also write anything to a file, including the entire REPL session

Introducing neoterm
Neoterm is a small but complete Neovim plugin. It looks after Neovim’s terminal buffers for you, and provides commands for sending snippets of text from an ordinary buffer to a running terminal.

It installs like any other Neovim plugin; instructions are on the neoterm GitHub repo.

Configuring neoterm
Neoterm needs some configuration to be useful. Here are the settings I arrived at.

let g:neoterm_autoscroll = '1'

^ This means the terminal will always scroll down to reveal the latest output. This isn’t necessary when you’re interacting with the terminal like an ordinary shell, but if you’re sending bits of code to it from another buffer it’s a must-have.

let g:neoterm_size = 16

^ By default neoterm’s buffer occupies 50% of the Neovim window, which is a bit much for me.

command! -nargs=+ TT Topen | T

^ This :TT command was kindly provided to me by kassio, the author of neoterm. It has the effect of opening a new terminal if none is already running, or reusing an existing terminal. Either way, the terminal it summons runs whatever shell command you typed in after :TT — a bit like :!, but smarter.

nnoremap § :TT

^ This is a shortcut. § is the key to the left of 1 on my keyboard. Pressing it in normal mode opens up a :TT on the Neovim command line ready for a shell command.

vnoremap <Leader>2 :TREPLSendSelection
nnoremap <Leader>2 :TREPLSendLine

^ These mappings are for interacting with the REPL. 2 will behave selection- or line-wise in visual or normal mode respectively. The effect is to send all the chosen code to the terminal and hit return at the end of each line.

To be useful, the Neovim terminal should be running a Ruby console when you send it Ruby code. If there’s no pre-existing terminal, one will be started for you and a console booted. Neoterm is smart enough to start an appropriate language REPL for the file you’re currently editing. If you’re in a Rails project, it will start a Rails console for you.

nnoremap <Leader>3 :Tmap
nnoremap <Leader>0 :Ttoggle
let g:neoterm_automap_keys = '<F5>'

^ The neoterm docs are a little bit unclear on what :Tmap does. It lets you define a command which will be repeated when you hit the neoterm_automap_keys key, in my case F5. I’ve found this useful in the shell, where I use it to re-run my script, rake task or whatever over and over again.

Finally, :Ttoggle will hide the current terminal. It can be brought back by sending something to the REPL, or starting afresh with a :TT

With a little practice you will be sending code to the REPL with a flick of 2 and then folding away your console with 0 when you’re done. Lost something in the output? Switch to the terminal buffer and use ordinary Vim movements to swoop up and down through history.

A note about pry
The way the pry console paginates long output disrupts the way neoterm interacts with the REPL; I think Pry expects a q to break out of the pagination and neoterm does not know to send one. I added a line to my ~/.pryrc file to disable the pager when pry was running inside Vim.

if ENV['VIM']
  Pry.config.pager = false

My Neovim configuration is on Github with all my other dotfiles.

A Rubik’s cube in Clojure: querying the cube

I’m a Ruby programmer trying to learn Clojure and functional programming by making a Rubik’s cube in code, making mistakes as I go. In the last post I made a data structure, which is a 26-element vector of maps representing the little cubes that comprise the Rubik’s cube. Each one contains information about its position and the colours of its faces. Read more about the data structure here.

I need a way to manipulate the cube. And a way to look at it too, because if I can’t see what’s on each side, there’s no way I’m going to know if the various transformations we do to it are working. I decided to name the sides of my cube North, South, East and West, and the top and bottom faces Up and Down. I deliberately didn’t follow the convention of Left, Right, Front and Back because I didn’t want to have to deal with relative positions when I was transforming the cube. This turned out to be a slightly overcomplicated way of doing things, because I spent quite a lot of effort translating NSEWUD into x, y and z.

Anyway, I ended up with the following functions. You can find their implementations on GitHub. I didn’t consciously design anything up front. My process was more like

  • take stab at implementing the whole operation (reading the faces out of a side)
  • refactor that function into smaller functions that are easier to test
  • repeat

For example, the first function I wrote was called ‘get-face’. During this process it delegated a few things down into other functions, and it also lost some responsibilities, which I pushed upwards into functions which called it.  Having imagined it would be my public interface, in fact I ended up with it somewhere in the middle of the chain of functions required to get the colours of a face.

;; take a cube and a face and return an array of colours in a predictable order.
(get-face-colours (get-sorted-face face cube)) 

;; infer which aspect (x, y, or z) a set of pieces share. 
;; For this function to return x, for example, all
;; the pieces must have colours in the x axis, though 
;; some of them might also have colours in the y or z axis.
(get-aspect pieces) 

;; sort a collection of pieces into a consistent order, 
;; taking into account which plane it's in (x, y or z). 
;; This function is 17 lines long (!) and consists mostly 
;; of a huge (case) statement. 
(get-sorted-face face cube) 

;; this function exists only to execute the various permutations
;; of operators and planes from get-sorted-face using sort-by
(sort-pieces-by-axis plane operator pieces)

As you can see, concerns fell out while I was writing this and they became other functions. This process is not very different from writing a program in Ruby, though obviously without a class to wrap these things up there is no state to lean on so there are a few more params flying around. In that respect I feel like I am programming in a more functional style.

Am I happy with these functions? Sort of.

BIG TICK: They are all pure functions
SMALLER, MORE AMBIVALENT TICK: It’s possible to test each one of them. However, because they pass my cube and piece objects between themselves, their tests all require fully-fledged data structures. This makes them feel not-very-reusable. They all need to know about a map I defined elsewhere called face-mapping, which translates :N, :S, :E, :W etc into an internal map the contains some information about that face, like which axis it’s in and how to filter out its pieces from the cube.
NO TICK: Most of them are 2 or 3 lines long, except get-sorted-face which is a 17-line monster. This is probably because get-sorted-face knows a lot about how I want my pieces sorted when I ask for them in a face. It feels like it would be nice to substitute this with something a bit more generalised, and then my code would not have to worry about my data structure.

This is something that I haven’t experienced before. With object-oriented programming, there is rarely the opportunity to decouple the implementation of the program from the data structure, because the bit of the program that’s concerned with the data lives with that data. Although my functions live on the wide open prairies of a Clojure namespace, they won’t be much use to anyone who meets them out there.

I revisited the last 10 minutes of ‘Simple Made Easy’, where I heard this:

[Objects] were never meant to be applied to information… It’s wrong. But I can now say it’s wrong for a reason. It’s wrong because it’s complex. In particular, it ruins your ability to build generic data manipulation things. If you leave data alone, you can build things once that manipulate data, and you can reuse them all over the place, and you know they’re right once and you’re done

Between us, I’m not sure whether striving for generic functions might actually be better than just chiselling out functions that accommodate the data a la 99 bottles, but I’m here to learn so I’m going to try it. So: two concrete lessons I’ll take from this:

  1. Instead of enabling my functions to work on complex data, I should try to make my data simpler so my functions can be too.
  2. I could probably find a library to provide those basic functions for me. So the only thing I would have to do would be get my data into shape.

Next post: graphics!



Representing a Rubik’s cube in Clojure

I’ve just finished a working draft of my first Clojure program, which is a representation of a Rubik’s cube in code. The cube can be transformed using the usual Rubik’s cube moves of rotating a face through the x, y or z axes, and each side of a cube can be ‘rendered’ as text.

;; (get-face :N cube)
((:blue :blue :blue)
 (:blue :blue :blue)
 (:red :red :red))

This exercise touched several programming concepts I’d never come into contact with before, in particular 3D transformations. It was also my first Clojure project, too, so I had some adventures in that direction.

In this and following posts I’ll write about some things I learned about Clojure and about cubes while I did this.

Starting out

Before doing any manipulating or rendering it was necessary to model the cube in some kind of data structure. I dug up some suggestions from the internet and evaluated them:

An array of arrays

The idea is to have six 3×3 arrays of colours, and this looks simple but feels like it gets ugly quite quickly. What happens, for example, when you rotate a layer? The colours on that layer do not change, but the adjacent colours all do. These colours need to be accessed by numerical array indices and them rotated in the correct direction. This felt like it was going to be fiddly, especially when it was necessary to rotate around three different axes, and I was likely to make mistakes. Also, the implementation would have involved a lot of swapping things around in arrays, which doesn’t make for very readable code.

Interlocking linked lists

Another suggestion from Stackoverflow was to use interlocking linked lists. At first I thought this would be ideal: Lisp is linked lists all the way down, right? This approach defines each cube piece as a struct containing pointers to its vertical and horizontal neighbours, plus its colour. The idea of everything being defined relative to everything else was rather exciting, but it made me uneasy about rendering the state of the cube: I imagined a situation where I got lost in the middle of these linked lists without knowing where faces began and ended. I suppose I’d maintain a pointer to a single piece in a known position and use that as the starting point to figure out where everything else is. Anyway, this felt like biting off a bit more than I could chew; perhaps I’ll try it in future.

An array of objects

In the end I went for the most verbose data structure available, which is the one described in this Stackoverflow answer. The author uses objects to represent the bits of the cube, noting that this verbose approach is not terribly efficient. To me it it is the most readable and that’s what I care about.

Of course, this is Clojure and there are no objects, so I chose to use maps. In this structure, each piece is defined by its co-ordinates in space and a map of planes to colours which tells us what the orientation of the piece is. Here, for example, is the definition of a piece:

{:position {:x 1 :y -1 :z 1}
:colors {
:y :green
: x :red
:z :white

To put that in context, here is an artist’s impression of the cube with some axes attached. The piece described is the one at the near corner of the white top of the cube:

Screen Shot 2017-05-07 at 21.27.52

The :colors key maps colours to axes. If we’re looking at a piece from the top or the bottom of the cube we care about its z-axis; from the East, the x-axis; from the South, the y-axis. It’s an amusing quirk, and possibly a technical flaw, of this implementation that we do not need to care about which direction we’re looking at a piece from as long as we’re looking at it in the right axis.

At the heart of the program is a (def default-cube) which defines a 26-element vector of maps like the above, which collectively describe an entire cube in the completed state. The rest of the program a) chooses and b) tweaks elements of this vector to return a whole new 26-element vector representing the new state of the cube, which it can then render.

In the next post I’ll write about how that happens.

Clojure: a beginner’s notes

I started learning Clojure earlier this year because I had heard that it was simple and beautiful and that learning Lisp was a mind-expanding thing to do. In no particular order, here are some of my observations:

For context, my day job is as a Rails developer.

You can’t just muddle through

When learning other languages I’ve found that learning the syntax for methods, iteration etc is usually enough to be dangerous. Not so in Clojure, where you have to learn the concepts. Some things are extremely awkward if not impossible if you don’t have a clue about partial, zipmap, interleave, drop , union and other things which you might have merrily ignored in eg Ruby.

I am half-expecting enlightenment

Every time I sit down to write some Clojure I hope in a small way that I am about to have some sort of transcendental experience brought on by the elegance of it all. This hasn’t happened yet, but I have thought that things were quite cool.

I do not know if the things I thought were cool are the things that others consider to be elegant about Clojure. I wonder what those things are.

4Clojure is great

But even some of the problems marked Easy are hard. The solutions are often more or less mind-blowing. I am glad I ran through as many of these as I did before I tried to implement anything. I recommend following dzholev.

Please don’t make me use emacs

I’m a vim user. I installed vim-fireplace and vim-sexp, and things seem kind of ok. I can’t help but wonder what I’m missing out on from emacs. I am good at vim, but I find it fairly horrible to edit s-expressions in it, even with vim-surround.

The REPL is convenient

For example:

  1. Write function and use vim-fireplace to evaluate it in the REPL.
  2. Invoke function from REPL, revealing unexpected behaviour
  3. Change function and re-evaluate it.
  4. Press up in the REPL and run it again, yielding different results.

This is great! However,

The REPL is difficult and/or confusing

Your code might not have state, but the REPL does, and I find it difficult to manage. I’d run a test from Vim and I couldn’t be sure if the result was a pass or false positive, because I couldn’t be sure which version of the function I was testing I’d last evaluated. Super confusing!

You build programs bit by bit

My first Clojure program is a data model for a Rubik’s cube. My approach has been to roughly chop up the processing into functions, try to write them and unit test them, find they do too much (or too little), chop up again and repeat. I’ve ended up with a handful of well-tested low-level functions that operate on little parts of the cube (ie the mini-cubes) and a layer of hardly-tested higher-level functions that operate on whole sections (rows, columns etc) of the cube.

I haven’t tested those much because tests for big data structures are hard to fit on one page and in my head. Also, I have a reasonable amount of confidence that if I map over a function I have tested to death, I don’t need to worry about that working—as long as I got the syntax for that higher-level function right. I wonder if this is what functional programming feels like.

I like the feeling that if I can trust my little functions, the big functions will do the right thing.

The approach I’m taking after writing a bit of Clojure

If I have a choice between writing a function that operates on a set of things and a function that operates on an individual thing from that set, I’ll choose the latter because it is much easier to reuse.

Functions are either doers or gluers. Doers do things like filtering, transforming and updating. Gluers call ->mappartialapply etc to link doers together. Doers are words, gluers are sentences.

Functions that take params plus a data structure to work on like to take the data structure as their last argument, so they can be partially applied.

Running tests with lein test is not an option

lein is amazingly slow! On my laptop it takes over 30 seconds to load and about 10 ms to run the tests. The solution seems to be to run the tests from your editor (but then you contend with state in the REPL).

You will be writing SQL by hand

Coming from Rails, I was amazed to discover that Luminus, which is as far as I can tell the pre-eminent web framework for Clojure, expects you to write your SQL queries by hand. By hand!

Composition over… everything else

Luminus is not so much a framework as a collection of libraries that provide the bones of an application, and once it’s installed you’re on your own with regard to how you organise your code. It’s closer to express.js than any of the “proper” MVC frameworks. There is nothing wrong with this but it’s a bit daunting for a beginner, especially one who is a bit worried about code organisation and namespaces.


Coming from Ruby, I’m almost inclined to treat namespaces a like classes and group functions that operate on the same data structure in them. But I suspect I’m just being set in my object-orientated ways. Another idea is to group functions that do the same sort of thing (eg printing functions, transforming functions, filtering functions, etc), but I wonder if that will end up with large namespaces full of functions, which I’d then split up into groups, and end up with an explosion of subdirectories and small files. Maybe that isn’t so bad .

I haven’t found that requiring namespace A inside namespace B and then requiring namespace B in namespace C means namespace A is available in namespace C. I am probably missing something here.

Must you already appreciate mathematics to appreciate Clojure?

I am bit worried the famous simplicity of the language is not the kind of simplicity I consider to be simple.

I suppose I’ll find out.

Single Table Inheritance vs Polymorphic Associations in Rails

These two things are sometimes confused because they both involve columns with the word type in. In fact they serve very different purposes.

Single Table Inheritance is when models share a database table but have different behaviour. For example, in your beekeeping newsletter application you might have a Subscription model which comes in two flavours: PaidBeeSubscription and FreeBeeSubscription. These are identical in all respects except the way they react to the method #price. There is a field on the Subscription class called type, which tells Rails which class to give you when you load that model.

A polymorphic association is when you have different models that need to have_many of the same thing. For example, you might have a Donkey class and a Tortoise class in your petting zoo application, and both of these things can have Pictures associated with them. Follow the Rails Guides to learn how it works, but the effect is that a Picture carries a relation to an imageable model which could be a Donkey or a Tortoise — Picture doesn’t care which. To help Picture know which model to give you when you request its imageable, it has a column called imageable_type, which records whether the associated model is a Donkey or a Tortoise or a Porpoise or whatever.

As you can see, these things are more or less totally unrelated. But people get them confused because they both have columns with the word type in, and those type columns both contain the names of Rails models.