Jake McCrary

Reading in 2014

At the beginning of last year I took some time and reviewed my 2013 reading using Clojure and Incanter to generate some stats. It was a useful exercise to reflect back on my reading and play around with Incanter again.

Over the last couple of weeks I’ve taken a similar look at my 2014 reading. The rest of this post highlights some of the top books from the previous year and then posts some numbers at the end.

I review every book I read using Goodreads. If you want to see more of what I’ve been reading you can find me here. I track and review every book I read and have found this practice to be extremely rewarding.

2014 Goals

I entered 2014 without a volume goal. Unlike 2013, I didn’t have a page or book count goal. I entered 2014 with the desire to reread two specific books and the nebulous goal of reading more non-fiction.

2014 Results

I ended up setting a new volume record. I read 69 books for a total of almost 23,000 pages. I also read every week of Day One, a weekly literary journal containing one short story and one poem from new authors. This doesn’t count towards my page or book count but is reading I enjoy. It exposes me to many different styles.

More than a third of my reading was non-fiction. I don’t have numbers for 2013 but that feels like an increase. I consider my goal of reading more non-fiction achieved.

I also reread the two books I had planned on rereading. I wanted to reread Infinite Jest and Hard-Boiled Wonderland and the End of the World and succeeded in rereading both of them.

Recommendations

I awarded seven books a five out of five star rating. I’ve listed them below in (in no particular order). Each book I’d recommend without hesitation. Instead of reworking or copying my previous reviews I’ve provided links to Goodreads. The titles link to Amazon.

I’m recommending a specific translation of Meditations. I attempted to read different one first and it was so painful to read I ended up giving up. The linked translation is modern and contains a useful forward giving you background information on the time.

I only read one series this year but it was a good one. The Magicians, by Lev Grossman, was recommended by a friend who described it as “Harry Potter but with characters battling depression.” I’m not sure that fully captures the feel of the series but it is a start. The series introduces you to a world like our own but with magic. You follow cynical, self-absorbed students as they attend school, graduate, and grow up living in both the magical and non-magical world. The first book in the series is the weakest so if you read that and find it enjoyable you should definitely pick up the next two books.

2015 Goals

2015 isn’t going to have an easily measured goal. I don’t feel the need to set number of books or pages goals any more. I’m hoping to increase the quality of my reading. This is a pretty unclear goal. To me this doesn’t mean increasing the average rating of books I read but instead I want to get more out of what I read. I want to think a bit deeper about the subjects I’m reading.

2014 Measurements

Below are some random measurements that are probably only interesting to me.

This year I recorded the format of the books I read. This was the year of the ebook; over 90% of the books I read were electronic. I’d guess that this is a higher percentage of ebooks than previous years. I wish I had recorded the formats read in previous years.

1
2
3
4
5
| Binding   | Number of books |
|-----------+-----------------|
| Hardcover |               1 |
| Paperback |               4 |
| Kindle    |              64 |

My average rating has been going down over the last four years.

1
2
3
4
5
6
| Year | Average Rating |
|------+----------------|
| 2011 | 3.84           |
| 2012 | 3.66           |
| 2013 | 3.67           |
| 2014 | 3.48           |

In 2014, three authors composed nearly 25% of my reading (by page count). The top six authors by page count are below.

1
2
3
4
5
6
7
8
| Author               | My Average Rating | Number of Books | Number of Pages | Percent of Total Page Count |
|----------------------+-------------------+-----------------+-----------------+-----------------------------|
| David Mitchell       |                 4 |               5 |            2334 |                      10.19% |
| David Foster Wallace |       4.333333333 |               3 |            1753 |                       7.65% |
| Lev Grossman         |       3.666666667 |               3 |            1244 |                       5.43% |
| Marisha Pessl        |               3.5 |               2 |            1153 |                       5.03% |
| Haruki Murakami      |               3.5 |               2 |             768 |                       3.35% |
| Cormac McCarthy      |               3.5 |               2 |             650 |                       2.84% |

My top six authors by average rating (with ties broken by number of books) are below.

1
2
3
4
5
6
7
8
| Author               | My Average Rating | Number of Books | Number of Pages | Percent of Total Page Count |
|----------------------+-------------------+-----------------+-----------------+-----------------------------|
| Gerald M. Weinberg   |                 5 |               1 |             228 |                       1.00% |
| Kent Beck            |                 5 |               1 |             224 |                       0.98% |
| Jay Fields           |                 5 |               1 |             204 |                       0.89% |
| Kurt Vonnegut        |               4.5 |               2 |             377 |                       1.65% |
| David Foster Wallace |       4.333333333 |               3 |            1753 |                       7.65% |
| David Mitchell       |                 4 |               5 |            2334 |                      10.19% |

I did top six for both of these because otherwise David Mitchell would not have been in the second one. I’ve devoured his writing in the last year and a half for a reason. I’m consistently rating his books highly.

Restricting access to certain routes

Recently I’ve been working on adding authentication and authorization to a Clojure web service. The project uses compojure for routing and friend for authentication and authorization. My pair and I wanted to restrict access to specific routes while leaving some routes completely public. It took a few tries until we figured out how to do this in a way that made us happy.

The rest of this post shows the approximate path we took to our current solution. It focuses on using friend to restrict access to specific routes. It does not go into details about adding authentication to your web service.

Below is an example of the routes before adding authorization checks.

1
2
3
4
5
6
7
8
9
10
(ns example.server
  (:require [compojure.core :refer [GET defroutes] :as compojure]
            [compojure.route :as route]))

(defroutes app
  (GET "/status" _ (status))
  (GET "/cars" _ (fetch-cars))
  (GET "/attributes" _ (fetch-attributes))
  (GET "/drivers" _ (fetch-drivers))
  (route/not-found "NOT FOUND"))

We wanted to make /cars, /attributes, and /drivers require that the request satisfies the :example.server/user role. Requesting /status should not require authorization. The first attempt left us with the following code.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
(ns example.server
  (:require [compojure.core :refer [GET defroutes] :as compojure]
            [compojure.route :as route]
            [cemerick.friend :as friend]))

(defroutes app
  (GET "/status" _ (status))
  (GET "/cars" _
       (friend/authorize #{::user}
                         (fetch-cars)))
  (GET "/attributes" _
       (friend/authorize #{::user}
                         (fetch-attributes)))
  (GET "/drivers" _
       (friend/authorize #{::user}
                         (fetch-drivers)))
  (route/not-found "NOT FOUND"))

The above works but it suffers from repetition. You could write a macro to minimize the repetition but we thought there must be a better way.

After reading more of friend’s documentation we discovered friend/wrap-authorize. This is middleware that only allows requests through if the request satisfies the required roles. Our first pass at using friend/wrap-authorize looked like the following example.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
(ns example.server
  (:require [compojure.core :refer [GET defroutes] :as compojure]
            [compojure.route :as route]
            [cemerick.friend :as friend]))

(defroutes protected-routes
  (GET "/cars" _ (fetch-cars))
  (GET "/attributes" _ (fetch-attributes))
  (GET "/drivers" _ (fetch-drivers)))

(defroutes app
  (GET "/status" _ (status))
  (friend/wrap-authorize protected-routes #{::user})
  (route/not-found "NOT FOUND"))

This is much nicer. The repetition is removed by extracting routes that require authorization into a separate defroutes and wrapping it with friend/wrap-authorize.

This introduces a subtle bug. A response with status code 404 is no longer returned if a non-existent resource is requested and the request is unauthorized. This is because the authorization check happens before matching a route. friend’s documentation warns against this and suggests using compojure/context to scope usage of friend/wrap-authorize. This doesn’t solve the problem but it at least narrows its scope. We can do better.

Compojure 1.2.0 introduced the function wrap-routes. wrap-routes applies middleware after a route is matched. By using this we can have all of the benefits of using friend/wrap-authorize without breaking returning 404 responses.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
(ns example.server
  (:require [compojure.core :refer [GET defroutes] :as compojure]
            [compojure.route :as route]
            [cemerick.friend :as friend]))

(defroutes protected-routes
  (GET "/cars" _ (fetch-cars))
  (GET "/attributes" _ (fetch-attributes))
  (GET "/drivers" _ (fetch-drivers)))

(defroutes app
  (GET "/status" _ (status))
  (compojure/wrap-routes protected-routes
                         friend/wrap-authorize
                         #{::user})
  (route/not-found "NOT FOUND"))

There we have it. A solution without duplication that still responds properly to requests for non-existent resources. compojure/wrap-routes is a useful function to know about.

An Effective Code Review Process

See all of my remote/working-from-home articles here.

The above was tweeted 1 recently and it resulted in some decent discussion about code reviews. In the past six months at Outpace, I’ve been part of a handful of code review sessions that have been extremely productive. After the reviews many developers have expressed shock at the effectiveness of the process. A tweet-sized overview of the process we’ve followed can be found in Carin Meier’s responses to the above tweet. Since you can’t fit details into tweets the rest of this post expands on our code review process.

Some background before we dive into the details. Outpace is a software company that practices, despite every programmer working remotely, nearly 100% pair programming. In addition, the team Carin and I are on do most of our work through GitHub pull requests. Before merging with master, the pull requests are reviewed by other teammates. Between pairing and pull requests many eyes see every line of code as changes are made.

Even with all this, we’ve found value in having more traditional code reviews. We’ve found that different feedback and action items emerge from reviewing code that we already have than from reviews of code changes (e.g., pull requests).

In addition to working for the team described above, the process below has been successfully used to review an internal library where the reviewers where mostly interested users with a couple contributors. It has also been successful on teams that were not adherent to doing work through reviewed pull requests.

The Code Review Process

Step 1: Select the code to review

Typically we do this between a week and two weeks before the code review. Here we identify the code we want to review and create a two-hour meeting on a Friday at the end of day.

Having the meeting late on Friday helps create a relaxed environment. The review becomes a time to unwind, enjoy a beverage of choice, and talk about code. I haven’t met a developer that doesn’t enjoy discussing how to make code better and this lets everyone finish the week doing just that. The code review becomes an uplifting way to finish a week.

Step 2: Open the code review

A few days (typically late Tuesday or early Wednesday) before the Friday code review meeting we start the review. We do this by opening a GitHub pull request. The following steps will create a pull request where you can comment every line of code being reviewed.

  1. Create a local branch.
  2. Delete the code being reviewed and commit locally.
  3. Push the branch to GitHub.
  4. Open a pull request.

These steps are necessary because GitHub pull requests only let you view code that has changed. This process marks every line as deleted, which causes every line to appear the Files changed tab.

Opening the pull request a few days before the review meeting provides a location for pre-meeting comments to be added. This lets reviewers spend a couple days thinking about and commenting on the code. Comments on the pull request indicate a conversation should happen during the code review meeting.

Step 3: The code review meeting

Its finally Friday and time to review the code as a group. Everyone joins a video conference and someone volunteers to lead the code review. At least one other person volunteers to be a note taker.

The leader directs the code review and keeps it moving forward. To do this the leader shares their screen with the video conference and scrolls through the Files changed view of the pull request. When a comment appears on screen the leader stops scrolling and discussion starts.

The comments are read (often silently) and discussion happens. The leader tries to recognize when a conclusion has been reached or when further discussion, outside of the code review, needs to happen. When a conclusion is reached someone (often the leader) states a quick summary and a note taker records the next steps. The next steps are added as additional comments in the comment thread being discussed. As the next steps are recorded the leader moves on to the next comment.

This continues until either time runs out or the group runs out of things to discuss.

After the code review a volunteer turns the next steps comments into Trello cards and we take care of the code review items as part of our usual work.

Results

We’ve seen impressive improvements to code quality in the projects that have undergone this style of code review. Both small and large changes have happened as a result. Code has become simpler, clearer, and better understood. Additionally, the feeling of collective code ownership has increased.

Teammates have been surprised at how well this process has worked. More than a couple have said that historically they have not found code reviews useful but that these were.

This style of code review has worked in a few different settings and I encourage you to give it a shot.


  1. Reading through the discussion on Twitter after this tweet can give some hints as to what it takes to have an effective code review.

ErgoDox: Turn on an LED When Not on the Main Layer

The ErgoDox is a great keyboard. One its appeals is that you can build your own firmware. This makes it possible to rearrange the keys however you want and tweak other functionality. The firmware is fairly advanced and allows you to have multiple layers to your keyboard.

Multiple layers allow the ErgoDox to have fewer physical keys than traditional keyboards. How often do you use an F key? If you are like me the answer is almost never. Why bother having a dedicated key?

Another benefit of multiple layers is that your keyboard is multiple keyboards in one. Do you use the Dvorak layout and your roommate use Qwerty? Program both a Dvorak layer and a Qwerty layer into your keyboard and switch between them with the push of a button.

The only downside I’ve noticed of multiple layers is that I’ll switch between them by accident. This is frustrating as all of a sudden your keyboard works differently and there is no indication that you are on a different layer.

The ErgoDox has a few LEDs in it that I have never used. I don’t even have the needed keys as part of my keyboard layout (Caps lock? Who uses caps lock? I don’t need to shout that often). I decided to repurpose the num lock LED as an indicator that I’m off the main layer.

This was a straight forward change. In the firmware there is a variable that holds what keyboard layer is active. All I had to do to get the num lock LED on when I changed layers was to move the layers_head variable higher in main.c and then change the conditional to turn on the num lock LED when layers_head != 0. This is the commit that does this change. It could have been done as a three line change.

I highly recommend making this change. Now I just need to find a transparent DSA keycaps so I can see the LED easier.

Book Review: Haskell Data Analysis Cookbook

Packt Publishing recently asked me to write a review of the book Haskell Data Analysis Cookbook by Nishant Shukla. The book is broken into small sections that show you how to do a particular task related to data analysis. These tasks vary from reading a csv file or parsing json to listening to a stream of tweets.

I’m not a Haskell programmer. My Haskell experience is limited to reading some books (Learn You a Haskell for Great Good and most of Real World Haskell) and solving some toy problems. All of reading and programming happened years ago though so I’m out of practice.

This book is not for a programmer that is unfamiliar with Haskell. If you’ve never studied it before you’ll find yourself turning towards documentation. If you enter this book with a solid understanding of functional programming you can get by with a smaller understanding of Haskell but you will not get much from the book.

I’ve only read a few cookbook style books and this one followed the usual format. It will be more useful as a quick reference than as something you would read through. It doesn’t dive deep into any topic but does point you toward libraries for various tasks and shows a short example of using them.

A common critic I have of most code examples applies to this book. Most examples do not do qualified imports of namespaces or selective imports of functions from namespaces. This is especially useful when your examples might be read by people who are not be familiar with the languages standard libraries. Reading code and immediately knowing where a function comes from is incredibly useful to understanding.

The code for this book is available on GitHub. It is useful to look at the full example for a section. The examples in the book are broken into parts with English explanations and I found that made it hard to fully understand how the code fit together. Looking at the examples in the GitHub repo helped.

Recommendation

I’d recommend this book for Haskell programmers who find the table of contents interesting. If you read the table of contents and think it would be useful to have a shallow introduction to the topics listed then you’ll find this book useful. It doesn’t give a detailed dive into anything but at least gives you a starting point.

If you either learning Haskell or using Haskell then this book doesn’t have much to offer you.

Building the ErgoDox Keyboard

Earlier this year I built an ErgoDox. The ErgoDox is a split hand mechanical keyboard whose design has been released under the GNU GPLv3. There are a few standard 1 ways of getting the parts. It basically comes down to sourcing all the parts yourself or buying a bundle from Massdrop. I opted to wait until Massdrop was selling them and bought a kit from them.

My ErgoDox

Why?

  1. I’ve used an ergonomic keyboard for years and was intrigued by the split hand design.
  2. I wanted to try out Cherry MX key switches.
  3. Using your thumb for more than just space bar made a lot of sense to me.
  4. The firmware lets you have multiple layers. I thought this could be really useful.
  5. The project sounded fun. I used to make physical devices and this seemed like a good way to do that again.

Buying

As mentioned earlier I bought my parts from Massdrop. In the buy I participated in I had the option of a full hand case or the traditional case and I opted for the full hand. As part of the buy I also bought additional aluminum top layers, a blank set of DSA 2 keycaps, and Cherry MX blue key switches.

If I were doing it again I would not buy the extra aluminum top layer. I built one of my hands using the aluminum and the other with the basic acrylic top. I enjoy both the look and feel of the acrylic hand better.

I would also not buy the set of DSA keycaps from Massdrop. It was convenient and a bit cheaper to buy from them but had I known I could get different colors from Signature Plastics I would have done that.

I also bought eight “deep-dish” DSA keys direct from Signature Plastics. These keys feel different which lets me know when my fingers are above the home row. I’d recommend doing this. You can order from this page.

For key switches I bought Cherry MX Blues through Massdrop. Blues are extremely clicky. You can easily hear me typing in every room of my apartment. It is very satisfying.

After using the keyboard for about a week I also ended up buying some pads for my wrists. I occasionally rest my wrists on the keyboard and the keyboard’s edge would dig into me.

Building

I followed Massdrop’s step-by-step guide and this YouTube video. Another great resource is the community at GeekHack. I’d recommend reading and watching as much as possible before starting your build.

I built this using a cheap soldering iron I’ve had for years, very thin solder, solder wick, and a multimeter. I don’t know if this would have been easier with better tools or not but those got the job done.

While soldering the surface mount diodes I was in the zone and soldered a few locations that didn’t actually need to be soldered. When you are soldering the diodes you should only be soldering them to the locations that have the key silk screen.

My system for minimizing errors while soldering the diodes is the following five steps.

  1. Lay down some solder on one of the pads.
  2. Put the correct end of the diode on top of that solder, reheat and push down.
  3. Test the connection with a multimeter.
  4. Solder the other half of the diode.
  5. Test the connection.

I batched up the steps. I’d do a whole row of the first step, then move to the second for the entire row, then do the third, etc. Being rigorous about testing every connection is important. Catching mistakes early makes it easier to fix the mistakes.

If you solder a diode on the wrong way there is a huge difference (at least for me using solder wick) between the difficulty of fixing the error when only one pad has been soldered versus two pads. I soldered more than one diode backwards and a mistake noticed after soldering only one pad was easy to fix. After soldering both pads it took serious effort.

Eventually you’ll need to cut open a USB cable. I ended up removing the plastic housing using a Dremel. When soldering the wires to the USB holes I was too concerned with it looking pretty and did not leave plenty of wire. This made it harder to solder and as a result I ended up doing a poor job that resulted in a short. After desoldering and destroying another cable, but leaving more wire, I managed to do a better job. I originally noticed the short because I kept getting warnings from my computer about my USB Keyboard drawing too much power.

I’ve annotated a copy of Massdrop’s instructions using Evernote. It contains the above tips inline.

Firmware

After you physically build your keyboard you need to build the firmware. There are a few different firmwares that can work and you can discover those on GeekHack. I’m using a fork of what Massdrop’s graphical configuration tool uses. It is based off benblazak/ergodox-firmware.

One of the exciting things about the ErgoDox is tweaking the firmware. I took the base firmware and modified it to have media key support and light up the LEDs when I’m on any layer besides the base. Some people have added the ability to record keyboard macros and other neat features. I encourage you to take a look at the source even if you use the graphical configuration tool. I haven’t explored beyond benblazak/ergodox-firmware so I can’t compare it to other firmwares.

Conclusion

I really enjoy it. Building it was both fun and frustrating 3.

After using the keyboard for a few months I’ve found that I really only use three (on each hand) of the thumb cluster keys. I also don’t use the keyboard layers too often. I have three layers programmed and I always stay on the main one unless I want to hit a media key.

Would I recommend building your own ErgoDox? If you already can or are willing to learn to solder and this sounds at all interesting to you I would recommend it. The project can be frustrating but the result is great.

The Future

There is still a lot left to explore in the custom keyboard space. Even so I have no plans on leaving the ErgoDox anytime soon. In terms of improving my ErgoDox, I plan on poking around the different firmwares at some point. I’d also like to explore tenting options.

Resources


  1. I feel a bit odd using the word standard to describe acquiring parts to build a keyboard.
  2. This page has diagrams that shows the different keycap families.
  3. Those surface mount diodes are so tiny.

Using Emacs to Explore an HTTP API

Recently I rediscovered an Emacs package that allows you to interact with HTTP endpoints from the comfort of an Emacs buffer. restclient.el provides restclient-mode. This mode allows you to write and execute HTTP requests in an Emacs buffer. This package can be found in MELPA.

Below is an example buffer that touches the GitHub API.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
:github = https://api.github.com

# get users orgs

GET :github/users/jakemcc/orgs

# rendor markdown

POST :github/markdown

{
  "text" : "## Title"
}

# rendor markdown raw

POST :github/markdown/raw
Content-Type: text/plain

Title
-----

The example above has a few interesting snippets. :github is an example of a variable. Lines 8-14 show an example of posting json to an endpoint. You put the data you want to send below the query. The last POST shows how to set headers for a request.

The location of your cursor decides what query to execute. Comments start with # and break your document into sections. The query in the same section as your cursor is the one that is executed. If the cursor is anywhere on lines 3-6 and I hit C-c C-c then Emacs queries GitHub for my organizations. Below is what pops up in a buffer.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
[
    {
        "avatar_url": "https:\/\/avatars.githubusercontent.com\/u\/1826953?",
        "public_members_url": "https:\/\/api.github.com\/orgs\/speakerconf\/public_members{\/member}",
        "members_url": "https:\/\/api.github.com\/orgs\/speakerconf\/members{\/member}",
        "events_url": "https:\/\/api.github.com\/orgs\/speakerconf\/events",
        "repos_url": "https:\/\/api.github.com\/orgs\/speakerconf\/repos",
        "url": "https:\/\/api.github.com\/orgs\/speakerconf",
        "id": 1826953,
        "login": "speakerconf"
    },
    {
        "avatar_url": "https:\/\/avatars.githubusercontent.com\/u\/4711436?",
        "public_members_url": "https:\/\/api.github.com\/orgs\/outpace\/public_members{\/member}",
        "members_url": "https:\/\/api.github.com\/orgs\/outpace\/members{\/member}",
        "events_url": "https:\/\/api.github.com\/orgs\/outpace\/events",
        "repos_url": "https:\/\/api.github.com\/orgs\/outpace\/repos",
        "url": "https:\/\/api.github.com\/orgs\/outpace",
        "id": 4711436,
        "login": "outpace"
    }
]
// HTTP/1.1 200 OK
// Server: GitHub.com
// Date: Fri, 04 Jul 2014 17:34:26 GMT
// Content-Type: application/json; charset=utf-8
// other headers removed for space consideration on blog

C-c C-c triggers restclient-http-send-current which runs a query and pretty prints the result. I could have used C-c C-r to trigger restclient-http-send-current-raw which executes a query and shows the raw result.

It isn’t a perfect mode. One issue I’ve come across is that queries targeting localhost fail. The solution is to query 127.0.0.1.

restclient-mode makes Emacs a useful tool for exploring and testing HTTP APIs. Since it operates on a simple text format it allows you to easily share executable documentation with others. I highly recommend restclient.el.

Comparing Clojure Testing Libraries: Output

I recently became interested in how Clojure testing libraries help you when there is a test failure. This interest resulted in me exploring different Clojure testing libraries. I created the same tests using clojure.test (with and without humane-test-output), expectations, Midje, and Speclj and looked at the output.

I ran all of these examples using Leiningen. Midje, Speclj, and expectations color their output but I’m not going to try to reproduce that here. The color added by Midje and expectations is useful. Speclj color hurt its readability. I use a dark colored terminal and Speclj colors the line that tells where the failure occurs black. This made it hard to read.

I’m not going to show what the tests look like for each testing library past the first comparison. How a test in expressed is important but not what I want to focus on in this post.

Comparing Strings

Going to start off with a basic string comparison. The failing test compares two strings that only differ by one character.

clojure.test

Most (hopefully all) Clojure programmers should be familiar with clojure.test. It is the testing library that is included with Clojure.

1
2
3
4
5
6
(ns example.string-test
  (:require [clojure.test :refer :all]))

(deftest string-comparisons
  (is (= "strings equal" "strings equal"))
  (is (= "space" "spice")))

The output below is what you get when the above test runs. Even in this simple example it isn’t the easiest to read. It doesn’t make it easy to find the expected or actual values.

clojure.test output
1
2
3
FAIL in (string-comparisons) (string_test.clj:6)
expected: (= "space" "spice")
  actual: (not (= "space" "spice"))

Below is the same test but with humane-test-output enabled. It is easy to read the output and see the expected and actual value. It even provides a diff between them although in this situation it isn’t that useful.

clojure.test with humane-test-output
1
2
3
4
5
FAIL in (string-comparisons) (string_test.clj:6)
expected: "space"
  actual: "spice"
    diff: - "space"
          + "spice"
expectations

Another testing library is Jay Field’s expectations. You can see from the example that it has a fairly minimal syntax.

1
2
3
4
5
(ns example.string-expectations
  (:require [expectations :refer :all]))

(expect "strings equal" "strings equal")
(expect "space" "spice")
expectations output
1
2
3
4
5
6
7
8
9
failure in (string_expectations.clj:5) : example.string-expectations
(expect "space" "spice")

           expected: "space"
                was: "spice"

           matches: "sp"
           diverges: "ace"
                  &: "ice"

The output from expectations is very readable. You can easily pick out the expected and actual values. It also shows you where the string starts to diverge.

Speclj

Before writing this post I had zero experience with Micah Martin’s Speclj. Below is my translation of the failing string test and its output.

1
2
3
4
5
6
(ns example.string-spec
  (:require [speclj.core :refer :all]))

(describe "String comparisons"
  (it "have nice error message"
      (should= "space" "spice")))
Speclj
1
2
3
4
  9) String comparisons have nice error message
     Expected: "space"
          got: "spice" (using =)
     /Users/jake/src/jakemcc/example/spec/example/string_spec.clj:7

Speclj’s test output above is an improvement over clojure.test. You can easily find the expected and actual values. It doesn’t provide any help with diagnosing how those values are different.

Midje

I have a little bit of experience with Brian Marick’s Midje. Unlike the other libraries it switches up the assertion syntax. In Midje the expected value is on the right side of =>.

1
2
3
4
5
6
7
8
(ns example.string-test
  (:require [midje.sweet :refer :all]))

(fact "strings are equal"
  "string is equal" => "string is equal")

(fact "strings not equal"
   "spice" => "space")
Midje
1
2
3
FAIL "strings not equal" at (string_test.clj:8)
    Expected: "space"
      Actual: "spice"

Midje’s output is similar to Speclj’s. You can quickly find the expected and actual values but it doesn’t help you spot the difference.

String Comparison Winner

expectations wins for best output. You can easily spot the expected and actual values and it also helps you find the difference between the strings.

The worst output comes from clojure.test. It doesn’t make it easy to spot the difference or even find the expected and actual values.

Comparing Maps

For maps I’ve setup three assertions. The first has an extra key-value pair in the actual value. The second has an extra in the expected value. The final assertion has a different value for the :cheese key. The clojure.test example is below.

1
2
3
4
(deftest map-comparisons
  (is (= {:sheep 1} {:cheese 1 :sheep 1}))
  (is (= {:sheep 1 :cheese 1} {:sheep 1}))
  (is (= {:sheep 1 :cheese 1} {:sheep 1 :cheese 5})))
clojure.test
1
2
3
4
5
6
7
8
9
10
11
FAIL in (map-comparisons) (map_test.clj:5)
expected: (= {:sheep 1} {:cheese 1, :sheep 1})
  actual: (not (= {:sheep 1} {:cheese 1, :sheep 1}))

FAIL in (map-comparisons) (map_test.clj:6)
expected: (= {:sheep 1, :cheese 1} {:sheep 1})
  actual: (not (= {:cheese 1, :sheep 1} {:sheep 1}))

FAIL in (map-comparisons) (map_test.clj:7)
expected: (= {:sheep 1, :cheese 1} {:sheep 1, :cheese 5})
  actual: (not (= {:cheese 1, :sheep 1} {:cheese 5, :sheep 1}))

Unsurprisingly the default clojure.test output for maps suffers from the same problems found in the string comparisons. To find the actual and expected values you need to manually parse the output.

clojure.test with humane-test-output
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
FAIL in (map-comparisons) (map_test.clj:5)
expected: {:sheep 1}
  actual: {:cheese 1, :sheep 1}
    diff: + {:cheese 1}

FAIL in (map-comparisons) (map_test.clj:6)
expected: {:cheese 1, :sheep 1}
  actual: {:sheep 1}
    diff: - {:cheese 1}

FAIL in (map-comparisons) (map_test.clj:7)
expected: {:cheese 1, :sheep 1}
  actual: {:cheese 5, :sheep 1}
    diff: - {:cheese 1}
          + {:cheese 5}

Above is the output of using clojure.test with humane-test-output. It is a big improvement over the default clojure.test. You can quickly see the expected and actual values. Unlike with the string assertions the diff view is actually helpful. The diffs do a good job of helping you identify the error.

expectations
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
failure in (map_expectations.clj:6) : example.map-expectations
(expect {:sheep 1} {:sheep 1, :cheese 1})

           expected: {:sheep 1}
                was: {:cheese 1, :sheep 1}

           in expected, not actual: null
           in actual, not expected: {:cheese 1}

failure in (map_expectations.clj:7) : example.map-expectations
(expect {:sheep 1, :cheese 1} {:sheep 1})

           expected: {:cheese 1, :sheep 1}
                was: {:sheep 1}

           in expected, not actual: {:cheese 1}
           in actual, not expected: null

failure in (map_expectations.clj:8) : example.map-expectations
(expect {:sheep 1, :cheese 5} {:sheep 1, :cheese 1})

           expected: {:cheese 5, :sheep 1}
                was: {:cheese 1, :sheep 1}

           in expected, not actual: {:cheese 5}
           in actual, not expected: {:cheese 1}

expectations does a pretty good job helping you as well. As before, you can clearly read the expected and actual values. expectations also provides some hint as to what is different between the maps. I find the English descriptions a bit easier to read than humane-test-output’s diff view. Still seeing lines like line 7 (in expected, not actual: null) is a bit confusing and the output would be improved if it was suppressed.

I’m just going to lump Speclj and Midje together. The output for each is below. They both improve over clojure.test by making it easy to see the expected and actual value. They both don’t do anything beyond that.

Speclj
1
2
3
4
5
6
7
8
9
10
11
12
13
14
  4) map comparisons have nice error messages when extra entries keys present
     Expected: {:sheep 1}
          got: {:cheese 1, :sheep 1} (using =)
     /Users/jake/src/jakemcc/example/spec/example/map_spec.clj:7

  5) map comparisons have nice error messages when missing an entry
     Expected: {:cheese 1, :sheep 1}
          got: {:sheep 1} (using =)
     /Users/jake/src/jakemcc/example/spec/example/map_spec.clj:9

  6) map comparisons have nice error messages when mismatched values
     Expected: {:cheese 5, :sheep 1}
          got: {:cheese 1, :sheep 1} (using =)
     /Users/jake/src/jakemcc/example/spec/example/map_spec.clj:11
Midje
1
2
3
4
5
6
7
8
9
10
11
FAIL "map is missing an entry" at (map_test.clj:5)
    Expected: {:cheese 1, :sheep 1}
      Actual: {:sheep 1}

FAIL "map has an extra entry" at (map_test.clj:8)
    Expected: {:sheep 1}
      Actual: {:cheese 1, :sheep 1}

FAIL "map has a different value" at (map_test.clj:11)
    Expected: {:cheese 5, :sheep 1}
      Actual: {:cheese 1, :sheep 1}

Map Comparison Winner

Tie between humane-test-output and expectations. Both do a good job of helping the reader spot the difference.

Comparing Sets

Next up are sets. Only two assertions for this section. One with the actual value having an extra member and one test where it is missing a member.

1
2
3
4
5
6
(ns example.set-test
  (:require [clojure.test :refer :all]))

(deftest set-comparisons
  (is (= #{:a :b} #{:a :b :c}))
  (is (= #{:a :b :c} #{:a :b})))

First up is the basic clojure.test output. It suffers from the same problem it has suffered this entire time. It doesn’t make it easy to read the expected and actual values.

clojure.test
1
2
3
4
5
6
7
FAIL in (set-comparisons) (set_test.clj:5)
expected: (= #{:b :a} #{:c :b :a})
  actual: (not (= #{:b :a} #{:c :b :a}))

FAIL in (set-comparisons) (set_test.clj:6)
expected: (= #{:c :b :a} #{:b :a})
  actual: (not (= #{:c :b :a} #{:b :a}))

No surprises with humane-test-output. It improves the clojure.test output by making it easy to read the expected and actual values. The diff view also helps figure out what is causing the assertion to fail.

clojure.test with humane-test-output
1
2
3
4
5
6
7
8
9
FAIL in (set-comparisons) (set_test.clj:5)
expected: #{:b :a}
  actual: #{:c :b :a}
    diff: + #{:c}

FAIL in (set-comparisons) (set_test.clj:6)
expected: #{:c :b :a}
  actual: #{:b :a}
    diff: - #{:c}

expectations once again delivers nice output. It continues to be easy to find the expected and actual values and helps you spot the differences with a diff view.

expectations
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
failure in (set_expectations.clj:4) : example.set-expectations
(expect #{:b :a} #{:c :b :a})

           expected: #{:b :a}
                was: #{:c :b :a}

           in expected, not actual: null
           in actual, not expected: #{:c}

failure in (set_expectations.clj:5) : example.set-expectations
(expect #{:c :b :a} #{:b :a})

           expected: #{:c :b :a}
                was: #{:b :a}

           in expected, not actual: #{:c}
           in actual, not expected: null

Speclj and Midje both have better output than the basic clojure.test.

Speclj
1
2
3
4
5
6
7
8
9
  7) set comparisons have nice error messages when missing item
     Expected: #{:b :a}
          got: #{:c :b :a} (using =)
     /Users/jake/src/jakemcc/example/spec/example/set_spec.clj:9

  8) set comparisons have nice error messages when more items
     Expected: #{:c :b :a}
          got: #{:b :a} (using =)
     /Users/jake/src/jakemcc/example/spec/example/set_spec.clj:11
Midje
1
2
3
4
5
6
7
FAIL "set is superset of expected" at (set_test.clj:5)
    Expected: #{:a :b}
      Actual: #{:a :b :c}

FAIL "set is subset of expected" at (set_test.clj:8)
    Expected: #{:a :b :c}
      Actual: #{:a :b}

Set Comparison Winner

Similar to the winner of the map comparisons I’m going to split the victory between expectations and humane-test-output.

Comparing Lists

Next up we compare lists (and lists to vectors). There are three comparisons; one with an extra element, one with same length but a mismatched element, and one comparing a vector and list with drastically different contents.

1
2
3
4
5
6
7
(ns example.seq-test
  (:require [clojure.test :refer :all]))

(deftest list-comparisons
  (is (= '(1 2 3) '(1 2 3 4)))
  (is (= '(1 2 4) '(1 2 3)))
  (is (= '(9 8 7) [1 2 3])))

First up clojure.test. Same issues as with all the previous comparisons.

clojure.test
1
2
3
4
5
6
7
8
9
10
11
FAIL in (list-comparisons) (seq_test.clj:5)
expected: (= (quote (1 2 3)) (quote (1 2 3 4)))
  actual: (not (= (1 2 3) (1 2 3 4)))

FAIL in (list-comparisons) (seq_test.clj:6)
expected: (= (quote (1 2 4)) (quote (1 2 3)))
  actual: (not (= (1 2 4) (1 2 3)))

FAIL in (list-comparisons) (seq_test.clj:7)
expected: (= (quote (9 8 7)) [1 2 3])
  actual: (not (= (9 8 7) [1 2 3]))

Once again humane-test-output improves upon clojure.test. Only interesting difference from previous comparisons is that the diff view ends up having nil values in it where the elements are the same.

clojure.test with humane-test-output
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
FAIL in (list-comparisons) (seq_test.clj:5)
expected: (1 2 3)
  actual: (1 2 3 4)
    diff: + [nil nil nil 4]

FAIL in (list-comparisons) (seq_test.clj:6)
expected: (1 2 4)
  actual: (1 2 3)
    diff: - [nil nil 4]
          + [nil nil 3]

FAIL in (list-comparisons) (seq_test.clj:7)
expected: (9 8 7)
  actual: [1 2 3]
    diff: - [9 8 7]
          + [1 2 3]

expectations continues to have good output. It tries to help you out as well. You’ll notice that it also has nil values inserted where the lists are the same.

expectations
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
failure in (list_expectations.clj:4) : example.list-expectations
(expect '(1 2 3) '(1 2 3 4))

           expected: (1 2 3)
                was: (1 2 3 4)

           in expected, not actual: null
           in actual, not expected: [nil nil nil 4]
           actual is larger than expected

failure in (list_expectations.clj:5) : example.list-expectations
(expect '(1 2 4) '(1 2 3))

           expected: (1 2 4)
                was: (1 2 3)

           in expected, not actual: [nil nil 4]
           in actual, not expected: [nil nil 3]

failure in (list_expectations.clj:6) : example.list-expectations
(expect '(9 8 7) [1 2 3])

           expected: (9 8 7)
                was: [1 2 3]

           in expected, not actual: [9 8 7]
           in actual, not expected: [1 2 3]

Unsurprisingly, Speclj and Midje are better than clojure.test but again don’t go beyond making easy to find the expected and actual values.

Speclj
1
2
3
4
5
6
7
8
9
10
11
12
13
14
  1) List/vector comparisons when there is an extra element
     Expected: (1 2 3)
          got: (1 2 3 4) (using =)
     /Users/jake/src/jakemcc/example/spec/example/string_spec.clj:7

  2) List/vector comparisons when there is a mismatched element
     Expected: (1 2 4)
          got: (1 2 3) (using =)
     /Users/jake/src/jakemcc/example/spec/example/string_spec.clj:9

  3) List/vector comparisons when comparing different types
     Expected: (9 8 7)
          got: [1 2 3] (using =)
     /Users/jake/src/jakemcc/example/spec/example/string_spec.clj:11
Midje
1
2
3
4
5
6
7
8
9
10
11
FAIL "lists are different sizes" at (seq_test.clj:5)
    Expected: (1 2 3)
      Actual: (1 2 3 4)

FAIL "lists have different entries" at (seq_test.clj:8)
    Expected: (1 2 4)
      Actual: (1 2 3)

FAIL "compare very different list like values" at (seq_test.clj:14)
    Expected: (9 8 7)
      Actual: [1 2 3]

List Comparison Winner

I find the clojure.test with humane-test-output to be a bit easier to read than expectations. Both have better output than the basic clojure.test, Speclj, and Midje.

Overall Winner

If I were picking a testing library based entirely on what a failing test looks like I would use expectations. My second pick would be clojure.test with humane-test-output.

It is great that Clojure ships with clojure.test. It is unfortunate that it does so little to help you read a failing test. Every library I tried has better output than clojure.test.

Addendum

Added 2014/06/23

Colin Jones points out that Speclj provides should==. should== checks that the expected and actual value have the same contents. He provided a gist that shows the difference.

Quicker Feedback From Failing Tests

Over the last couple of years I’ve desired quicker feedback from my Clojure tests. This has resulted in the development of lein-autoexpect and more recently lein-test-refresh. Each tool monitors your project for changes and on change uses tools.namespace to reload your code and then reruns either your expectations or clojure.test tests. Using tools like these has changed my development process.

Version 0.5.0 of lein-test-refresh was released last week. This release enables even quicker feedback by tracking which tests fail and after reloading your code it runs those tests first. Only when your previously failed tests pass does it then rerun all of your tests.

lein-test-refresh has had quite a few features added since I last wrote about it. The readme will always have the latest list but as of the time of writing this they include:

  • Reloads code and reruns tests on changes to your project’s code.
  • Runs previously failing tests first.
  • Supports custom notification commands.
  • Built in Growl support.
  • Can notify after test success and failure or just after failure.
  • Supports a subset of Leiningen test selectors.
  • Reports on your tests running time.

I don’t have enough experience with the new lein-test-refresh to say how having failing tests will affect my development practices. I don’t expect this to change my development practices but it will enable quicker feedback. Quick feedback cycles are what it is all about.

Acknowledgments

Most of the ‘rerun failed tests first’ feature was hashed out and spiked during a mob programming session organized by Zee Spencer. This happened at a company conference put on by Outpace in Las Vegas. Many developers were involved but two that most influenced the final result were Joel Holdbrooks and Timothy Pratley.

Book Review: Clojure for Machine Learning

I was recently given a review copy of Clojure for Machine Learning. I have an academic familiarity with machine learning techniques and presented on a few at speakerconf 2012. I haven’t explored machine learning in Clojure since preparing that talk and was excited to read a book on the topic.

The book gives a shallow introduction to many different topics. It does so through a bit of mathematics and much more code. Depending on the section, the code examples implement the algorithm being discussed, show you how to use a specific library, or do both.

An aspect I particularly enjoy about the code examples is that they always start by showing what dependencies should be added to your project.clj file. This is done even if the library has been used in a previous chapter. Because of this every example can stand on its own.

Something that can almost always be improved about Clojure examples is that namespaces should be referenced using the require form with a namespace alias. Even if that require requires a namespace with a terrible alias, such as (require '[example :as e]), it makes the example code easier to understand. Being able to read e/a-func instead of a-func makes it more explicit as to where that function is located and aides understanding.

I rate all my books by the goodreads five star scale1. This book earns three stars. Even with my limited machine learning background I didn’t learn anything new but I was introduced to some Clojure libraries and enjoyed seeing Clojure implementations of machine learning techniques.

If you enjoy Clojure and the table of contents excites you then you’ll most likely find this book interesting. If you want to maximize your learning I’d recommend taking an online course in machine learning2. It will be a larger time investment but you’ll leave with a deeper understanding.


  1. 1 star = did not like, 2 stars = it was ok, 3 stars = liked it, 4 stars = really liked it, 5 stars = loved it.
  2. I took the original offering from Stanford when it was first offered. Post about it here.