Jake McCrary

Remote Pairing

| Comments

Over a year ago I joined Outpace. All of Outpace’s developers are remote but we still practice pair programming. As a result I’ve done a lot of remote pairing. I was skeptical before joining that it would work well and I’m happy to report that I was wrong. Remote pairing works.

Why remote pairing?

The usual pair programming benefits apply to remote pairing; more people know the code, quality is higher, and it provides an opportunity for mentorship. Another benefit, more beneficial in a remote setting, is that it increases social interaction.

The most common response I receive when I tell someone I work from my apartment is “I’d miss the interaction with co-workers.” When you work remote you do miss out on the usual in office interaction. Pair programming helps replace some of this. It helps you build and maintain relationships with your remote colleagues.

Communication

Communication is an important part of pair programming. When you’re pairing in person you use both physical and vocal communication. When remote pairing you primarily use vocal communication. You can pick up on some physical cues with video chat but it is hard. You will never notice your pair reaching for their keyboard.

I’ve used Google Hangouts, Zoom, and Skype for communication. Currently I’m primarily using Zoom. It offers high quality video and audio and usually doesn’t consume too many resources.

I recommend not using your computers built-in microphone. You should use headphones with a mic or a directional microphone. You’ll sound better and you’ll stop your pair from hearing themselves through your computer.

I use these headphones1. They are cheap, light, and open-eared but are wired. I’ve been told I sound the best when I’m using them. I also own these wireless headphones. They are closed-ear, heavier, and wireless. The wireless is great but the closed-ear design causes me to talk differently and by the end of the day my throat is hoarse. Both of these headphones are widely used by my colleagues and I don’t think you can go wrong with either one.

Some people don’t like wearing headphones all day. If you are one of those I’d recommend picking up a directional microphone. Many of my colleagues use a Snowball.

Connecting the machines

So now you can communicate with your pair. It is time to deal with the main problem in remote pairing. How do you actually work on the same code with someone across the world?

At Outpace we’ve somewhat cheated and have standardized our development hardware. Everyone has a computer running OS X and, if they want it, at least one 27 inch monitor (mostly Apple 27 inch displays or a Dell) with a resolution of 2560x1440. Since everyone has nearly identical hardware and software we are able to pair using OS X’s built-in screen sharing. This allows full sharing of the host’s desktop. This full desktop sharing is the best way to emulate working physically next to your pair. This enable the use of any editor and lets you both look at the same browser windows (useful for testing UIs or reading reference material). With decent internet connections both programmers can write code with minimal lag. This is my preferred way of pairing.

Another option that works well is tmate. tmate is a fork of tmux that makes remote pairing easy. It makes it dead simple to have remote developer connect to your machine and share your terminal. This means you are stuck using tools that work in a terminal and, if you are working on a user interface, you need to share that some other way. There generally is less lag when the remote developer is typing.

A third option is to have the host programmer share their screen using screen sharing built-in to Google Hangouts or Zoom. This is a quick way to share a screen and is my preferred way of sharing GUIs with more than one other person. With both Zoom and Google Hangouts the remote developer can control the host’s machine but it isn’t a great experience. If you are pairing this way the remote developer rarely touches the keyboard.

Soloing

It might seem weird to have a section on soloing in an article about remote pairing. Soloing happens and even in an environment that almost entirely pairs it is important. Not everyone can or wants to pair 100% of the time. Soloing can be recharging. It is important to be self-aware and recognize if you need solo time. Below are a few tips for getting that solo time.

One way to introduce solo time is to take your lunch at a different time than your pair. This provides both of you and your pair with an opportunity to do a bit of soloing.

Other short soloing opportunities happen because of meetings and interviews. It isn’t uncommon for half of a pair to leave for a bit to join a meeting, give an interview, or jump over to help out another developer for a bit.

Soloing also happens as a result of uneven team numbers. If your team is odd numbered than there are plenty of opportunities for being a solo developer. Try to volunteer to be the solo developer but be aware of becoming too isolated.

Conclusion

Remote pairing works. Working at Outpace has shown me how well it can work. With the right people and modern technology almost makes it feel as if your pair is in the same room as you.


  1. This link and all others to Amazon in this post are affiliate links.

Overview of My Leiningen profiles.clj

| Comments

Leiningen, a Clojure build tool, has the concept of profiles. One thing profiles are useful for is allowing you to have development tools available to a project without having them as dependencies when you release your project. An example of when you might want to do this is when you are using a testing library like expectations.

Some development tools, such as lein-test-refresh, are useful to have across most of your Clojure projects. Rather nicely, Leiningen supports adding global profiles to ~/.lein/profiles.clj. These profiles are available in all your projects.

Below is most of my profiles.clj. I’ve removed some sensitive settings and what is left are the development tools that I find useful.

Entire :user profile
1
2
3
4
5
6
7
8
9
10
11
12
13
{:user {:plugin-repositories [["private-plugins" {:url "private repo url"}]]
        :dependencies [[pjstadig/humane-test-output "0.6.0"]]
        :injections [(require 'pjstadig.humane-test-output)
                     (pjstadig.humane-test-output/activate!)]
        :plugins [[cider/cider-nrepl "0.8.2"]
                  [refactor-nrepl "0.2.2"]
                  [com.jakemccrary/lein-test-refresh "0.5.5"]
                  [lein-autoexpect "1.4.2"]
                  [lein-ancient "0.5.5"]
                  [jonase/eastwood "0.2.1"]
                  [lein-kibit "0.0.8"]
                  [lein-pprint "1.1.2"]]
        :test-refresh {:notify-command ["terminal-notifier" "-title" "Tests" "-message"]}}}

:plugin-repositories [["private-plugins" {:url "private repo url"}]] sets a private plugin repository. This allows me to use Outpace’s private Leiningen templates for setting up new projects for work.

The next few lines are all related. They setup humane-test-output. humane-test-output makes clojure.test output more readable. It makes using clojure.test much more enjoyable. I highly recommend it. Sample output can be found in my Comparing Clojure Testing Libraries post.

humane-test-output setup in the :user profile
1
2
3
:dependencies [[pjstadig/humane-test-output "0.6.0"]]
:injections [(require 'pjstadig.humane-test-output)
             (pjstadig.humane-test-output/activate!)]

Next we get to my :plugins section. This is the bulk of my profiles.clj.

:plugins section of my :user profile
1
2
3
4
5
6
7
8
:plugins [[cider/cider-nrepl "0.8.2"]
          [refactor-nrepl "0.2.2"]
          [com.jakemccrary/lein-test-refresh "0.5.5"]
          [lein-autoexpect "1.4.2"]
          [lein-ancient "0.5.5"]
          [jonase/eastwood "0.2.1"]
          [lein-kibit "0.0.8"]
          [lein-pprint "1.1.2"]]

The first entry is for cider/cider-nrepl. I write Clojure using Emacs and CIDER and much of CIDER’s functionality exists in nrepl middleware found in cider/cider-nrepl. This dependency is required for me to be effective while writing Clojure.

refactor-nrepl is next. clj-refactor.el requires it for some refactorings. I actually don’t use any of those refactorings (I only use move to let, extract to let, and introduce let refactorings) but I still keep it around.

com.jakemccrary/lein-test-refresh is next. This lets me use lein-test-refresh globally. lein-test-refresh runs your clojure.test tests whenever a file changes in your project. This is another key development tool in my process.

Up next is lein-autoexpect. It was the first Leiningen plugin I wrote and it enables continuous testing with expectations.

Both lein-autoexpect and lein-test-refresh are projects I created and maintain. Writing lein-autoexpect was my first exposure to continuous testing and it changed how I develop code. I find it frustrating to develop without such a tool.

Next up is lein-ancient. It checks your project.clj for outdated dependencies and plugins. It isn’t something that gets used every day but it is super useful when you need it.

The next two entries are for jonase/eastwood and lein-kibit. They are both tools that look at your Clojure code and report common mistakes. I don’t use either consistently but I do find them useful. I’ve found bugs with eastwood.

The final plugin is lein-pprint. lein-pprint prints out your project map. It is useful for trying to grasp what is going on when messing around with various Leiningen options.

The final part, seen below, of my profiles.clj is configuration for lein-test-refresh. It configures lein-test-refresh to use terminal-notifier to notify me when my tests pass or fail. Using a continuous tester that allows flexible notification is useful. Not having to glance at a terminal to see if your tests are passing or failing is great.

1
:test-refresh {:notify-command ["terminal-notifier" "-title" "Tests" "-message"]}

That is my ~/.lein/profiles.clj. I don’t think it contains anything mind blowing but it definitely contains a useful collection of Clojure development tools. I encourage you to check out them out and to think about what tools you should be putting into your global :user profile.

Reading in 2014

| Comments

At the beginning of last year I took some time and reviewed my 2013 reading using Clojure and Incanter to generate some stats. It was a useful exercise to reflect back on my reading and play around with Incanter again.

Over the last couple of weeks I’ve taken a similar look at my 2014 reading. The rest of this post highlights some of the top books from the previous year and then posts some numbers at the end.

I review every book I read using Goodreads. If you want to see more of what I’ve been reading you can find me here. I track and review every book I read and have found this practice to be extremely rewarding.

2014 Goals

I entered 2014 without a volume goal. Unlike 2013, I didn’t have a page or book count goal. I entered 2014 with the desire to reread two specific books and the nebulous goal of reading more non-fiction.

2014 Results

I ended up setting a new volume record. I read 69 books for a total of almost 23,000 pages. I also read every week of Day One1, a weekly literary journal containing one short story and one poem from new authors. This doesn’t count towards my page or book count but is reading I enjoy. It exposes me to many different styles.

More than a third of my reading was non-fiction. I don’t have numbers for 2013 but that feels like an increase. I consider my goal of reading more non-fiction achieved.

I also reread the two books I had planned on rereading. I wanted to reread Infinite Jest and Hard-Boiled Wonderland and the End of the World and succeeded in rereading both of them.

Recommendations

I awarded seven books a five out of five star rating. I’ve listed them below in (in no particular order). Each book I’d recommend without hesitation. Instead of reworking or copying my previous reviews I’ve provided links to Goodreads. The titles link to Amazon.

I’m recommending a specific translation of Meditations. I attempted to read different one first and it was so painful to read I ended up giving up. The linked translation is modern and contains a useful forward giving you background information on the time.

I only read one series this year but it was a good one. The Magicians, by Lev Grossman, was recommended by a friend who described it as “Harry Potter but with characters battling depression.” I’m not sure that fully captures the feel of the series but it is a start. The series introduces you to a world like our own but with magic. You follow cynical, self-absorbed students as they attend school, graduate, and grow up living in both the magical and non-magical world. The first book in the series is the weakest so if you read that and find it enjoyable you should definitely pick up the next two books.

2015 Goals

2015 isn’t going to have an easily measured goal. I don’t feel the need to set number of books or pages goals any more. I’m hoping to increase the quality of my reading. This is a pretty unclear goal. To me this doesn’t mean increasing the average rating of books I read but instead I want to get more out of what I read. I want to think a bit deeper about the subjects I’m reading.

2014 Measurements

Below are some random measurements that are probably only interesting to me.

This year I recorded the format of the books I read. This was the year of the ebook; over 90% of the books I read were electronic. I’d guess that this is a higher percentage of ebooks than previous years. I wish I had recorded the formats read in previous years.

1
2
3
4
5
| Binding   | Number of books |
|-----------+-----------------|
| Hardcover |               1 |
| Paperback |               4 |
| Kindle    |              64 |

My average rating has been going down over the last four years.

1
2
3
4
5
6
| Year | Average Rating |
|------+----------------|
| 2011 | 3.84           |
| 2012 | 3.66           |
| 2013 | 3.67           |
| 2014 | 3.48           |

In 2014, three authors composed nearly 25% of my reading (by page count). The top six authors by page count are below.

1
2
3
4
5
6
7
8
| Author               | My Average Rating | Number of Books | Number of Pages | Percent of Total Page Count |
|----------------------+-------------------+-----------------+-----------------+-----------------------------|
| David Mitchell       |                 4 |               5 |            2334 |                      10.19% |
| David Foster Wallace |       4.333333333 |               3 |            1753 |                       7.65% |
| Lev Grossman         |       3.666666667 |               3 |            1244 |                       5.43% |
| Marisha Pessl        |               3.5 |               2 |            1153 |                       5.03% |
| Haruki Murakami      |               3.5 |               2 |             768 |                       3.35% |
| Cormac McCarthy      |               3.5 |               2 |             650 |                       2.84% |

My top six authors by average rating (with ties broken by number of books) are below.

1
2
3
4
5
6
7
8
| Author               | My Average Rating | Number of Books | Number of Pages | Percent of Total Page Count |
|----------------------+-------------------+-----------------+-----------------+-----------------------------|
| Gerald M. Weinberg   |                 5 |               1 |             228 |                       1.00% |
| Kent Beck            |                 5 |               1 |             224 |                       0.98% |
| Jay Fields           |                 5 |               1 |             204 |                       0.89% |
| Kurt Vonnegut        |               4.5 |               2 |             377 |                       1.65% |
| David Foster Wallace |       4.333333333 |               3 |            1753 |                       7.65% |
| David Mitchell       |                 4 |               5 |            2334 |                      10.19% |

I did top six for both of these because otherwise David Mitchell would not have been in the second one. I’ve devoured his writing in the last year and a half for a reason. I’m consistently rating his books highly.


  1. This link and all others to Amazon in this post is an affiliate link.

Restricting Access to Certain Routes

| Comments

Recently I’ve been working on adding authentication and authorization to a Clojure web service. The project uses compojure for routing and friend for authentication and authorization. My pair and I wanted to restrict access to specific routes while leaving some routes completely public. It took a few tries until we figured out how to do this in a way that made us happy.

The rest of this post shows the approximate path we took to our current solution. It focuses on using friend to restrict access to specific routes. It does not go into details about adding authentication to your web service.

Below is an example of the routes before adding authorization checks.

1
2
3
4
5
6
7
8
9
10
(ns example.server
  (:require [compojure.core :refer [GET defroutes] :as compojure]
            [compojure.route :as route]))

(defroutes app
  (GET "/status" _ (status))
  (GET "/cars" _ (fetch-cars))
  (GET "/attributes" _ (fetch-attributes))
  (GET "/drivers" _ (fetch-drivers))
  (route/not-found "NOT FOUND"))

We wanted to make /cars, /attributes, and /drivers require that the request satisfies the :example.server/user role. Requesting /status should not require authorization. The first attempt left us with the following code.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
(ns example.server
  (:require [compojure.core :refer [GET defroutes] :as compojure]
            [compojure.route :as route]
            [cemerick.friend :as friend]))

(defroutes app
  (GET "/status" _ (status))
  (GET "/cars" _
       (friend/authorize #{::user}
                         (fetch-cars)))
  (GET "/attributes" _
       (friend/authorize #{::user}
                         (fetch-attributes)))
  (GET "/drivers" _
       (friend/authorize #{::user}
                         (fetch-drivers)))
  (route/not-found "NOT FOUND"))

The above works but it suffers from repetition. You could write a macro to minimize the repetition but we thought there must be a better way.

After reading more of friend’s documentation we discovered friend/wrap-authorize. This is middleware that only allows requests through if the request satisfies the required roles. Our first pass at using friend/wrap-authorize looked like the following example.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
(ns example.server
  (:require [compojure.core :refer [GET defroutes] :as compojure]
            [compojure.route :as route]
            [cemerick.friend :as friend]))

(defroutes protected-routes
  (GET "/cars" _ (fetch-cars))
  (GET "/attributes" _ (fetch-attributes))
  (GET "/drivers" _ (fetch-drivers)))

(defroutes app
  (GET "/status" _ (status))
  (friend/wrap-authorize protected-routes #{::user})
  (route/not-found "NOT FOUND"))

This is much nicer. The repetition is removed by extracting routes that require authorization into a separate defroutes and wrapping it with friend/wrap-authorize.

This introduces a subtle bug. A response with status code 404 is no longer returned if a non-existent resource is requested and the request is unauthorized. This is because the authorization check happens before matching a route. friend’s documentation warns against this and suggests using compojure/context to scope usage of friend/wrap-authorize. This doesn’t solve the problem but it at least narrows its scope. We can do better.

Compojure 1.2.0 introduced the function wrap-routes. wrap-routes applies middleware after a route is matched. By using this we can have all of the benefits of using friend/wrap-authorize without breaking returning 404 responses.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
(ns example.server
  (:require [compojure.core :refer [GET defroutes] :as compojure]
            [compojure.route :as route]
            [cemerick.friend :as friend]))

(defroutes protected-routes
  (GET "/cars" _ (fetch-cars))
  (GET "/attributes" _ (fetch-attributes))
  (GET "/drivers" _ (fetch-drivers)))

(defroutes app
  (GET "/status" _ (status))
  (compojure/wrap-routes protected-routes
                         friend/wrap-authorize
                         #{::user})
  (route/not-found "NOT FOUND"))

There we have it. A solution without duplication that still responds properly to requests for non-existent resources. compojure/wrap-routes is a useful function to know about.

An Effective Code Review Process

| Comments

The above was tweeted 1 recently and it resulted in some decent discussion about code reviews. In the past six months at Outpace, I’ve been part of a handful of code review sessions that have been extremely productive. After the reviews many developers have expressed shock at the effectiveness of the process. A tweet-sized overview of the process we’ve followed can be found in Carin Meier’s responses to the above tweet. Since you can’t fit details into tweets the rest of this post expands on our code review process.

Some background before we dive into the details. Outpace is a software company that practices, despite every programmer working remotely, nearly 100% pair programming. In addition, the team Carin and I are on do most of our work through GitHub pull requests. Before merging with master, the pull requests are reviewed by other teammates. Between pairing and pull requests many eyes see every line of code as changes are made.

Even with all this, we’ve found value in having more traditional code reviews. We’ve found that different feedback and action items emerge from reviewing code that we already have than from reviews of code changes (e.g., pull requests).

In addition to working for the team described above, the process below has been successfully used to review an internal library where the reviewers where mostly interested users with a couple contributors. It has also been successful on teams that were not adherent to doing work through reviewed pull requests.

The Code Review Process

Step 1: Select the code to review

Typically we do this between a week and two weeks before the code review. Here we identify the code we want to review and create a two-hour meeting on a Friday at the end of day.

Having the meeting late on Friday helps create a relaxed environment. The review becomes a time to unwind, enjoy a beverage of choice, and talk about code. I haven’t met a developer that doesn’t enjoy discussing how to make code better and this lets everyone finish the week doing just that. The code review becomes an uplifting way to finish a week.

Step 2: Open the code review

A few days (typically late Tuesday or early Wednesday) before the Friday code review meeting we start the review. We do this by opening a GitHub pull request. The following steps will create a pull request where you can comment every line of code being reviewed.

  1. Create a local branch.
  2. Delete the code being reviewed and commit locally.
  3. Push the branch to GitHub.
  4. Open a pull request.

These steps are necessary because GitHub pull requests only let you view code that has changed. This process marks every line as deleted, which causes every line to appear the Files changed tab.

Opening the pull request a few days before the review meeting provides a location for pre-meeting comments to be added. This lets reviewers spend a couple days thinking about and commenting on the code. Comments on the pull request indicate a conversation should happen during the code review meeting.

Step 3: The code review meeting

Its finally Friday and time to review the code as a group. Everyone joins a video conference and someone volunteers to lead the code review. At least one other person volunteers to be a note taker.

The leader directs the code review and keeps it moving forward. To do this the leader shares their screen with the video conference and scrolls through the Files changed view of the pull request. When a comment appears on screen the leader stops scrolling and discussion starts.

The comments are read (often silently) and discussion happens. The leader tries to recognize when a conclusion has been reached or when further discussion, outside of the code review, needs to happen. When a conclusion is reached someone (often the leader) states a quick summary and a note taker records the next steps. The next steps are added as additional comments in the comment thread being discussed. As the next steps are recorded the leader moves on to the next comment.

This continues until either time runs out or the group runs out of things to discuss.

After the code review a volunteer turns the next steps comments into Trello cards and we take care of the code review items as part of our usual work.

Results

We’ve seen impressive improvements to code quality in the projects that have undergone this style of code review. Both small and large changes have happened as a result. Code has become simpler, clearer, and better understood. Additionally, the feeling of collective code ownership has increased.

Teammates have been surprised at how well this process has worked. More than a couple have said that historically they have not found code reviews useful but that these were.

This style of code review has worked in a few different settings and I encourage you to give it a shot.


  1. Reading through the discussion on Twitter after this tweet can give some hints as to what it takes to have an effective code review.

ErgoDox: Turn on an LED When Not on the Main Layer

| Comments

The ErgoDox is a great keyboard. One its appeals is that you can build your own firmware. This makes it possible to rearrange the keys however you want and tweak other functionality. The firmware is fairly advanced and allows you to have multiple layers to your keyboard.

Multiple layers allow the ErgoDox to have fewer physical keys than traditional keyboards. How often do you use an F key? If you are like me the answer is almost never. Why bother having a dedicated key?

Another benefit of multiple layers is that your keyboard is multiple keyboards in one. Do you use the Dvorak layout and your roommate use Qwerty? Program both a Dvorak layer and a Qwerty layer into your keyboard and switch between them with the push of a button.

The only downside I’ve noticed of multiple layers is that I’ll switch between them by accident. This is frustrating as all of a sudden your keyboard works differently and there is no indication that you are on a different layer.

The ErgoDox has a few LEDs in it that I have never used. I don’t even have the needed keys as part of my keyboard layout (Caps lock? Who uses caps lock? I don’t need to shout that often). I decided to repurpose the num lock LED as an indicator that I’m off the main layer.

This was a straight forward change. In the firmware there is a variable that holds what keyboard layer is active. All I had to do to get the num lock LED on when I changed layers was to move the layers_head variable higher in main.c and then change the conditional to turn on the num lock LED when layers_head != 0. This is the commit that does this change. It could have been done as a three line change.

I highly recommend making this change. Now I just need to find a transparent DSA keycaps so I can see the LED easier.

Book Review: Haskell Data Analysis Cookbook

| Comments

Packt Publishing recently asked me to write a review of the book Haskell Data Analysis Cookbook by Nishant Shukla. The book is broken into small sections that show you how to do a particular task related to data analysis. These tasks vary from reading a csv file or parsing json to listening to a stream of tweets.

I’m not a Haskell programmer. My Haskell experience is limited to reading some books (Learn You a Haskell for Great Good and most of Real World Haskell) and solving some toy problems. All of reading and programming happened years ago though so I’m out of practice.

This book is not for a programmer that is unfamiliar with Haskell. If you’ve never studied it before you’ll find yourself turning towards documentation. If you enter this book with a solid understanding of functional programming you can get by with a smaller understanding of Haskell but you will not get much from the book.

I’ve only read a few cookbook style books and this one followed the usual format. It will be more useful as a quick reference than as something you would read through. It doesn’t dive deep into any topic but does point you toward libraries for various tasks and shows a short example of using them.

A common critic I have of most code examples applies to this book. Most examples do not do qualified imports of namespaces or selective imports of functions from namespaces. This is especially useful when your examples might be read by people who are not be familiar with the languages standard libraries. Reading code and immediately knowing where a function comes from is incredibly useful to understanding.

The code for this book is available on GitHub. It is useful to look at the full example for a section. The examples in the book are broken into parts with English explanations and I found that made it hard to fully understand how the code fit together. Looking at the examples in the GitHub repo helped.

Recommendation

I’d recommend this book for Haskell programmers who find the table of contents interesting. If you read the table of contents and think it would be useful to have a shallow introduction to the topics listed then you’ll find this book useful. It doesn’t give a detailed dive into anything but at least gives you a starting point.

If you either learning Haskell or using Haskell then this book doesn’t have much to offer you.

Building the ErgoDox Keyboard

| Comments

Earlier this year I built an ErgoDox. The ErgoDox is a split hand mechanical keyboard whose design has been released under the GNU GPLv3. There are a few standard 1 ways of getting the parts. It basically comes down to sourcing all the parts yourself or buying a bundle from Massdrop. I opted to wait until Massdrop was selling them and bought a kit from them.

My ErgoDox

Why?

  1. I’ve used an ergonomic keyboard for years and was intrigued by the split hand design.
  2. I wanted to try out Cherry MX key switches.
  3. Using your thumb for more than just space bar made a lot of sense to me.
  4. The firmware lets you have multiple layers. I thought this could be really useful.
  5. The project sounded fun. I used to make physical devices and this seemed like a good way to do that again.

Buying

As mentioned earlier I bought my parts from Massdrop. In the buy I participated in I had the option of a full hand case or the traditional case and I opted for the full hand. As part of the buy I also bought additional aluminum top layers, a blank set of DSA 2 keycaps, and Cherry MX blue key switches.

If I were doing it again I would not buy the extra aluminum top layer. I built one of my hands using the aluminum and the other with the basic acrylic top. I enjoy both the look and feel of the acrylic hand better.

I would also not buy the set of DSA keycaps from Massdrop. It was convenient and a bit cheaper to buy from them but had I known I could get different colors from Signature Plastics I would have done that.

I also bought eight “deep-dish” DSA keys direct from Signature Plastics. These keys feel different which lets me know when my fingers are above the home row. I’d recommend doing this. You can order from this page.

For key switches I bought Cherry MX Blues through Massdrop. Blues are extremely clicky. You can easily hear me typing in every room of my apartment. It is very satisfying.

After using the keyboard for about a week I also ended up buying some pads for my wrists. I occasionally rest my wrists on the keyboard and the keyboard’s edge would dig into me.

Building

I followed Massdrop’s step-by-step guide and this YouTube video. Another great resource is the community at GeekHack. I’d recommend reading and watching as much as possible before starting your build.

I built this using a cheap soldering iron I’ve had for years, very thin solder, solder wick, and a multimeter. I don’t know if this would have been easier with better tools or not but those got the job done.

While soldering the surface mount diodes I was in the zone and soldered a few locations that didn’t actually need to be soldered. When you are soldering the diodes you should only be soldering them to the locations that have the key silk screen.

My system for minimizing errors while soldering the diodes is the following five steps.

  1. Lay down some solder on one of the pads.
  2. Put the correct end of the diode on top of that solder, reheat and push down.
  3. Test the connection with a multimeter.
  4. Solder the other half of the diode.
  5. Test the connection.

I batched up the steps. I’d do a whole row of the first step, then move to the second for the entire row, then do the third, etc. Being rigorous about testing every connection is important. Catching mistakes early makes it easier to fix the mistakes.

If you solder a diode on the wrong way there is a huge difference (at least for me using solder wick) between the difficulty of fixing the error when only one pad has been soldered versus two pads. I soldered more than one diode backwards and a mistake noticed after soldering only one pad was easy to fix. After soldering both pads it took serious effort.

Eventually you’ll need to cut open a USB cable. I ended up removing the plastic housing using a Dremel. When soldering the wires to the USB holes I was too concerned with it looking pretty and did not leave plenty of wire. This made it harder to solder and as a result I ended up doing a poor job that resulted in a short. After desoldering and destroying another cable, but leaving more wire, I managed to do a better job. I originally noticed the short because I kept getting warnings from my computer about my USB Keyboard drawing too much power.

I’ve annotated a copy of Massdrop’s instructions using Evernote. It contains the above tips inline.

Firmware

After you physically build your keyboard you need to build the firmware. There are a few different firmwares that can work and you can discover those on GeekHack. I’m using a fork of what Massdrop’s graphical configuration tool uses. It is based off benblazak/ergodox-firmware.

One of the exciting things about the ErgoDox is tweaking the firmware. I took the base firmware and modified it to have media key support and light up the LEDs when I’m on any layer besides the base. Some people have added the ability to record keyboard macros and other neat features. I encourage you to take a look at the source even if you use the graphical configuration tool. I haven’t explored beyond benblazak/ergodox-firmware so I can’t compare it to other firmwares.

Conclusion

I really enjoy it. Building it was both fun and frustrating 3.

After using the keyboard for a few months I’ve found that I really only use three (on each hand) of the thumb cluster keys. I also don’t use the keyboard layers too often. I have three layers programmed and I always stay on the main one unless I want to hit a media key.

Would I recommend building your own ErgoDox? If you already can or are willing to learn to solder and this sounds at all interesting to you I would recommend it. The project can be frustrating but the result is great.

The Future

There is still a lot left to explore in the custom keyboard space. Even so I have no plans on leaving the ErgoDox anytime soon. In terms of improving my ErgoDox, I plan on poking around the different firmwares at some point. I’d also like to explore tenting options.

Resources


  1. I feel a bit odd using the word standard to describe acquiring parts to build a keyboard.

  2. This page has diagrams that shows the different keycap families.

  3. Those surface mount diodes are so tiny.

Using Emacs to Explore an HTTP API

| Comments

Recently I rediscovered an Emacs package that allows you to interact with HTTP endpoints from the comfort of an Emacs buffer. restclient.el provides restclient-mode. This mode allows you to write and execute HTTP requests in an Emacs buffer. This package can be found in MELPA.

Below is an example buffer that touches the GitHub API.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
:github = https://api.github.com

# get users orgs

GET :github/users/jakemcc/orgs

# rendor markdown

POST :github/markdown

{
  "text" : "## Title"
}

# rendor markdown raw

POST :github/markdown/raw
Content-Type: text/plain

Title
-----

The example above has a few interesting snippets. :github is an example of a variable. Lines 8-14 show an example of posting json to an endpoint. You put the data you want to send below the query. The last POST shows how to set headers for a request.

The location of your cursor decides what query to execute. Comments start with # and break your document into sections. The query in the same section as your cursor is the one that is executed. If the cursor is anywhere on lines 3-6 and I hit C-c C-c then Emacs queries GitHub for my organizations. Below is what pops up in a buffer.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
[
    {
        "avatar_url": "https:\/\/avatars.githubusercontent.com\/u\/1826953?",
        "public_members_url": "https:\/\/api.github.com\/orgs\/speakerconf\/public_members{\/member}",
        "members_url": "https:\/\/api.github.com\/orgs\/speakerconf\/members{\/member}",
        "events_url": "https:\/\/api.github.com\/orgs\/speakerconf\/events",
        "repos_url": "https:\/\/api.github.com\/orgs\/speakerconf\/repos",
        "url": "https:\/\/api.github.com\/orgs\/speakerconf",
        "id": 1826953,
        "login": "speakerconf"
    },
    {
        "avatar_url": "https:\/\/avatars.githubusercontent.com\/u\/4711436?",
        "public_members_url": "https:\/\/api.github.com\/orgs\/outpace\/public_members{\/member}",
        "members_url": "https:\/\/api.github.com\/orgs\/outpace\/members{\/member}",
        "events_url": "https:\/\/api.github.com\/orgs\/outpace\/events",
        "repos_url": "https:\/\/api.github.com\/orgs\/outpace\/repos",
        "url": "https:\/\/api.github.com\/orgs\/outpace",
        "id": 4711436,
        "login": "outpace"
    }
]
// HTTP/1.1 200 OK
// Server: GitHub.com
// Date: Fri, 04 Jul 2014 17:34:26 GMT
// Content-Type: application/json; charset=utf-8
// other headers removed for space consideration on blog

C-c C-c triggers restclient-http-send-current which runs a query and pretty prints the result. I could have used C-c C-r to trigger restclient-http-send-current-raw which executes a query and shows the raw result.

It isn’t a perfect mode. One issue I’ve come across is that queries targeting localhost fail. The solution is to query 127.0.0.1.

restclient-mode makes Emacs a useful tool for exploring and testing HTTP APIs. Since it operates on a simple text format it allows you to easily share executable documentation with others. I highly recommend restclient.el.

Comparing Clojure Testing Libraries: Output

| Comments

I recently became interested in how Clojure testing libraries help you when there is a test failure. This interest resulted in me exploring different Clojure testing libraries. I created the same tests using clojure.test (with and without humane-test-output), expectations, Midje, and Speclj and looked at the output.

I ran all of these examples using Leiningen. Midje, Speclj, and expectations color their output but I’m not going to try to reproduce that here. The color added by Midje and expectations is useful. Speclj color hurt its readability. I use a dark colored terminal and Speclj colors the line that tells where the failure occurs black. This made it hard to read.

I’m not going to show what the tests look like for each testing library past the first comparison. How a test in expressed is important but not what I want to focus on in this post.

Comparing Strings

Going to start off with a basic string comparison. The failing test compares two strings that only differ by one character.

clojure.test

Most (hopefully all) Clojure programmers should be familiar with clojure.test. It is the testing library that is included with Clojure.

1
2
3
4
5
6
(ns example.string-test
  (:require [clojure.test :refer :all]))

(deftest string-comparisons
  (is (= "strings equal" "strings equal"))
  (is (= "space" "spice")))

The output below is what you get when the above test runs. Even in this simple example it isn’t the easiest to read. It doesn’t make it easy to find the expected or actual values.

clojure.test output
1
2
3
FAIL in (string-comparisons) (string_test.clj:6)
expected: (= "space" "spice")
  actual: (not (= "space" "spice"))

Below is the same test but with humane-test-output enabled. It is easy to read the output and see the expected and actual value. It even provides a diff between them although in this situation it isn’t that useful.

clojure.test with humane-test-output
1
2
3
4
5
FAIL in (string-comparisons) (string_test.clj:6)
expected: "space"
  actual: "spice"
    diff: - "space"
          + "spice"
expectations

Another testing library is Jay Field’s expectations. You can see from the example that it has a fairly minimal syntax.

1
2
3
4
5
(ns example.string-expectations
  (:require [expectations :refer :all]))

(expect "strings equal" "strings equal")
(expect "space" "spice")
expectations output
1
2
3
4
5
6
7
8
9
failure in (string_expectations.clj:5) : example.string-expectations
(expect "space" "spice")

           expected: "space"
                was: "spice"

           matches: "sp"
           diverges: "ace"
                  &: "ice"

The output from expectations is very readable. You can easily pick out the expected and actual values. It also shows you where the string starts to diverge.

Speclj

Before writing this post I had zero experience with Micah Martin’s Speclj. Below is my translation of the failing string test and its output.

1
2
3
4
5
6
(ns example.string-spec
  (:require [speclj.core :refer :all]))

(describe "String comparisons"
  (it "have nice error message"
      (should= "space" "spice")))
Speclj
1
2
3
4
  9) String comparisons have nice error message
     Expected: "space"
          got: "spice" (using =)
     /Users/jake/src/jakemcc/example/spec/example/string_spec.clj:7

Speclj’s test output above is an improvement over clojure.test. You can easily find the expected and actual values. It doesn’t provide any help with diagnosing how those values are different.

Midje

I have a little bit of experience with Brian Marick’s Midje. Unlike the other libraries it switches up the assertion syntax. In Midje the expected value is on the right side of =>.

1
2
3
4
5
6
7
8
(ns example.string-test
  (:require [midje.sweet :refer :all]))

(fact "strings are equal"
  "string is equal" => "string is equal")

(fact "strings not equal"
   "spice" => "space")
Midje
1
2
3
FAIL "strings not equal" at (string_test.clj:8)
    Expected: "space"
      Actual: "spice"

Midje’s output is similar to Speclj’s. You can quickly find the expected and actual values but it doesn’t help you spot the difference.

String Comparison Winner

expectations wins for best output. You can easily spot the expected and actual values and it also helps you find the difference between the strings.

The worst output comes from clojure.test. It doesn’t make it easy to spot the difference or even find the expected and actual values.

Comparing Maps

For maps I’ve setup three assertions. The first has an extra key-value pair in the actual value. The second has an extra in the expected value. The final assertion has a different value for the :cheese key. The clojure.test example is below.

1
2
3
4
(deftest map-comparisons
  (is (= {:sheep 1} {:cheese 1 :sheep 1}))
  (is (= {:sheep 1 :cheese 1} {:sheep 1}))
  (is (= {:sheep 1 :cheese 1} {:sheep 1 :cheese 5})))
clojure.test
1
2
3
4
5
6
7
8
9
10
11
FAIL in (map-comparisons) (map_test.clj:5)
expected: (= {:sheep 1} {:cheese 1, :sheep 1})
  actual: (not (= {:sheep 1} {:cheese 1, :sheep 1}))

FAIL in (map-comparisons) (map_test.clj:6)
expected: (= {:sheep 1, :cheese 1} {:sheep 1})
  actual: (not (= {:cheese 1, :sheep 1} {:sheep 1}))

FAIL in (map-comparisons) (map_test.clj:7)
expected: (= {:sheep 1, :cheese 1} {:sheep 1, :cheese 5})
  actual: (not (= {:cheese 1, :sheep 1} {:cheese 5, :sheep 1}))

Unsurprisingly the default clojure.test output for maps suffers from the same problems found in the string comparisons. To find the actual and expected values you need to manually parse the output.

clojure.test with humane-test-output
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
FAIL in (map-comparisons) (map_test.clj:5)
expected: {:sheep 1}
  actual: {:cheese 1, :sheep 1}
    diff: + {:cheese 1}

FAIL in (map-comparisons) (map_test.clj:6)
expected: {:cheese 1, :sheep 1}
  actual: {:sheep 1}
    diff: - {:cheese 1}

FAIL in (map-comparisons) (map_test.clj:7)
expected: {:cheese 1, :sheep 1}
  actual: {:cheese 5, :sheep 1}
    diff: - {:cheese 1}
          + {:cheese 5}

Above is the output of using clojure.test with humane-test-output. It is a big improvement over the default clojure.test. You can quickly see the expected and actual values. Unlike with the string assertions the diff view is actually helpful. The diffs do a good job of helping you identify the error.

expectations
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
failure in (map_expectations.clj:6) : example.map-expectations
(expect {:sheep 1} {:sheep 1, :cheese 1})

           expected: {:sheep 1}
                was: {:cheese 1, :sheep 1}

           in expected, not actual: null
           in actual, not expected: {:cheese 1}

failure in (map_expectations.clj:7) : example.map-expectations
(expect {:sheep 1, :cheese 1} {:sheep 1})

           expected: {:cheese 1, :sheep 1}
                was: {:sheep 1}

           in expected, not actual: {:cheese 1}
           in actual, not expected: null

failure in (map_expectations.clj:8) : example.map-expectations
(expect {:sheep 1, :cheese 5} {:sheep 1, :cheese 1})

           expected: {:cheese 5, :sheep 1}
                was: {:cheese 1, :sheep 1}

           in expected, not actual: {:cheese 5}
           in actual, not expected: {:cheese 1}

expectations does a pretty good job helping you as well. As before, you can clearly read the expected and actual values. expectations also provides some hint as to what is different between the maps. I find the English descriptions a bit easier to read than humane-test-output’s diff view. Still seeing lines like line 7 (in expected, not actual: null) is a bit confusing and the output would be improved if it was suppressed.

I’m just going to lump Speclj and Midje together. The output for each is below. They both improve over clojure.test by making it easy to see the expected and actual value. They both don’t do anything beyond that.

Speclj
1
2
3
4
5
6
7
8
9
10
11
12
13
14
  4) map comparisons have nice error messages when extra entries keys present
     Expected: {:sheep 1}
          got: {:cheese 1, :sheep 1} (using =)
     /Users/jake/src/jakemcc/example/spec/example/map_spec.clj:7

  5) map comparisons have nice error messages when missing an entry
     Expected: {:cheese 1, :sheep 1}
          got: {:sheep 1} (using =)
     /Users/jake/src/jakemcc/example/spec/example/map_spec.clj:9

  6) map comparisons have nice error messages when mismatched values
     Expected: {:cheese 5, :sheep 1}
          got: {:cheese 1, :sheep 1} (using =)
     /Users/jake/src/jakemcc/example/spec/example/map_spec.clj:11
Midje
1
2
3
4
5
6
7
8
9
10
11
FAIL "map is missing an entry" at (map_test.clj:5)
    Expected: {:cheese 1, :sheep 1}
      Actual: {:sheep 1}

FAIL "map has an extra entry" at (map_test.clj:8)
    Expected: {:sheep 1}
      Actual: {:cheese 1, :sheep 1}

FAIL "map has a different value" at (map_test.clj:11)
    Expected: {:cheese 5, :sheep 1}
      Actual: {:cheese 1, :sheep 1}

Map Comparison Winner

Tie between humane-test-output and expectations. Both do a good job of helping the reader spot the difference.

Comparing Sets

Next up are sets. Only two assertions for this section. One with the actual value having an extra member and one test where it is missing a member.

1
2
3
4
5
6
(ns example.set-test
  (:require [clojure.test :refer :all]))

(deftest set-comparisons
  (is (= #{:a :b} #{:a :b :c}))
  (is (= #{:a :b :c} #{:a :b})))

First up is the basic clojure.test output. It suffers from the same problem it has suffered this entire time. It doesn’t make it easy to read the expected and actual values.

clojure.test
1
2
3
4
5
6
7
FAIL in (set-comparisons) (set_test.clj:5)
expected: (= #{:b :a} #{:c :b :a})
  actual: (not (= #{:b :a} #{:c :b :a}))

FAIL in (set-comparisons) (set_test.clj:6)
expected: (= #{:c :b :a} #{:b :a})
  actual: (not (= #{:c :b :a} #{:b :a}))

No surprises with humane-test-output. It improves the clojure.test output by making it easy to read the expected and actual values. The diff view also helps figure out what is causing the assertion to fail.

clojure.test with humane-test-output
1
2
3
4
5
6
7
8
9
FAIL in (set-comparisons) (set_test.clj:5)
expected: #{:b :a}
  actual: #{:c :b :a}
    diff: + #{:c}

FAIL in (set-comparisons) (set_test.clj:6)
expected: #{:c :b :a}
  actual: #{:b :a}
    diff: - #{:c}

expectations once again delivers nice output. It continues to be easy to find the expected and actual values and helps you spot the differences with a diff view.

expectations
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
failure in (set_expectations.clj:4) : example.set-expectations
(expect #{:b :a} #{:c :b :a})

           expected: #{:b :a}
                was: #{:c :b :a}

           in expected, not actual: null
           in actual, not expected: #{:c}

failure in (set_expectations.clj:5) : example.set-expectations
(expect #{:c :b :a} #{:b :a})

           expected: #{:c :b :a}
                was: #{:b :a}

           in expected, not actual: #{:c}
           in actual, not expected: null

Speclj and Midje both have better output than the basic clojure.test.

Speclj
1
2
3
4
5
6
7
8
9
  7) set comparisons have nice error messages when missing item
     Expected: #{:b :a}
          got: #{:c :b :a} (using =)
     /Users/jake/src/jakemcc/example/spec/example/set_spec.clj:9

  8) set comparisons have nice error messages when more items
     Expected: #{:c :b :a}
          got: #{:b :a} (using =)
     /Users/jake/src/jakemcc/example/spec/example/set_spec.clj:11
Midje
1
2
3
4
5
6
7
FAIL "set is superset of expected" at (set_test.clj:5)
    Expected: #{:a :b}
      Actual: #{:a :b :c}

FAIL "set is subset of expected" at (set_test.clj:8)
    Expected: #{:a :b :c}
      Actual: #{:a :b}

Set Comparison Winner

Similar to the winner of the map comparisons I’m going to split the victory between expectations and humane-test-output.

Comparing Lists

Next up we compare lists (and lists to vectors). There are three comparisons; one with an extra element, one with same length but a mismatched element, and one comparing a vector and list with drastically different contents.

1
2
3
4
5
6
7
(ns example.seq-test
  (:require [clojure.test :refer :all]))

(deftest list-comparisons
  (is (= '(1 2 3) '(1 2 3 4)))
  (is (= '(1 2 4) '(1 2 3)))
  (is (= '(9 8 7) [1 2 3])))

First up clojure.test. Same issues as with all the previous comparisons.

clojure.test
1
2
3
4
5
6
7
8
9
10
11
FAIL in (list-comparisons) (seq_test.clj:5)
expected: (= (quote (1 2 3)) (quote (1 2 3 4)))
  actual: (not (= (1 2 3) (1 2 3 4)))

FAIL in (list-comparisons) (seq_test.clj:6)
expected: (= (quote (1 2 4)) (quote (1 2 3)))
  actual: (not (= (1 2 4) (1 2 3)))

FAIL in (list-comparisons) (seq_test.clj:7)
expected: (= (quote (9 8 7)) [1 2 3])
  actual: (not (= (9 8 7) [1 2 3]))

Once again humane-test-output improves upon clojure.test. Only interesting difference from previous comparisons is that the diff view ends up having nil values in it where the elements are the same.

clojure.test with humane-test-output
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
FAIL in (list-comparisons) (seq_test.clj:5)
expected: (1 2 3)
  actual: (1 2 3 4)
    diff: + [nil nil nil 4]

FAIL in (list-comparisons) (seq_test.clj:6)
expected: (1 2 4)
  actual: (1 2 3)
    diff: - [nil nil 4]
          + [nil nil 3]

FAIL in (list-comparisons) (seq_test.clj:7)
expected: (9 8 7)
  actual: [1 2 3]
    diff: - [9 8 7]
          + [1 2 3]

expectations continues to have good output. It tries to help you out as well. You’ll notice that it also has nil values inserted where the lists are the same.

expectations
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
failure in (list_expectations.clj:4) : example.list-expectations
(expect '(1 2 3) '(1 2 3 4))

           expected: (1 2 3)
                was: (1 2 3 4)

           in expected, not actual: null
           in actual, not expected: [nil nil nil 4]
           actual is larger than expected

failure in (list_expectations.clj:5) : example.list-expectations
(expect '(1 2 4) '(1 2 3))

           expected: (1 2 4)
                was: (1 2 3)

           in expected, not actual: [nil nil 4]
           in actual, not expected: [nil nil 3]

failure in (list_expectations.clj:6) : example.list-expectations
(expect '(9 8 7) [1 2 3])

           expected: (9 8 7)
                was: [1 2 3]

           in expected, not actual: [9 8 7]
           in actual, not expected: [1 2 3]

Unsurprisingly, Speclj and Midje are better than clojure.test but again don’t go beyond making easy to find the expected and actual values.

Speclj
1
2
3
4
5
6
7
8
9
10
11
12
13
14
  1) List/vector comparisons when there is an extra element
     Expected: (1 2 3)
          got: (1 2 3 4) (using =)
     /Users/jake/src/jakemcc/example/spec/example/string_spec.clj:7

  2) List/vector comparisons when there is a mismatched element
     Expected: (1 2 4)
          got: (1 2 3) (using =)
     /Users/jake/src/jakemcc/example/spec/example/string_spec.clj:9

  3) List/vector comparisons when comparing different types
     Expected: (9 8 7)
          got: [1 2 3] (using =)
     /Users/jake/src/jakemcc/example/spec/example/string_spec.clj:11
Midje
1
2
3
4
5
6
7
8
9
10
11
FAIL "lists are different sizes" at (seq_test.clj:5)
    Expected: (1 2 3)
      Actual: (1 2 3 4)

FAIL "lists have different entries" at (seq_test.clj:8)
    Expected: (1 2 4)
      Actual: (1 2 3)

FAIL "compare very different list like values" at (seq_test.clj:14)
    Expected: (9 8 7)
      Actual: [1 2 3]

List Comparison Winner

I find the clojure.test with humane-test-output to be a bit easier to read than expectations. Both have better output than the basic clojure.test, Speclj, and Midje.

Overall Winner

If I were picking a testing library based entirely on what a failing test looks like I would use expectations. My second pick would be clojure.test with humane-test-output.

It is great that Clojure ships with clojure.test. It is unfortunate that it does so little to help you read a failing test. Every library I tried has better output than clojure.test.

Addendum

Added 2014/06/23

Colin Jones points out that Speclj provides should==. should== checks that the expected and actual value have the same contents. He provided a gist that shows the difference.