I’m now writing Clojure nearly 100% of my time and as a result am spending more time in Emacs. I’m working in a few different projects and wanted a quicker way to jump between them. My first attempt at this ended with me defining many functions that looked like the following.

1
2
3

(defun b/test-refresh ()
  (interactive)
  (find-file "~/src/jakemcc/lein-test-refresh/test-refresh/project.clj"))

After writing a couple of these I decided the computer could do this better than I could and decided to write some code to automate it. A sample of my directory structure is shown below.

1
2
3
4
5

jakemcc/
├── bookrobot
│   └── project.clj
└── lein-autoexpect
    └── project.clj

Taking advantage of this structure I wrote some Emacs lisp to walk a directory and define functions that open up any found project.clj files.

1
2
3
4
5
6
7
8
9
10
11
12

;; -*- lexical-binding: t -*-

(defun open-file-fn (file)
  (lambda ()
    (interactive)
    (find-file file)))

(defun create-project-shortcuts (prefix base)
  (dolist (elt (directory-files base))
    (let ((project (concat base "/" elt "/project.clj")))
      (when (file-exists-p project)
        (fset (intern (concat prefix elt)) (open-file-fn project))))))

open-file-fn creates an anonymous interactive function (meaning the function can be called interactively) that opens file. It takes advantage of the feature in Emacs 24 that enables lexical scoping by adding ;; -*- lexical-binding: t -*- to the top of your Emacs lisp file. This lets the anonymous function capture file.

create-project-shortcuts takes in a prefix and a base directory. It searches base for directories that contain a project.clj file. For each found project.clj file a function is created (using fset) with the name of the containing directory prefixed by prefix.

With those two functions defined all that is left is to call create-project-shortcuts.

1

(create-project-shortcuts "b/" "~/src/jakemcc")

Now b/bookrobot and b/lein-autoexpect are available after hitting M-x.

I’ve used this code to create quick shortcuts to all of my work and non-work projects. It has been immensely useful for jumping around projects.

Last year I wrote about how I managed my windows under OS X, Windows, and Linux. I’m a big fan of having an orderly layout and try to use grid managers. Since then I’ve changed jobs and now my main machine is an MacBook Pro running OS X Mavericks with two 27 inch cinema displays. As a result I’ve started experimenting with more OS X window managers. After trying a few out I’m going to stick with Phoenix.

Before Phoenix

Last year I was satisfied using Spectacle. It is (or at least was, I haven’t used it in a while) easy to install and had good defaults. I’d still recommend it for most people.

At the recommendation from a reader, I switched to Slate. Slate has a ton of features and I barely scratched the surface in how I used it. I used it as a replacement for Spectacle and didn’t touch any of the advanced features. Before I had the urge to explore the advanced features I ended up becoming dissatisfied with Slate. I ran into an issue where after running for a while (talking at least a week) it would start to respond slowly. I’d try to move a window to another monitor and it wouldn’t move. Eventually I’d be in another process and the command would register sending whatever window I was currently focused on to another monitor.

Introducing Phoenix

I was looking for solutions to Slate’s unresponsiveness when I stumbled on Phoenix. I was drawn in by its stated goal; it “aims for efficiency and a very small footprint.” The fact that it is still being actively developed was also a huge selling point. Knowing that any bugs I find have a potential to be fixed is great.

Phoenix provides a JavaScript API that allows you to interact with your running applications or launch applications. It doesn’t provide anything out of the box; it is up to you to make it useful by writing your own (or taking another persons) configuration.

This is a double-edged sword. This means you get exactly the features you want. It also means you might spend significant amounts of time figuring out how to get the features you want.

Luckily there are examples that you can use as a starting point. Browsing through the examples is a great way of becoming familiar with what is possible and can be inspiring.

My configuration is relatively minimal. I’ve written code to move windows between monitors (rotating between three added some complexity to this), start or focus certain applications, and resize windows. This is enough for me to feel efficient.

I encourage you to use a tool to help manage your windows. Personally I think Phoenix is pretty great and don’t mind tinkering with my configuration and strongly recommend it. As a bonus it is a young project where the maintainer is open to suggestions. If you have an idea for a useful feature it has a possibility of being added pretty quickly.

lein-test-refresh has always supported notifying you of your tests' status through growl. With the release of version 0.3.4 it now will notify you using whatever program you want.

To make my Mac whisper my the results of running my tests I can use the following project.clj

1
2
3
4

(defproject sample "1.2.3"
  :dependencies [[org.clojure/clojure "1.5.1"]]
  :profiles {:dev {:plugins [[com.jakemccrary/lein-test-refresh "0.3.4"]]}}
  :test-refresh {:notify-command ["say" "-v" "Whisper"]})

The specification of the command is found in the :test-refresh {:notify-command ["say" "-v" "Whisper"]} entry in the above project.clj. After running your tests lein-test-refresh will pass a (usually) short summary message as the final parameter to the specified command.

Now you can finally have the results of running your tests whispered to you.

At the suggestion of my coworker Jeff Bay you can now hit a single keystroke to cause lein-test-refresh to rerun your tests. Now you can hit enter in the terminal running lein test-refresh to cause your tests to be run.

Add the below text to your project.clj to start using lein-test-refresh today.

As a reminder if you call you can pass the argument :growl to lein-test-refresh. If you pass :growl as an argument then you’ll be notified of test success and failures through growl. On top of the quick feedback cycles that lein-test-refresh (and lein-autoexpect) provides the growl notification is my favorite feature. I’d highly recommend giving it a shot.

I use goodreads to keep track of my reading and have since early 2010. I find very useful for capturing what I want to read and reminding me of how I felt about books I’ve read. I thought it would be fun to take a closer look at what I read in 2013. I’m doing this using Clojure with Incanter. I haven’t used Incanter since I wrote this post and thought this would be a good opportunity to visit it again.

First I need to get my data out of goodreads. I’ve worked with the Goodreads API before ¹ but am not going to use it for this exercise. Instead I’m using the goodreads export functionality (at goodreads follow the links: My Books > import/export) to export a csv file. Having the csv file also lets me cleanup some of the data since some of the book’s page counts were missing ².

Now that I have data it is time to start playing with it. Run lein new goodreads-summary and edit the project.clj file to have a dependency on Incanter.

1
2
3
4
5

(defproject goodreads-summary "0.1.0-SNAPSHOT"
  :dependencies [[org.clojure/clojure "1.5.1"]
                 [org.clojure/data.csv "0.1.2"]
                 [incanter "1.5.4"]
                 [clj-time "0.6.0"]])

Next I’m going to take the csv file and transform it into an Incanter dataset. This is easily done with incanter.io/read-dataset. It isn’t well documented but by passing :keyword-headers false to read-dataset the headers from the csv are not converted to keywords. I’m doing this because some of the goodreads headers contain spaces and dealing with spaces in keywords is a pain. The snippet below has all of the necessary requires for the remainder of the examples.

1
2
3
4
5
6
7
8
9
10
11

(ns goodreads-summary.core
  (:require [clojure.data.csv :as csv]
            [clojure.string :as str]
            [incanter.core :as incanter]
            [incanter.io :as io]
            [incanter.charts :as charts]
            [clj-time.core :as tc]
            [clj-time.format :as tf]))

(defn read-csv [filepath]
  (io/read-dataset filepath :header true :keyword-headers false))

Calling read-csv with the path to the exported goodreads data results in dataset. If you want to view the data use incanter.core/view. Running (incanter/view (read-csv "goodreads_export.csv")) pops up a grid of with all the data. I don’t care about most of the columns so lets define a function that selects out the few I care about.

1
2

(defn select-columns [dataset]
  (incanter/sel dataset :cols ["Number of Pages" "Date Read" "Bookshelves" "Exclusive Shelf"]))

Selecting columns is done with incanter.core/sel. Like most Incanter functions it has many overloads. One way to use it is to pass a dataset with a vector of columns you want to select.

Filtering a dataset is done using incanter.core/$where. Goodreads has three default shelves to-read, currently-reading, and read. To select all your read books you filter of the Exclusive Shelf column for read books.

1
2

(defn finished [dataset]
  (incanter/$where {"Exclusive Shelf" "read"} dataset))

Filtering for books read in 2013 is a bit more complicated. First I convert the Date Read column from a string to a org.joda.time.DateTime. This is done with the combination of transform-date-read-column and parse-date. Some of the my data is missing a Date Read value. I’m choosing to handle this by treating missing data as the result of (clj-time.core/date-time 0).

The $where in books-read-in-2013 is a bit more complicated than the filtering in finished. Here I’m providing a predicate to use instead of just doing an equality comparison.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18

(defn parse-date [date-str]
  (if date-str
    (tf/parse (tf/formatter "yyyy/MM/dd") date-str)
    (tc/date-time 0)))

(defn transform-date-read-column [dataset]
  (incanter/transform-col dataset "Date Read" parse-date))

(defn date-greater-than-pred [date]
  (fn [challenger]
    (> (.compareTo challenger date) 0)))

(defn books-read-in-2013 [dataset]
  (let [finished (finished (select-columns dataset))
        with-dates (incanter/$where {"Date Read" {:fn identity}} finished)
        with-date-objects (transform-date-read-column with-dates)]
    (incanter/$where {"Date Read" {:fn (date-greater-than-pred (parse-date "2012/12/31"))}}
                     with-date-objects)))

Now we have a dataset that that contains only books read in 2013 (well, until I read a book in 2014 and the filter above also grabs books in 2014). Now to generate some analytic for each month. First lets add a Month column to our data. Originally I wrote the function below. It uses incanter.core/$map to generate the data, makes a dataset with the new data, and then adds that to the original dataset.

1
2
3
4
5

(defn add-month-read-column [dataset]
  (let [month-read (incanter/$map tc/month "Date Read" dataset)
        month-dataset (incanter/dataset ["Month"] month-read)
        with-month-read (incanter/conj-cols dataset month-dataset)]
    with-month-read))

When I wrote the above code it seemed like there should be a better way. While writing this post I stumbled across incanter.core/add-derived-column. Switching to add-derived-column makes add-month-read-column almost trivial.

1
2

(defn add-month-read-column [dataset]
  (incanter/add-derived-column "Month" ["Date Read"] tc/month dataset))

Now that we have add-month-read-column we can now start aggregating some stats. Lets write code for calculating the pages read per month.

1
2
3
4

(defn pages-by-month [dataset]
  (let [with-month-read (add-month-read-column dataset)]
    (->> (incanter/$rollup :sum "Number of Pages" "Month" with-month-read)
         (incanter/$order "Month" :asc))))

That was pretty easy. Lets write a function to count the number of books read per month.

1
2
3
4

(defn book-count-by-month [dataset]
  (let [with-month-read (add-month-read-column dataset)]
    (->> (incanter/$rollup :count "Number of books" "Month" with-month-read)
         (incanter/$order "Month" :asc))))

pages-by-month and book-count-by-month are very similar. Each uses incanter.core/$rollup to calculate per month stats. The first argument to $rollup can be a function that takes a sequence of values or one of the supported magical “function identifier keywords”.

Next lets combine the data together so we can print out a nice table. While we are at it lets add another column.

1
2
3
4
5
6
7
8
9

(defn stats-by-month [dataset]
  (->> (incanter/$join ["Month" "Month"]
                     (pages-by-month dataset)
                     (book-count-by-month dataset))
       (incanter/rename-cols {"Number of Pages" "Page Count"
                              "Number of books" "Book Count"})
       (incanter/add-derived-column "Pages/Books"
                                  ["Page Count" "Book Count"]
                                  (fn [p b] (Math/round (double (/ p b)))))))

stats-by-month returns a dataset which when printed looks like the following table. It joins the data, renames columns, and adds a derived column.

| Month | Book Count | Page Count | Pages/Books |
|-------+------------+------------+-------------|
|     1 |          6 |       1279 |         213 |
|     2 |          2 |       1251 |         626 |
|     3 |          8 |       2449 |         306 |
|     4 |          5 |       1667 |         333 |
|     5 |          6 |       2447 |         408 |
|     6 |          5 |       1609 |         322 |
|     7 |          5 |       1445 |         289 |
|     8 |          5 |       2229 |         446 |
|     9 |          2 |        963 |         482 |
|    10 |          5 |       1202 |         240 |
|    11 |          5 |       2248 |         450 |
|    12 |          7 |       1716 |         245 |

Great. Now we have a little ascii table. Lets get graphical and make some bar charts.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18

(defn chart-column-by-month [column dataset]
  (let [select (fn [column] (incanter/sel dataset :cols column))
        months (select "Month")]
    (charts/bar-chart months (select column)
                      :y-label column :x-label "Month")))

(defn chart-page-count-by-month [dataset]
  (chart-column-by-month "Page Count" dataset))

(defn chart-book-count-by-month [dataset]
  (chart-column-by-month "Book Count" dataset))

(defn view-page-count-chart []
  (-> (read-csv "goodreads_export.csv")
      books-read-in-2013
      stats-by-month
      chart-page-count-by-month
      incanter/view))

Running the snippet view-page-count-chart produces a pop-up with the below bar chart. The chart actually surprises me as I fully expected to have higher page counts during the winter months than the summer months. This chart and analysis is pretty useless though without knowing the difficulty of the pages read. For example, last February I read Infinite Jest. Knowing that I don’t feel like having a low page count in that month is slacking at all.

2013 Summary

2013 was a pretty big year of reading. I read more books this past year than all other years that I have data. I also read some of the best books I’ve ever read. Not only that but I actually created multiple ³ custom Kindle dictionaries to help improve my (and others) reading experience.

Summary table ⁴:

Plans for 2014

I’m planning on reading a similar amount in this upcoming year but will probably have a bit more non-fiction books. First step towards doing that is to start classifying my books as non-fiction or fiction. I’m also planning on rereading at least two books that I’ve read in the last few years. This is unusual for me because I don’t often reread books that quickly.

If you have any book recommendations feel free to leave them in the comments or contact me through twitter or email.

I was recently approached by Packt Publishing asking if I’d review Shantanu Kuma’s book Clojure High Performance Programming . It sounded interesting so I took them up on their offer for a free copy and read it over two flights.

Unsurprisingly the table of contents does a good job describing the book. This book doesn’t dive too deep into any one topic but instead gives you a taste of each.

Overall the book was pretty good. It provides interesting examples of real world Clojure code that solve specific performance problems. It talks about host performance, both JVM and hardware, concerns which are both areas that shouldn’t be overlooked. I thought the book was best when showing examples of well performing code from libraries.

I’d recommend this book for developers who aren’t past the beginning stages of writing performant code. It does a good job introducing the topics you’ll want to think about when trying to craft well performing programs.

It isn’t for the developer who has spent years optimizing code for performance. Those developers are already going to be familiar with the language and concerns of writing high performance code.

If I could add anything to the book it would be a chapter about measuring performance in production. If you are writing high performance programs it has been my experience that you must measure in production. This is easiest to do if you build measuring in from the very beginning.

I recently switched companies and find myself working on a project that uses clojure.test. I haven’t worked with clojure.test since I started using expectations with lein-autoexpect. This combination spoiled me when it comes to testing Clojure code. I can no longer stand running tests by hand; I’m too used to having a tool run them for me. As a result I tried out some clojure.test continuous testing tools.

I wasn’t satisfied with what I found. Since I wrote lein-autoexpect, a continous tester for expectations, it was easy for me to fork it and and create lein-test-refresh. lein-test-refresh solves the issues I ran into with the other clojure.test tools.

To use lein-test-refresh follow these steps (latest version found in image at end):

1. Add [com.jakemccrary/lein-test-refresh "0.1.2"] to the :plugins section in your project.clj or ~/.lein/profiles.clj file.
2. Run lein test-refresh or lein test-refresh :growl.
3. Enjoy your minimal feedback delays between editing your Clojure code and seeing if your tests pass.

lein-test-refresh watches the source and test directories specified in your project.clj and reloads code when files changes. After reloading your code your clojure.test tests are run and the output is printed to your console. When you pass :growl as a command line argument the plugin will use growl to notify you of success and failures. This is one of my favorite features about lein-test-refresh as it allows me to continuously run my tests without taking up space on my monitors.

I hope you enjoy lein-test-refresh. It has made using clojure.test much more enjoyable.

Latest version from Clojars:

My text editor of choice is Emacs. Its extensibility is a major contributor to this preference. The ease of adding additional functionality means you can customize it to your liking. You should not go overboard and change too much of the default behavior but you should feel free to add additional features.

I recently found myself often editing a file in emacs and then switching to a terminal and running a bash script to see how the output changed. This is part of my work flow for shutting down or starting new server processes. Since this is something I’ll be doing quite frequently in the future, I wrote some Emacs Lisp to run the shell script and display the output in a temporary buffer. With this function in place I no longer have to toggle to a terminal and run a command.

I’m picky and I wanted this output buffer to have the same behavior as the help buffer. That is, I wanted to be able to close the buffer by just hitting the letter q. It took me a while to figure out how to do this so I thought I would share it here in hopes it might benefit others.

First I’ll show the code and then I’ll explain what it is doing.

1
2
3
4
5
6
7

(defun blog-example ()
  (interactive)
  (with-output-to-temp-buffer "*blog-example*"
    (shell-command "echo This is an example"
                   "*blog-example*"
                   "*Messages*")
    (pop-to-buffer "*blog-example*")))

The above snippet defines a function named blog-example. It takes no arguments and is interactive (as indicated by the second line calling interactive). This call to interactive makes blog-example available to be called interactively, meaning you can call it after triggering M-x. This is probably a simplification of what is actually does, so if you care the documentation is available here.

After the call to interactive we hit the core of this function, the call to with-output-to-temp-buffer. This function a buffer name as a first argument and additional forms. The output of those forms is put into the named buffer.

The form I’m passing to with-output-to-temp-buffer is a call to shell-command. shell-command will run echo This is an example synchronously and redirect stdout to *blog-example* and stderr to *Messages*.

The final line opens the buffer and switches focus to it. Now you can look at the output and when you are ready to return just hit q.

This is a simplified example but it shows how easy it is to extend Emacs functionality. Doing something similar to this made a task I do frequently more pleasant.

My use case is a bit more complicated and involves saving the buffer I’m currently editing and then running a command against the saved file. Below is some sample code that does something similar.

1
2
3
4
5
6
7
8
9
10
11
12

(defun example2 ()
  (interactive)
  (when (and (buffer-modified-p)
             (y-or-n-p (format "Save file %s? " (buffer-file-name))))
    (save-buffer))
  (with-output-to-temp-buffer "*blog-example*"
    (shell-command (concat "wc -l"
                           " "
                           (expand-file-name (buffer-file-name)))
                   "*blog-example*"
                   "*Messages*")
    (pop-to-buffer "*blog-example*")))

Put together a new release of lein-autoexpect today. lein-autoexpect is a plugin for Leiningen that monitors your source directories for changes and then reloads your code and runs your expectations. It reports test output to the console and optionally sends notifications to Growl (and Growl like notification tools).

To use lein-autoexpect, add :plugins [[lein-autoexpect "1.0"]] to your either your project’s project.clj or your global ~/.lein/profiles.clj. To use the plugin run lein autoexpect. This will display the test results to the console. To also have results reported using Growl run lein autoexpect :growl.

Release 1.0 of lein-autoexpect upgrades its dependency on org.clojure/tools.namespace to version 0.2.4. It also no longer crashes if there isn’t a Growl connection available.

If you haven’t tried using expectations and lein-autoexpect I encourage you to give it a try. Having my tests run automatically made a huge positive difference on my development experience.

You may have seen me tweeting about building custom Kindle dictionaries. A few months ago I made a custom Kindle dictionary for Dune and my reading experience greatly improved. Being able to look up unknown terms as easily as English words was amazing. Ever since I’ve been looking for other books that could benefit from having a custom dictionary. While reading Fogus’s Functional JavaScript I saw the opportunity to make a another.

I was taking my time reading Fogus’s book and, as a result, found myself forgetting the implementation of functions defined earlier in the book. I wanted to be able to look up implementations easily and realized that a dictionary of function names to implementations would solve my problem.

I found the book’s repo and confirmed the license would allow this. Then extracted the data (wrote a simple parser in Clojure, extracts functions that follow this format) and made a dictionary.

Steps to using my custom dictionary:

1. Download the dictionary (titled Functional JavaScript Companion).
2. Put it on your e-ink Kindle (transfer over USB or email it).
3. Change your default English dictionary to Functional JavaScript Companion.
4. Start reading Functional JavaScript. Look up function implementations the same way you would normal English words.

You can change your Kindle’s default dictionary by navigating to Settings > Device Options > Language and Dictionaries. You don’t need to do this with all custom dictionaries but it is pretty much a requirement for this one. Many of the function names are English words and as a result if you don’t change the default to Functional JavaScript Companion you’ll end up looking up the definitions of standard English words.

This dictionary isn’t perfect but it did improve my reading experience. One example of where it fails is if you look up the function partial1 it will look up partial. This is result of how the Kindle looks up words. Another minor issue is that the functions are often too large to fit in the pop-up window. The fix to both of these is to click the “Show Full Definition” button of the pop-up to be taken to the dictionary. Another issue is that the numerous functions defined by composition (example: isOdd) are not parsed by my parser and therefor not part of the dictionary.

This was definitely a larger challenge than creating my custom Dune dictionary. It forced me to dive into the Amazon documentation a bit and figure out more of the markup language. I have notes on my experience creating Kindle dictionaries and sometime in the future will be writing a post with details about what I’ve learned.

I can’t recommend Fogus’s Functional JavaScript enough. If you do read it give my dictionary a shot. I think it makes the reading experience a bit nicer.