Leiningen test selectors are great. They allow you to filter what tests run by applying a function to the test's metadata. If that function returns a truthy value then that test will run. lein-test-refresh supports them and even includes a built in one for its focus feature.
I was recently asked if test-refresh could support filtering tests using a regular expression against the name of a namespace or test. Lucky for me, test-refresh already supports this because of its support of test selectors.
Most of the examples of Leiningen test selectors show very simple functions that look for the existence of a keyword in the metadata. We can do more than that. We can write a predicate that does whatever we want with the metadata.
To take a look at a test's metadata, I generated a new project and looked at the generated default test file.
(ns selector.core-test
(:require [clojure.test :refer :all]
[selector.core :refer :all]))
(deftest a-test
(testing "FIXME, I fail."
(is (= 0 1))))
I then used my repl and to see what metadata was on the test.
selector.core-test> (meta #'a-test)
{:test #function[selector.core-test/fn--17267],
:line 5,
:column 1,
:file "/Users/jake/src/jakemcc/blog-examples/selector/test/selector/core_test.clj",
:name a-test,
:ns #namespace[selector.core-test]}
Given the metadata above, I wrote the selector below which lets us select only integration tests.
:test-selectors {:integration (fn [m]
(or (clojure.string/includes? (str (:ns m))
"integration")
(clojure.string/includes? (str (:name m))
"integration")))}
You could write the above code is many different ways. Whatever you write, it needs to look for the existence of integration in either the test's name or namespace.
If you wanted to make lein test or lein test-refresh only run non-integration tests you can add a default test selector to the project.clj.
:test-selectors {:default (fn [m]
(not (or (clojure.string/includes? (str (:ns m))
"integration")
(clojure.string/includes? (str (:name m))
"integration"))))
:integration (fn [m]
(or (clojure.string/includes? (str (:ns m))
"integration")
(clojure.string/includes? (str (:name m))
"integration")))}
Enjoy! I hope this example helps you run a subset1 of your Clojure tests through Leiningen test selectors.
Running a subset of your tests can be helpful and test-refresh has a few features that help you do that. If you can, I'd still recommend making all your tests fast enough to run them all the time.↩
Nearly three years ago I wrote an overview of my Leiningen profiles.clj. That post is one of my most visited articles, so I thought I'd give an update on what I currently keep in ~/.lein/profiles.clj.
{:user {:plugin-repositories [["private-plugins" {:url "private url"}]]
:dependencies [[pjstadig/humane-test-output "0.8.2"]]
:injections [(require 'pjstadig.humane-test-output)
(pjstadig.humane-test-output/activate!)]
:plugins [[io.sattvik/lein-ancient "0.6.11"]
[lein-pprint "1.1.2"]
[com.jakemccrary/lein-test-refresh "0.21.1"]
[lein-autoexpect "1.9.0"]]
:signing {:gpg-key "B38C2F8C"}
:test-refresh {:notify-command ["terminal-notifier" "-title" "Tests" "-message"]
:quiet true
:changes-only true}}}
The biggest difference between my profiles.clj from early 2015 and now is that I've removed all of the CIDER related plugins. I still use CIDER, but CIDER no longer requires you to list its dependencies explicitly.
I’ve also removed Eastwood and Kibit from my toolchain. I love static analysis, but these tools fail too frequently with my projects. As a result, I rarely used them and I’ve removed them. Instead, I’ve started using joker for some basic static analysis and am really enjoying it. It is fast, and it has made refactoring in Emacs noticeably better.
lein-test-refresh, lein-autoexpect, and humane-test-output have stuck around and have been updated to the latest versions. These tools make testing Clojure much nicer.
I'm also taking advantage of some new features that lein-test-refresh provides. These settings enable the most reliable, fastest feedback possible while writing tests. My recommended testing setup article goes into more details.
lein-ancient and lein-pprint have stuck around. I rarely use lein-pprint but it comes in handy when debugging project.clj problems. lein-ancient is great for helping you keep your project's dependencies up to date. I use a forked version that contains some changes I need to work with my company's private repository.
And there you have it. My updated profiles.clj1.
Some of you might wonder why I don't just link to this file in version control somewhere? Well, it is kept encrypted in a git repository because it also contains some secrets that should not be public that I've removed for this post.↩
Recently my team deployed a new version of our ClojureScript UI and it had a minor bug. It was trivial to fix the problem, a ClojureScript build warning pointed us to the cause. As a result we started thinking it would be nice to have build warnings count as errors and fail our ClojureScript build.
We use Leiningen (version 2.5.3) and lein-cljsbuild (version 1.1.1). After some searching we found that lein-cljsbuild supports specifying custom warning handlers as the value to the :warning-handlers key. The lein-cljsbuild README even provides an example, which we took and added a (System/exit 1) to the end of it. This resulted in a build configuration that looked similar to below.
{:id "prod"
:warning-handlers [(fn [warning-type env extra]
(when-let [s (cljs.analyzer/error-message warning-type extra)]
(binding [*out* *err*]
(println "WARNING:" (cljs.analyzer/message env s)))
(System/exit 1)))]
:source-paths ["src/cljc" "src/cljs"]
:compiler {:output-to "resources/public/js/compiled/ui.js"
:externs ["resources/intercom-externs.js"
"resources/mixpanel-externs.js"]
:optimizations :advanced}}
This worked! Well, it sort of worked. Our build failed whenever there was a warning but now we were seeing spurious warnings. We saw "Use of undeclared Var" warnings when functions created in a letfn where calling each other. Definitely not a situation that warrants a warning and definitely not a build failure.
We weren't seeing this warning before so we opened ClojureScript's source and found the default warning handler. The default handler checks that warning-type has a truthy value in the map *cljs-warnings*. Inspired by the default handler we added the same check to the start of our warning handler.
:warning-handlers [(fn [warning-type env extra]
(when (warning-type cljs.analyzer/*cljs-warnings*)
(when-let [s (cljs.analyzer/error-message warning-type extra)]
(binding [*out* *err*]
(println "WARNING:" (cljs.analyzer/message env s)))
(System/exit 1))))]
Success! Now we no longer get incorrect warnings when compiling our letfn form and our build still fails if a warning occurs. Now we can build and deploy with a little more confidence.
Leiningen checkout dependencies are a useful feature. Checkout dependencies allow you to work on a library and consuming project at the same time. By setting up checkout dependencies you can skip running lein install in the library project; it appears on the classpath of the consuming project. An example of what this looks like can be found in the Leiningen documentation or in a previous post of mine.
By default, Leiningen adds the :source-paths, :test-paths, :resource-paths, and :compile-path directories of the checkout projects to your consuming project's classpath. It also recurses and adds the checkouts of your checkouts (and keeps recursing).
You can override what gets added to your classpath by :checkout-deps-shares to your project.clj. This key's value should be a vector of functions that when applied to your checkouts' project map return the paths that should be included on the classpath. The default values can be found here and an example of overriding the default behavior can be found in the sample.project.clj.
I ran into a situation this week where having my checkouts' :test-paths on the classpath caused issues my consuming project. My first pass at fixing this problem was to add :checkout-deps-shares [:source-paths :resource-paths :compile-path] to my project.clj. This didn't work. My project.clj looked like below.
(defproject example "1.2.3-SNAPSHOT"
:dependencies [[library "1.2.2"]
[org.clojure/clojure "1.6.0"]]
:checkout-deps-shares [:source-paths :resource-paths :compile-path])
Why didn't it work? It didn't work because of how Leiningen merges duplicate keys in the project map. When Leiningen merges the various configuration maps (from merging profiles, merging defaults, etc) and it encounters values that are collections it combines them (more details found in documentation). Using lein pprint :checkout-deps-shares shows what we end up with.
$ lein pprint :checkout-deps-shares
(:source-paths
:resource-paths
:compile-path
:source-paths
:test-paths
:resource-paths
:compile-path
#<Var@43e3a075:
#<classpath$checkout_deps_paths leiningen.core.classpath$checkout_deps_paths@6761b44b>>)
We've ended up with the default values and the values we specified in the project.clj. This isn't hard to fix. To tell Leiningen to replace the value instead of merging you add the ^:replace metadata to the value. Below is the same project.clj but with ^:replace added.
(defproject example "1.2.3-SNAPSHOT"
:dependencies [[library "1.2.2"]
[org.clojure/clojure "1.6.0"]]
:checkout-deps-shares ^:replace [:source-paths :resource-paths :compile-path])
This solves the problem of :test-paths showing up on the classpath but it introduces another problem. Checkouts' checkout dependencies no longer show up on the classpath. This is because leiningen.core.classpath/checkout-deps-paths is no longer applied to the checkouts.
Without leiningen.core.classpath/checkout-deps-paths Leiningen stops recursing and, as a result, no longer picks up checkouts' checkout dependencies. My first attempt at fixing this was to modify my project.clj so the :checkout-deps-shares section looked like below.
:checkout-deps-shares ^:replace [:source-paths :resource-paths :compile-path
leiningen.core.classpath/checkout-deps-paths]
The above fails. It runs but doesn't actually add the correct directories to the classpath. The next attempt is below.
:checkout-deps-shares ^:replace [:source-paths :resource-paths :compile-path
#'leiningen.core.classpath/checkout-deps-paths]
This attempt failed quicker. Now an exception is thrown when trying to run Leiningen tasks.
The next one works. It takes advantage of dynamic eval through read-eval syntax. With the below snippet the checkouts' checkouts are added to the classpath.
:checkout-deps-shares ^:replace [:source-paths :resource-paths :compile-path
#=(eval leiningen.core.classpath/checkout-deps-paths)]
Hopefully this is useful to someone else. It took a bit of digging to figure it out and many incorrect attempts to get correct. The full example project.clj is below.
(defproject example "1.2.3-SNAPSHOT"
:dependencies [[library "1.2.2"]
[org.clojure/clojure "1.6.0"]]
:checkout-deps-shares ^:replace [:source-paths :resource-paths :compile-path
#=(eval leiningen.core.classpath/checkout-deps-paths)])
2017-08-27: I've published an updated version here.
Leiningen, a Clojure build tool, has the concept of profiles. One thing profiles are useful for is allowing you to have development tools available to a project without having them as dependencies when you release your project. An example of when you might want to do this is when you are using a testing library like expectations.
Some development tools, such as lein-test-refresh, are useful to have across most of your Clojure projects. Rather nicely, Leiningen supports adding global profiles to ~/.lein/profiles.clj. These profiles are available in all your projects.
Below is most of my profiles.clj. I've removed some sensitive settings and what is left are the development tools that I find useful.
{:user {:plugin-repositories [["private-plugins" {:url "private repo url"}]]
:dependencies [[pjstadig/humane-test-output "0.6.0"]]
:injections [(require 'pjstadig.humane-test-output)
(pjstadig.humane-test-output/activate!)]
:plugins [[cider/cider-nrepl "0.8.2"]
[refactor-nrepl "0.2.2"]
[com.jakemccrary/lein-test-refresh "0.5.5"]
[lein-autoexpect "1.4.2"]
[lein-ancient "0.5.5"]
[jonase/eastwood "0.2.1"]
[lein-kibit "0.0.8"]
[lein-pprint "1.1.2"]]
:test-refresh {:notify-command ["terminal-notifier" "-title" "Tests" "-message"]}}}
:plugin-repositories [["private-plugins" {:url "private repo url"}]] sets a private plugin repository. This allows me to use Outpace's private Leiningen templates for setting up new projects for work.
The next few lines are all related. They setup humane-test-output. humane-test-output makes clojure.test output more readable. It makes using clojure.test much more enjoyable. I highly recommend it. Sample output can be found in my Comparing Clojure Testing Libraries post.
:dependencies [[pjstadig/humane-test-output "0.6.0"]]
:injections [(require 'pjstadig.humane-test-output)
(pjstadig.humane-test-output/activate!)]
Next we get to my :plugins section. This is the bulk of my profiles.clj.
:plugins [[cider/cider-nrepl "0.8.2"]
[refactor-nrepl "0.2.2"]
[com.jakemccrary/lein-test-refresh "0.5.5"]
[lein-autoexpect "1.4.2"]
[lein-ancient "0.5.5"]
[jonase/eastwood "0.2.1"]
[lein-kibit "0.0.8"]
[lein-pprint "1.1.2"]]
The first entry is for cider/cider-nrepl. I write Clojure using Emacs and CIDER and much of CIDER's functionality exists in nrepl middleware found in cider/cider-nrepl. This dependency is required for me to be effective while writing Clojure.
refactor-nrepl is next. clj-refactor.el requires it for some refactorings. I actually don't use any of those refactorings (I only use move to let, extract to let, and introduce let refactorings) but I still keep it around.
com.jakemccrary/lein-test-refresh is next. This lets me use lein-test-refresh globally. lein-test-refresh runs your clojure.test tests whenever a file changes in your project. This is another key development tool in my process.
Up next is lein-autoexpect. It was the first Leiningen plugin I wrote and it enables continuous testing with expectations.
Both lein-autoexpect and lein-test-refresh are projects I created and maintain. Writing lein-autoexpect was my first exposure to continuous testing and it changed how I develop code. I find it frustrating to develop without such a tool.
Next up is lein-ancient. It checks your project.clj for outdated dependencies and plugins. It isn't something that gets used every day but it is super useful when you need it.
The next two entries are for jonase/eastwood and lein-kibit. They are both tools that look at your Clojure code and report common mistakes. I don't use either consistently but I do find them useful. I've found bugs with eastwood.
The final plugin is lein-pprint. lein-pprint prints out your project map. It is useful for trying to grasp what is going on when messing around with various Leiningen options.
The final part, seen below, of my profiles.clj is configuration for lein-test-refresh. It configures lein-test-refresh to use terminal-notifier to notify me when my tests pass or fail. Using a continuous tester that allows flexible notification is useful. Not having to glance at a terminal to see if your tests are passing or failing is great.
:test-refresh {:notify-command ["terminal-notifier" "-title" "Tests" "-message"]}
That is my ~/.lein/profiles.clj. I don't think it contains anything mind blowing but it definitely contains a useful collection of Clojure development tools. I encourage you to check out them out and to think about what tools you should be putting into your global :user profile.