If you've got a ridiculously good memory and you've been reading my writing for a while, you know I'm a fan of processes notifying you when they are done. I often have some task running in a hidden terminal that performs actions when files change. This is most often running Clojure tests whenever a file changes using either test-refresh or lein-autoexpect. Another common watch task is rendering this website whenever one of the markdown files changes.

I don't like needing to have these processes always visible, since I mostly only care about when they finish. When running unit tests, I don't need to see the output unless a test is failing. When writing articles, I only care about when the rendering is done so I know I can refresh my browser to review the output.

On macOS, one way of doing this is using terminal-notifier. terminal-notifier makes it trivial to send notifications.

Below is the script I run while working on this website. It uses entr to monitor the input files; when changes are detected, it renders this site using my homegrown Babashka static site generator, and when that finishes, it uses terminal-notifier to alert me.

#!/bin/bash
while sleep 0.5; do 
    rg bb templates source --files -t css -t clojure -t markdown -t html \
        | entr -d -s 'rm -rf output/*; bb render && terminal-notifier -message "Rendering complete"';
done

This site renders quickly, so I can usually make some edits, save, and toggle to a browser to refresh and see the output. Still, it is nice to see that little notification pop-up on my screen so I know for sure that if I hit refresh, I'm seeing the latest render.

When I'm running my Clojure tests, both lein-autoexpect and test-refresh send a notification with a pass or fail message based on the status of the unit tests that just ran. If the tests are passing, I don't have to glance at my terminal. If they are failing, I do.

I'd encourage you to think about what processes you might want to get notifications from when they are done and look into how to set that up. terminal-notifier works great on macOS. I can't make recommendations for other operating systems since it has been years since I've used any alternatives besides SSHing into a Linux server.

It is worth the effort to figure out how to have notifications. They remove a trivial inconvenience (having to switch programs, needing to keep a window visible on your screen) and make life a little better. By stacking small, slightly life-improving techniques, all of a sudden you find yourself much more productive.

Selecta is a command-line utility that gives you the power to fuzzy select items from a list of text. What does that mean? It means you pipe selecta a list of text on stdin, it helps you make a choice from items in that list, and then selecta prints that choice to stdout.

Here is an example of me using it to help me narrow in on what file I'd like to pass to wc.

In this example, I search for markdown files using ripgrep (rg), type part of a filename, hit enter to select the match, and then see the wc stats of that file. This isn't the greatest example of using selecta but it adequately shows what it does.

Some number of years ago, I wrote a script called connect-db. This script used selecta, along with grep, sed, and cut, to provide a very pleasant command-line experience for connecting to known databases. My coworkers and I used this script frequently.

By combining selecta with other stdin/stdout friendly command-line tools you can build really enjoyable, time-saving tools. Selecta is a useful utility to add to your toolkit.

I'm currently keep notes on my computer using plain text and Org mode.

I keep my notes in a git repository in my home directory, ~/org/. I want my notes to be synced between my computers without me thinking about it. Historically, I've reached for something like Google Drive or Dropbox to do this but this time I reached for git and GitHub.

Below is the script that I ended up cobbling together from various sources found online. The script pushes and pulls changes from a remote repository and works on my macOS and linux machines.

The loop starting on line 38 does the work. Whenever a file-watcher notices a change or 10 minutes passes, the loop pulls changes from a remote repository, commits any local changes, and pushes to the remote repository. The lines before this are mostly checking that needed programs exist on the host.

I keep this running in a background terminal and I check periodically to confirm it is still running. I could do something fancier but this isn't a critical system and the overhead of checking every couple days is nearly zero. Most of the time checking happens by accident when I accidentally maximize the terminal that runs the script.

I've been using this script for a long time now and I've found it quite useful. I hope you do too.

#!/bin/bash

set -e

TARGETDIR="$HOME/org/"

stderr () {
    echo "$1" >&2
}

is_command() {
    command -v "$1" &>/dev/null
}

if [ "$(uname)" != "Darwin" ]; then
    INW="inotifywait";
    EVENTS="close_write,move,delete,create";
    INCOMMAND="\"$INW\" -qr -e \"$EVENTS\" --exclude \"\.git\" \"$TARGETDIR\""
else # if Mac, use fswatch
    INW="fswatch";
    # default events specified via a mask, see
    # https://emcrisostomo.github.io/fswatch/doc/1.14.0/fswatch.html/Invoking-fswatch.html#Numeric-Event-Flags
    # default of 414 = MovedTo + MovedFrom + Renamed + Removed + Updated + Created
    #                = 256 + 128+ 16 + 8 + 4 + 2
    EVENTS="--event=414"
    INCOMMAND="\"$INW\" --recursive \"$EVENTS\" --exclude \"\.git\" --one-event \"$TARGETDIR\""
fi

for cmd in "git" "$INW" "timeout"; do
    # in OSX: `timeout` => brew install coreutils
    # in OSX: `fswatch` => brew install fswatch
    is_command "$cmd" || { stderr "Error: Required command '$cmd' not found"; exit 1; }
done

cd "$TARGETDIR"
echo "$INCOMMAND"

while true; do
    eval "timeout 600 $INCOMMAND" || true
    git pull
    sleep 5
    STATUS=$(git status -s)
    if [ -n "$STATUS" ]; then
        echo "$STATUS"
        echo "commit!"
        git add .
        git commit -m "autocommit"
        git push origin
    fi
done

Lately, I've been using tmux a lot. This resulted in me figuring out how to get lein-test-refresh to send notifications using tmux.

The setup linked above works great for when I'm doing work all by myself. It showed a problem when using ssh and tmux to pair with another developer. Instead of both developers receiving a notification, only one did. One is better than none but not ideal.

Below is a GIF showing the problem. Each window simulates a different developer.

This wasn't too hard to fix. A little digging through the tmux manpage shows that tmux display-message takes an optional flag telling it which client receives the message. If we can get a list of all the clients then iterating over them and sending a message to each is straightforward.

tmux list-clients give us this list. Below is the output.

$ tmux list-clients
/dev/ttys002: 0 [78x41 xterm-256color] (utf8)
/dev/ttys006: 0 [78x42 xterm-256color] (utf8)

What we care about are the parts that look like /dev/ttys002. At first I used cut to grab these values but then I dug a bit deeper into the tmux manpage.

It turns out that you can specify a format to tmux list-clients. Running tmux list-clients -F "#{client_name}" gives us the output we care about.

$ tmux list-clients -F "#{client_name}"
/dev/ttys002
/dev/ttys006

We can combine that with xargs to send a message to multiple clients.

That command is a bit much to put into lein-test-refresh's configuration so I shoved it in a script called notify and configured lein-test-refresh to use it. Script and GIF of that below. Now both you and your pair can get notifications.

#!/bin/bash

USAGE="Usage: notify <message>

example: notify 'Tests passed!'"

if [ -z "$1" ]; then
    echo "$USAGE"
    exit 1
fi

message="$1"

tmux list-clients -F "#{client_name}" \
    | xargs -n1 -I{} tmux display-message -c {} "$message"

I've been using Emacs in a remote tmux session lately and I've been missing lein-test-refresh notifications when my Clojure tests pass or fail. Luckily, it only took me a little bit of searching to figure out a solution for when I'm working inside of tmux.

Below is a GIF of the notifications I get as my tests run and pass or fail.

With the above notifications, I can keep my focus on my code and only switch to the tmux window with lein test-refresh running when a test fails.

This was pretty easy to setup. You can trigger a message in tmux by running tmux display-message <MESSAGE_HERE>. To configure lein-test-refresh to send notifications to tmux simply include the following in your :test-refresh section of your project.clj or profiles.clj.

:test-refresh {:notify-command ["tmux" "display-message"]}

I hope you enjoy this. Its has made using a remote terminal with tmux and lein-test-refresh more enjoyable.

I recently found myself in a situation where I needed to confirm that a process took in a tab separated file, did some processing, and then output a new file containing the original columns with some additional ones. The feature I was adding allowed the process to die and restart while processing the input file and pick up where it left off.

I needed to confirm the output had data for every line in the input. I reached to the command line tool comm.

Below is a made up input file.

UNIQUE_ID	USER
1	38101838
2	19183819
3	19123811
4	10348018
5	19881911
6	29182918

And here is some made up output.

UNIQUE_ID	USER	MESSAGE
1	38101838	A01
2	19183819	A05
3	19123811	A02
4	10348018	A01
5	19881911	A02
6	29182918	A05

With files this size, it would be easy enough to check visually. In my testing, I was dealing with files that had thousands of lines. This is too many to check by hand. It is a perfect amount for comm.

comm reads two files as input and then outputs three columns. The first column contains lines found only in the first file, the second column contains lines only found in the second, and the last column contains lines in both. If it is easier for you to think about it as set operations, the first two columns are similar to performing two set differences and the third is similar to set intersection. Below is an example adapted from Wikipedia showing its behavior.

$ cat foo.txt
apple
banana
eggplant
$ cat bar.txt
apple
banana
banana
zucchini
$ comm foo.txt bar.txt
                  apple
                  banana
          banana
eggplant
          zucchini

So how is this useful? Well, you can also tell comm to suppress outputting specific columns. If we send the common columns from the input and output file to comm and suppress comm's third column then anything printed to the screen is a problem. Anything printed to the screen was found in one of the files and not the other. We'll select the common columns using cut and, since comm expects input to be sorted, then sort using sort. Let's see what happens.

$ comm -3 <(cut -f 1,2 input.txt | sort) <(cut -f 1,2 output.txt | sort)
$

Success! Nothing was printed to the console, so there is nothing unique in either file.

comm is a useful tool to have in your command line toolbox.

My ideal command history would let me search the history of every shell but when I hit the up arrow it would only cycle through my current shell's history. In February, I was able to achieve this setup in large part because of a utility called hstr.

What is hstr?

hstr is a neat Bash and Zsh utility that lets you easily search, view, and manage your command history. hstr provides a tool named hh that provides a text interface for manipulating your command history. To see what it looks like check out the README and this video tutorial. If you are running OS X and use Homebrew you can install it by running brew install hh.

Making global history searchable but arrows cycle through local history

hstr is a neat tool but my favorite part of my setup is how the global command history is searchable but only a shell's local history is cycled through with the arrow keys. This is achieved by manipulating where history is written and tweaking some environment variables.

The first step is to change your $PROMPT_COMMAND to append your shell's history to a global history file. Below is the snippet that does this from my .bashrc file.

# Whenever a command is executed, write it to a global history
PROMPT_COMMAND="history -a ~/.bash_history.global; $PROMPT_COMMAND"

The next step is to bind a keystroke to run hh, which is what hstr provides, with $HISTFILE pointing to ~/.bash_history.global. I wanted to fully replace the default command history searching (and I use Emacs style keyboard shortcuts) so I've bound these actions to ctrl-r.

# On C-r set HISTFILE and run hh
bind -x '"\C-r": "HISTFILE=~/.bash_history.global hh"'

With those two additions to my .bashrc I've achieved my ideal command history searching. When I hit ctrl-r I'm searching all of my history and yet I only cycle through a shell's local history with the arrow keys. This small addition¹ made my command line productivity higher.

1. My setup was inspired by this StackExchange post.↩