Targeted Use of GNU Tools on macOS using direnv

󰃭 2024-11-28

Working with bash scripts on macOS often leads to an interesting predicament. MacOS ships with the BSD variants of sed, date, base64 and others that diverge just enough from their GNU counterparts to cause incompatibilities.

Take sed for example:

# GNU (Linux) version - works fine
sed -i 's/foo/bar/' file.txt

# macOS version - fails
sed -i 's/foo/bar/' file.txt
# Error: sed: 1: "file.txt": undefined label 'ile.txt'

# macOS version - requires an extension
sed -i '' 's/foo/bar/' file.txt

There are similar stories for date

# GNU (Linux) version
date -d "2024-01-01" +%s
1704067200

# macOS version
date -d "2024-01-01" +%s
# Error: date: illegal option -- d

# macOS alternative
date -j -f "%Y-%m-%d" "2024-01-01" +%s

And base64

# GNU (Linux) version
echo "hello" | base64 -w 0

# macOS version - doesn't understand -w
echo "hello" | base64 -b 0

If you need truly portable scripts then you might have to use some OS detection to branch to the correct command.

# Using uname to detect OS
if [[ "$(uname)" == "Darwin" ]]; then
    # macOS-specific commands
    sed -i '' 's/foo/bar/' file.txt
else
    # GNU/Linux commands
    sed -i 's/foo/bar/' file.txt
fi

This tedious but sometimes it is the only option, but typically the people running the scripts (you!) have some technical ability and have some level of control over their Mac.

One option would be use docker to spin up a linux container to execute the script and there are situations where this is a great option, but I want to cover what I think is a simpler and more elegant solution, which is to make the Mac behave the way we need by installing GNU coreutils with homebrew. GNU coreutils will bring the GNU versions of sed, date, base64 and more to our Mac.

brew install coreutils

The conventional approach then suggests modifying your shell’s PATH via ~/.zshrc or similar:

export PATH="$(brew --prefix coreutils)/libexec/gnubin:$PATH"

But this is a global modification, it affects every terminal session you start. What if you have preexisting scripts on your system that expect to use BSD versions of these tools?

A more surgical approach leverages direnv, a program that can load and unload environment variables depending on the current directory, making it possible to modify the PATH to use coreutils only for the projects that need them. direnv is awesome, it has many uses, I highly recommend you use it!

Installation can be accomplished via Homebrew:

brew install direnv

(Or through asdf, which is my preference, but asdf is too big a topic to cover here)

Now, navigate to your project directory (for example, ~/projects/troublesome-repo/) and create a new file named .envrc:

cd ~/projects/troublesome-repo
touch .envrc

Add the following line to your .envrc:

PATH_add "$(brew --prefix coreutils)/libexec/gnubin"

When you first create or modify an .envrc file, direnv requires explicit authorisation for security reasons. You’ll see a message like “direnv: error .envrc is blocked.” Authorise it with:

direnv allow

Now, whenever you enter ~/projects/troublesome-repo/ or any of its subdirectories, direnv automatically adjusts your PATH to prioritise GNU tools. Step outside this directory, and your environment reverts to using the default BSD tools.

For example:

~ $ which sed
/usr/bin/sed
~ $ cd projects/troublesome-repo
~/projects/troublesome-repo $ which sed
/opt/homebrew/opt/coreutils/libexec/gnubin/sed

With this approach modifications are constrained to where they’re actually needed, rather than polluting the global namespace of your system and allow you to create and use scripts that are much more likely to just work on Mac and Linux without clunky os detection logic.