Setting up a development environment is a painful process, as I discussed previously when detailing how mine evolved to the combination of home-manager and Ubuntu Make. Little did I know, the article gained significant traction and likely resonated with many fellow developers. Coincidentally, I got into a new job earlier in the month, and this necessitated a revision to the setup on a MacOS environment. How did it go? Turbulent would be an appropriate adjective to describe the whole experience, as you may expect.
From Dotfile Hacks to Open Source: My Development Environment Evolution
A cute illustration from Copilot
The macOS Catalyst: A New Job, A New Challenge
Replicating a development setup over multiple machines can be a hard problem, and ensuring it works on different operating systems significantly increases that complexity. As we delve deeper into the topic, you will see how I shoot myself in the foot repeatedly, and the corresponding steps to recover.
Why a Mac?
Photo by Claudio Schwarz on Unsplash
Before starting my new role, I was asked if I would prefer a Linux machine or a Mac for work. Though I am relatively comfortable with a Linux machine, I picked Mac as I reckoned it is easier to get enterprise apps whenever necessary. I also reiterated this same reason when my coworker and I were pairing up for a task.
Despite working mostly with Ubuntu, I do have a Intel iMac for video calls and functioning as a second screen for documentation and web project previews. A minimal home-manager setup worked mostly fine, and I assumed expansion of the setup for development would be as painless.
Oh, how wrong I was.
As the assigned MacBook Pro became my primary device for work, all the minor problems surfaced. First, my shells were not initialized properly in the terminal emulator. After getting that fixed, I got adventurous and decided to retire rbenv to manage Ruby versions. Coupled with the struggle to set up my code editor, I was amazed how my feet survived all the gunshots.
Diving into the Rabbit Holes: macOS and Interpreter Woes
How hard can it be?
Photo by Nachristos on Unsplash
If one is looking for a statement to best represent underestimation, that would be it. Looking back, the problems look very trivial, but each was a great learning opportunity. Hang on tight, as we are going into a deep dive into all the rabbit holes I fell into with the revised setup, and to learn how to overcome them.
Compared to Windows, macOS offers a more familiar environment for developers with experience in Linux. We can interact with the system through a similar command-line interface. With a package manager, we can even add different shells not shipped with the operating system, like my favourite fish shell. Trying to configure an iTerm2 profile for fish is exactly the first hole I fell into.
Why would macOS not source .profile in a graphical session?
Unlike Ubuntu, macOS does not source .profile in a graphical session. Most of the time, the script updates the PATH variable, making locally installed applications discoverable. Subsequently, shell initialization scripts (like .bashrc for bash and config.fish for fish) then set up individual application based on the environment defined in .profile.
Multiple variations was attempted, until I stumbled upon a Q&A on superuser. Thinking it would be useful, I sent the proposed answer to Gemini and asked if it would be applicable for my case. Turns out I only needed a little tweak.
/bin/sh -lc "exec -l /path/to/fish" This is a breakdown of the command:
- /bin/sh: This specifies that the command should be executed by the Bourne shell, launching a new instance of this basic shell.
- /bin/sh -l: this tells the shell to launch as a login shell, ensuring it sources standard login files like .profile.
- /bin/sh -c: This tells the shell to execute the command provided in the string that follows.
- exec: This crucial command tells the shell to replace the current /bin/sh process with a new process (in this specific case, the fish shell).
- exec -l: This tells the new shell (e.g., fish) to launch as a login shell itself.
- /path/to/fish: This is the full path to your fish shell executable. With the -l flag in the exec command, this fish shell will then source its own login initialization configuration file (e.g., config.fish).
Similarly, the command can be adapted to work with byobu, my preferred terminal multiplexer. The -l flag is omitted for the exec command as byobu is not a shell.
/bin/sh -lc "exec /path/to/byobu new-session" For completeness’ sake, the setup for a bash profile is a lot simpler, as it sources .profile and .bashrc properly when launched as a login shell -l itself.
/path/to/bash -l What a ride.
Shifting from Python to Ruby as my main development language presented an opportunity to review the toolings involved. Previously I relied on pyenv to provide isolated Python interpreters for individual projects. Eventually, the functionality was replaced by uv, which also manages pre-built CPython distributions as well. Unfortunately, I wasn’t aware of an alternative for Ruby, and had to continue relying on rbenv for the purpose.
One of the main goals of using home-manager, as discussed in the previous article, was to achieve idempotency and to isolate dev packages from the system package manager (such as apt in Ubuntu). Thus, we need to first define the packages in the home-manager flake file, as shown below:
packages = with pkgs; [ ... # dev libraries openssl zlib-ng bzip2 readline sqlite ncurses xz tcl tk xml2 xmlsec libffi libtool libyaml libxml2 libxslt ]; Essentially, this yields an isolated filesystem namespace that is unrecognized by the operating system. In order to get the compiler to utilize the pulled development package, we need to tell rbenv where to look for them, as shown in the shell script below:
#!/usr/bin/env sh export CPPFLAGS="-I$HOME/.nix-profile/include $CPPFLAGS" export CFLAGS="-I$HOME/.nix-profile/include $CFLAGS" export LDFLAGS="-L$HOME/.nix-profile/lib $LDFLAGS" export PKG_CONFIG_PATH=$HOME/.nix-profile/lib/pkgconfig:$HOME/.nix-profile/share/pkgconfig:$PKG_CONFIG_PATH export C_INCLUDE_PATH=$HOME/.nix-profile/include:$C_INCLUDE_PATH export INCLUDE_PATH=$HOME/.nix-profile/include:$INCLUDE_PATH export LIBRARY_PATH=$HOME/.nix-profile/lib:$LIBRARY_PATH export LD_LIBRARY_PATH=$HOME/.nix-profile/lib:$LD_LIBRARY_PATH $HOME/.nix-profile/bin/rbenv install -vf 3.3.5 It worked fine, though I always found this a dirty hack, as it required explicitly defining the flags and include paths. There has to be a better way.
The Nix Pivot: Discovering a Declarative Path
If not rbenv, then how do we approach Ruby version management? Turns out we can use another feature provided by Nix to build a development environment through a declarative manner. So how is this relevant to the problem we are looking at though?
Photo by Jonathon B. Carreño on Unsplash
A friend of mine had been hinting for a while that I could completely replace rbenv with Nix but I didn’t explore deeper. That changed when I saw the announcement of the gemini-cli app. In the installation guide, it called for npx, but the tool was not available as a Nix package. After some exchanges with Gemini, it suggested I look into nix shell, and the conversation quickly evolved into an introductory session on nix develop.
Just a side note, gemini-cli was already packaged not long after the announcement, no need for npx. If I were to follow the installation guide, npx could be made available through a nix shell -p nodejs session. In case you missed how this could be related to our original problem, the equivalent for Ruby would be nix shell -p ruby.
What if I need to install gems that require some special dev libraries?
You just unlocked the reason the conversation turned into an introduction to nix develop. Like home-manager, the input for the command is a flake file, written in the Nix programming language, declaring the development environment setup. Naturally, this added complexity made me hesitant to commit to this strategy. I was still trying to get more comfortable with home-manager and wanted to take more conservative baby steps in adopting additional features offered by Nix.
Essentially, there are two important components to be declared in the flake: the list of packages and an initialization shell script. For example, we can define a development environment sporting Ruby 3.3 with the list of packages like so:
buildInputs = with pkgs; [ # fish is kept so that fish-specific completions from # packages can be made available. fish # Git is needed for fetching gems from git repositorie git # The Ruby interpreter (3.3.8 as the time of writing) ruby # Bundler for managing Ruby gems. bundler # Common dependencies for building native extensions. cmake gcc gnumake # Libraries often required by gems. openssl pkg-config libmysqlclient shared-mime-info re2 readline zlib ] ++ lib.optionals stdenv.isDarwin [ # Add macOS-specific dependencies. libiconv ]; Secondly, the initialization script definition
shellHook = '' # Ruby-specific setup export FREEDESKTOP_MIME_TYPES_PATH="${pkgs.shared-mime-info}/share/mime/packages/freedesktop.org.xml" echo "Ruby dev environment activated for ${system}!" ''; Incorporating these into the flake file makes nix develop a superior method to set up a development environment. Compared to just listing packages in nix shell, the shellHook component allows more control and further manipulation, enabling us to set up proper environment variables and run initialization commands. Additionally, the declarative nature of a flake file also means it is highly reproducible and can be easily shared.
A Harmonious Setup: Validation and Reflection
Remember I needed to redefine library path and compiler flags to compile Ruby interpreters? That problem eventually came back while I was installing gems for work projects. Long story short, I gave in and adopted nix develop. Does it work out of the box? Does the adoption of nix develop stop me from shooting myself in the foot?
In order to be cautious, I replicated and adapted the flake for my bigmeow side project, which is still managed by Poetry, and hence relied on pyenv. Fortunately, a satisfactory outcome was achieved after a few iterations. On the other hand, direnv also offers some integration with nix develop. Automated activation of a development environment can be achieved by adding one line to the respective .envrc:
use flake /path/to/folder # the folder that contains the flake.nix declaration
Photo by Sigmund on Unsplash
Applying the Ruby counterpart also went rather smoothly. There were some problems in the earlier iterations, but they were mostly trivial, and all it took was a regeneration by Gemini. Compared to the relatively smooth sailing in environment setup, taming my code editor to work with it was a lot more frustrating.
In recent years, most extensions bridging the code and development tools would often bundle the tool together. For instance, if I install an extension to format Python code with ruff formatter in Visual Studio Code, it would bundle ruff itself in the package. This saves the developer from needing to declare ruff as a dependency of the project.
Likely due to the highly modular design, the experience was a lot more complicated in the Ruby world. One of my work projects defined an ancient version of the Rubocop formatter, and none of the editor extensions offered in the marketplace would work with it. Forcing the extension to use the bundled gem would fail, because it wouldn’t work with the equally outdated rubocop extension gems (rubocop-rails, rubocop-minitest) installed in the project.
Updating the lock file is not a practical solution and I was stuck there for a while. Thankfully a coworker shared that I could explore the ruby-lsp extension. Apparently, this extension makes it possible to refer to an alternative Gemfile, i.e. I could have a Gemfile for all the development tools saved outside of the project. In this particular case, I could have a Gemfile like this
source "https://rubygems.org" gem "ruby-lsp" gem "rubocop", "1.26.1" gem "rubocop-rails" gem "rubocop-minitest" Solving that yields an functional setup that I was happy enough for work. Looking back, everything seemed rather trivial, but in reality I was stuck for a couple of days. Despite the added complexity, getting all these parts working together felt like putting together puzzle pieces. Getting them configured correctly is giving the same satisfaction as solving a complex jigsaw puzzle.
Is this increased complexity worth the effort? Would there be a better alternative strategy out there? I am curious to know about them and am open to recommendations and corrections. Meanwhile, I do find the revision to retire version managers a fruitful learning experience. Another part I liked about the current implementation is the elimination of explicit declaration of dev packages to build the interpreters. If you ask me, I would say replacing the version managers and removing dev packages for interpreter compilation with nix develop flake files is totally worth the effort.
Throughout this journey, Gemini served as a valuable editorial companion, helping refine my prose. However, the voice and code remain entirely my own. I welcome your feedback and recommendations in the comments below, and invite you to subscribe to my Medium for more content on my development adventures!
Top comments (0)