Nix Progress

Thanks to Divam, there are significant improvements to the Nix configuration in my various projects! I am currently digesting his pull requests and will push updates to all projects soon.

Improvements

One very important thing that I learned is that a default.nix file is used when a resource is accessed via Nix. I pinned the nixpkgs revision in the default.nix files of my projects, which is problematic when the package is used by other software via Nix. I had been sure to not include Nix configuration in the sdist tarballs that are uploaded to Hackage, but Nix users may pull the source directly from GitHub instead of using Hackage.

Divam also showed me a really nice way to configure multiple GHC versions, which is something that I have wanted to figure out how to do in Nix for quite some time. The solution uses nix-thunk, a “lightweight Nix dependency manager.” I can use it to provide an easy way to create a shell for a specified version of GHC as well as test building across all of the supported GHC versions.

Frustrations

While I am making progress, I still experience many frustrations. I have a lot more to learn, so many of my frustrations are likely due to my lack of knowledge.

Perhaps the biggest frustration is that Nix seems to be quite fragile. I speculate that it is due to the incredibly large amount of work required to maintain the ~60,000 packages. It is not feasible to test compatibility for every commit, and things break all the time. In my experience, building a non-trivial Haskell program using a new nixpkgs revision fails way more often than it succeeds. Perhaps I have poor luck.

To develop Haskell software using Nix, one has to find a nixpkgs revision in which the set of packages that is required to develop the software all works together. The following service has been invaluable in doing so:

https://lazamar.co.uk/nix-versions/?channel=nixpkgs-unstable&package=ghc

As I write this, there are no revisions listed for GHC 9.0.1, which is a particularly important version to test against at this time. Divam’s pull request uses revision de7e0f7d, which was probably just the HEAD when he started working on the pull request. TTC builds successfully using this revision, but the shell (including cabal-install) fails. Running a shell results in building a lot of software and eventually (after many hours on my not-too-impressive computer) fails with the following error:

...
Configuring memory-0.15.0...

Setup: Encountered missing or private dependencies:
base >=4.9 && <4.15
...
error: build of '/nix/store/jz6p2xixzl2q56vg3mn3d8w3gaxvhxf5-ghc-9.0.1-with-packages.drv' failed

GHC 9.0.1 uses base-4.15, which is out of the bounds listed in the error. The problem is that memory-0.15 is being used instead of memory-0.16. What in ghc-9.0.1-with-packages.drv is causing this? Since there is no way that the package can build with such an issue, how can that be committed to the repository? Was it not tested before committing? I doubt it, but I have no idea. I get the impression that Nix is fragile because I run into these kinds of errors so frequently.

This example also illustrates another major frustration that I have: using Nix requires a lot of time. Even when there is a cache hit, there are many huge downloads that can be pretty time-consuming. When there is a cache miss, building packages can kill my productivity at a level comparable to Windows updates. Finding a nixpkgs revisions for which the set of packages that I need work together can take days of compute time.

In this case, I suspect that my problem is that nixpkgs only works for older versions of GHC, even though it has GHC 9.0.1 packages. If true, this is also frustrating. (I have used GHC 9.0.1 with Stack for months, making sure that my open-source software works with the new compiler and package versions before they are included in snapshots.)

I plan on trying to build a GHC 9.0.1 shell one more time, using the current HEAD. If it does not work, I will simply not include GHC 9.0.1 support with Nix yet.

Entertainment

I recently discovered Ian Henry’s How to Learn Nix series of blog posts. They form a stream of consciousness diary as the author reads the Nix manual and tries to learn Nix. I like Ian’s sense of humor and have literally laughed out loud many times while reading. The blog posts are well-written, entertaining, as well as educational.