Reflex Tutorial Upgrade (Part 5)

I first attempted to upgrade the Reflex FRP calculator tutorial to use the latest version of Obelisk on the first, and I had no idea that I would still be pulling on that thread ten days later. My goal was to get some practice with managing a Reflex FRP project using Nix. I was indeed able to do that, but it is unfortunate that I gave up on that task without success. This blog entry is for reflections.

This task has taken a lot more time than I had anticipated. The primary reason is that it has not been my priority project. I have been busy with other things, including reading Really Feely: Farm and computer science papers to my baby. Long Nix build times have definitely contributed to the slow progress as well, though.

I gave up on trying to get the project to build using ob run when I reached ghcid issues with the Markdown-formatted source code. This is not an issue that I want to work on, as I do not think I will ever have such a situation in an actual project.

I stopped trying to get the project to build using nix-build just because I do not want to allocate any more time to this. The project uses a fork of mmark with commits that are not merged into newer versions. That fork does not work with the newer versions of dependencies that are provided by the new version of Obelisk. Creating a fork from a newer version of mmark and porting the special commits would likely resolve that issue.

Nix Cache

One thing that I was able to confirm is that my Nix cache settings are working. I was using the single-user mode when I started, and I was unsure if the cache was working fully because build times were so slow. I was unable to find any definitive documentation that states that /etc/nix/nix.conf is used with single-user mode.

I figured out how to do a multi-user Nix installation on my system. The permissions issue that I ran into before (see Uninstalling Multi-User Nix) seems to have been fixed, and I was able to debug the mess of shell configuration. I was able to see how things work without cache before restarting the Nix daemon, as well as confirm that cache was being used afterwards. I confirmed that the cache settings were indeed being used in the single-user mode. Slow build times are normal, and my system is not very fast.

ob --verbose Issue

Using the latest release of Obelisk, I ran into the ob --verbose invalid character failure issue, which made it impossible for me to see actual build failures. I initially got around this issue by hacking Obelisk and replacing Unicode characters with ASCII characters. A friend relayed some information about the cause, which I wrote about in the Nix Terminfo and Local Archive blog entry. I implemented a hack that sets locale environment variables to C.UTF-8, re-executing the program when necessary.

nix-thunk

Hacking on Obelisk gave me a chance to really try out the various ob thunk commands, implemented using nix-thunk. From a developer perspective, it provides a really convenient way to develop changes to dependencies. I really enjoyed using it!

Note that I was first introduced to nix-thunk by a friend last summer, when I was trying to use Nix to test software across many different versions of GHC. I ended up not using it because it is overkill for the simple task of selecting nixpkgs revisions that I would never unpack. It is much more useful in a Reflex FRP project!

Maintenance

I have the impression that Nix generally makes project maintenance difficult/expensive, and this experience did not change that impression. Nix projects tend to be carefully constructed snapshots of versions of software packages that work together. Upgrading seems to be a major task, because a new snapshot of mutually-compatible package versions needs to be found or created. I can easily imagine such projects getting stuck on old versions of dependencies and being very difficult to update, especially if there is employee turnover and the original developers are no longer around.

The mmark dependency illustrates one type of maintenance problem. A fork (branch) was created in order to make some changes that were required for the project, but those changes were never merged upstream. Such a dependency on a special branch increases the difficulty of upgrading the project to use newer dependencies because that dependency cannot simply be upgraded. I have seen many branches in repositories related to Reflex FRP and had assumed that they are feature branches, but I now worry that there may be a lot of such forks. “Feather branching” can make software difficult to maintain indeed.

The functionality that Reflex FRP provides is amazing, and Nix is a very important part of that. From a project management perspective, what can be done to make projects easier to maintain? The following are some ideas to try out in my own Reflex FRP projects.

One thing that I think would help is having clear releases. For example, I like to make/merge commits into a develop branch and only commit releases to the main/master branch. With a clear separation between development and releases, different quality standards can be established to greatly improve the quality of releases without putting too much burden on development.

Hacking on dependencies using nix-thunk is really convenient, but I think that some discipline is required to make things easier to maintain. For example, perhaps it would be a good policy to document any thunk dependencies that are not themselves releases. If a thunk points to a fork, then the parent project should document why that fork is being used instead of a release. It would probably also be wise to create releases that only use releases of dependencies, not forks. Forks are fine during development, but changes should be released before they are used in the release of another project.