FeedPipe

As I wrote in the Tagged Index blog post, I would like to implement the functionality in standalone software. I have been thinking about the design space for a few days, and I started to type up concrete specifications today in the form of configuration files.

Name

As is almost always the case, it is really difficult to decide on a name for a new project. That is one of the decisions that I have been deliberating over the past few days.

The primary goal of the software is to make it easy to discover new items of a specific category via RSS. For example, the Haskell Books index provides a way for people to be notified when an English Haskell book is published. The web content provides a way for people to discover older items as well, but I consider this an additional benefit since this service could also be implemented using a wiki. The tag filtering functionality is a minor convenience.

One naming theme that I considered is the idea of an oasis. This came to mind because people often claim that RSS is dead. If the fertility of the internet has been dried up by huge corporations, then remaining RSS feeds are like an oasis in the desert. I did not think of any suitable names that are not already in use, however.

Another naming theme that I considered is that of accretion in astrophysics. Massive objects attract matter by gravitational force, and space that is close to the object is relatively empty because any matter in that space is deep in the gravity well. The analogy here is that massive corporations pull in many users, while RSS feeds are bastions of the distributed internet that are away from those gravity wells. I did not think of any suitable names that are not already in use, however.

I then started considering names based on feed. I like the name “feeder,” but of course an RSS company already uses that name. I eventually selected the name FeedPipe. This idea comes from Chris Penner’s static site generator called SitePipe. (I am considering using Slick, the successor to SitePipe, in the implementation.)

Goals

Here are some major goals for the project:

• Since the software should be easy to use by many people, the software should be distributed as an executable. It should not require compilation of Haskell code.
• The software should create static output. It should be able to create standalone output as well as output that can be hosted on an existing website.
• The software should support any kind of item, not just books.
• It should be easy to create a site with any number of indexes and RSS feeds.
• The output should support any language, not just English.

Current Design Notes

Today, I wrote configuration files in “literate YAML” format. The configuration has extensive documentation/specifications in comments that can be rendered using LiterateX. Here are some brief notes about what the current design supports:

• It is easy to create any number of indexes and RSS Feeds.
• The language can be set per feed. For example, feeds for Haskell books in various languages can be implemented.
• The time zone can be set per feed. For example, the index of English Haskell books can use the UTC time zone while the index of Japanese Haskell books can use the Asia/Tokyo time zone.
• The locations that output files are written to is highly configurable, so that output can be written to the correct directories of an existing website. URLs in the HTML and RSS files are set accordingly.
• RSS feeds include all items by default, but a limit can be set.
• Two layout modes are supported. The directory layout mode renders an item with slug example to example/index.html and uses URLs that end with example/. The file layout mode renders an item with slug example to exampe.html.
• Any kind of item is supported, not just books. (The metadata can include arbitrary keys that are available when rendering templates.)
• Sorting rules can be configured per feed. Arbitrary keys can be specified, so Japanese content can be sorted correctly. (For example, Japanese book titles may use kanji, which cannot be lexicographically sorted. A separate string must be provided for correct sorting.) Locale-specific sorting functions are supported, such as English title sorting that ignores leading articles.
• Static files, CSS files, and JavaScript files can optionally be copied to specific output directories. Building of these files must be done before FeedPipe is run, which keeps FeedPipe simple.
• Comments are supported!
• Revisions are tracked as is done on this website. Major revisions can cause an item to be output in the RSS feed. For example, a new edition of a book should cause a notification. Minor revisions do not cause an item to be output in the RSS feed. For example, making corrections or adding links should not cause a notification. (I discovered that the implementation of RSS guid values in my website does not result in this behavior, so that is an issue that I will fix soon.)

haskell-feeds

I created a GitHub organization called haskell-feeds. I plan on testing the new implementation there, using GitHub Pages to host the output. When the software is ready, I will solicit feedback from the community. The indexes and feeds may stay at haskell-feeds, or they may move to a different location, so haskell-feeds should only be used for testing. When a final location is determined and configured, I will make an announcement.