RSS Part 5: Newsboat

Before moving on to RSS clients that I have not tried (recently), I would like to evaluate the other client that I use. I use Newsboat, an actively maintained fork of the Newsbeuter client, for monitoring software (library) release feeds. Such feeds generally have low throughput as well as very simple content, so I do not need the efficiency that is central to my client requirements. How would it work for my personal feeds? Let’s see!

Impressions

I already have Newsboat installed, and I confirmed that I have the latest version:

$ newsboat --version | head -n 1
Newsboat 2.25.0 - https://newsboat.org/

My first attempt to import my OPML file failed:

$ newsboat --import-from-opml path/to/rss-20211105.opml
Error: failed to read URLs from file "/home/tcard/.newsboat/urls":
  Failed to open file: No such file or directory (os error 2)

The ~/.newsboat directory exists, but I removed my old urls file for this user. Perhaps the software expects the urls file to exist if the directory exists… I removed the ~/.newsboat directory but got the same error. It recreated the directory, so I created an empty urls file. The import succeeded!

$ rmdir ~/.newsboat/
$ newsboat --import-from-opml ~/tmp/xfer/rss-20211105.opml
Error: failed to read URLs from file "/home/tcard/.newsboat/urls":
  Failed to open file: No such file or directory (os error 2)
$ mkdir ~/.newsboat
mkdir: cannot create directory ‘/home/tcard/.newsboat’: File exists
$ touch ~/.newsboat/urls
$ newsboat --import-from-opml ~/tmp/xfer/rss-20211105.opml
Import of path/to/rss-20211105.opml finished.

One of the features that I miss the most in Newsboat is hierarchical organization, which is critical for efficiency in processing feeds. Checking the urls file after import, I see that the label for each feed reflects the hierarchy that I have configured in Thunderbird. I noticed that it uses a slash as a separator but does not escape slashes in labels (such as /r/haskell).

Loading the client, it does not look like the feed labels are used anyway. The feeds are displayed in the order specified in the urls file. It is pretty random after importing from my OPML file, but one can of course manually sort the feeds to a desired order. When reloading all feeds, the titles of the feeds are as specified in the RSS/Atom feeds are displayed. Even with a fixed order, it does not seem near as organized as a hierarchical organization provides.

Displaying titles as specified in the feeds is problematic because many feeds do not specify very good titles, making it difficult to know what a feed is from the title alone. For example, some feeds are provided as a feature of some web software, and the content creators probably do not even think about (or perhaps even know about) the RSS feeds when they create content. The RSS title may be set to some string that makes sense in context of the website but not in a list of many RSS feeds. For example, I have a feed with title 「お知らせ」 (“Notices” in English).

The first check (called a “reload” in Newsboat) was surprisingly slow. I hoped that subsequent checks would be fast, but they are also slow. While I have not had problems using Newsboat with a small number of release feeds, this performance alone is a show-stopper for use with my many personal feeds. Perhaps it would be acceptable if always keeping Newsboat running and periodically checking feeds so that one does not have to wait when processing feeds, but it does not seem that the software has support for periodic checks.

I see that Newsboat works fine with multiple languages. I notice that the Japanese is displayed in a simple/naïve way: lines are broken based on column without regard to content. Even basic 禁則処理 rules are not followed. This is not a problem, though, as I prefer to read Japanese in the browser anyway.

While I do not like the flat organization, it is of course easy to navigate using the keyboard since Newsboat is a TUI. Item links are displayed at the top and can be easily copied using the mouse. An item link can be opened in the browser using the o key, which is great! I wish that Newsboat used Vim key bindings for navigation, though. It would greatly improve efficiency to avoid the arrow keys, for example.

The flat organization results in a lot of scrolling. I hoped that I could quickly navigate to a specific feed using a search, but the search functionality searches item contents. I therefore have to manually search for the feed, which seems like an unnecessary waste of time. (It got so annoying that I started to reference the urls file, using Vim to search, in order to find feeds more quickly.)

HTML tables are rendered in an ASCII format:

Difference between LTS Haskell 18.14 and LTS Haskell 18.15
+-----------------------------+---------------+---------------+
|Package name                 |Old            |New            |
+-----------------------------+---------------+---------------+
|Allure                       |0.10.2.0 [1]   |0.10.3.0 [2]   |
+-----------------------------+---------------+---------------+

I wonder if it would be better without the horizontal lines in cases where no cells are long enough to display on two lines on the screen… It is definitely easier to scan such tables in Thunderbird.

Images are not displayed.

Links in item descriptions are displayed with an underline and are annotated with a natural number. Pressing the 1 key opens the link annotated with [1] in the browser. I do not know how to open links greater than [9], however. For example, attempting to open the link annotated with [11] opens the link annotated with [1] once or twice, depending on typing speed. This is not a big issue in cases where the item links to a webpage with the same content, as the link can be opened in the browser and viewed there. Some feeds do not have online versions of feed items, however, making this link issue very significant! I could not find any way to iterate over links in an item description (as is common in TUI web browsers). The link URLs are listed at the bottom of the item description, so one can at least manually find the corresponding URL and copy it using the mouse.

Podcast enclosures are displayed at the top of the item page as well as in the list of links at the bottom. You can copy the URL using the mouse.

Reading the documentation, I discovered that what looked like labels in the urls file are actually “tags.” Pressing the t key opens a list of tags, and selecting a tag displays only feeds that are associated with that tag. This can be used to organize feeds into categories. This would be useful when checking feeds for specific categories, but I do not find it very useful for my common use case. I generally process all feeds in a single session, and I use the organization to group the feeds according to topic in order to improve my efficiency in processing the feeds. If I were to use Newsboat for my personal feeds, I would manually order the feeds so that they are grouped by topic, so I would not use the tagging functionality very often.

I added the feed that Thunderbird refuses to load due to validation issues, and Newsboat handles it without any problems. It displays the author names, for example, even though author should be an email address according to the specifications. Nice!

When using Newsboat, items cannot be deleted. They are marked as read (or unread) only. Newsboat stores the state in a ~/.newsboat/cache.db SQLite file. I like the choice of format, and backup is very easy!

My Client Requirements

How does Newsboat measure up to my client requirements?

Newsboat meets my essential requirements, with the exception of the speed of checking feeds. Checking feeds is way too slow and would have a definite impact on my current workflow.

While the TUI is great for a small number of release feeds, it definitely slows me down when processing a large number of personal feeds. I am able to filter items much more effectively using a GUI that uses variable-width fonts with HTML formatting, displaying images as well as rendered tables. Links are particularly troublesome with Newsboat, aside from the first nine.

I really miss hierarchical organization, but I think that I could do without it in this case. Manually specifying the order of feeds is sufficient for my general use case, where I process all feeds, and the tagging functionality helps in special cases where I need to check a specific feed.

The feed view leaves much to be desired. Displaying titles as specified in the RSS/Atom feeds is problematic. It is possible configure the software to display the feed link instead, but I greatly prefer specifying my own titles (for folders in Thunderbird). Note that using the RSS link is not very helpful because may podcast providers use links with numeric identifiers.

While Newsboat can search item content, I really wish there was an easy way to search for feeds or item titles (not descriptions). I am less efficient at finding specific items (or even feeds for that matter!) using Newsboat than I am with Thunderbird.

Newsboat does not have support for periodic checking.

Overall, Newsboat is not a good match for my needs. It works fine for checking a small number of release feeds, but I find that it does not work well with a large number of feeds.