Hey everyone,
after almost a year of work I'm very excited to announce Irdest release
v0.6.0!
This is probably the biggest update to Irdest (and in particular the
router ratmand) since the beginning of this project. In this e-mail I
want to highlight some of the changes and outline the future of
development from here on out.
Architectural changes
---------------------
The biggest change in our architecture is the change from "discrete"
messages to streamed messages. Previously, when trying to send a 2GB
payload ratmand would load all of it into memory, blockify it, encrypt
it, slice it, then send it. This had the massive limitations that the
router had to hold the whole message state in memory, and any error that
occured during transmission lead to data loss. Not to mention that the
router could easily run out of memory and crash.
This has now been changed. Every operation in ratmand is a stream, no
more than a few dozen megabytes are stored in memory at a time, and
things are streamed to and from a database journal on disk. This _can_
currently become quite large (several GB), which is something we will be
improving in future. It also means that the router crashing (or being
killed by external forces) half-way through a send procedure now is a
recoverable scenario.
User/ developer facing changes
------------------------------
The client facing API has been completely overhauled, ditching protocol
buffers in favor of a very simple hand-rolled header-payload frame
format (called "microframe"). The Rust client library implementation
(libratman) handles encoding and decoding of all of these payloads, but
the format is simple enough that alternate implementations won't be
difficult to write for third-party developers. There's some
documentation on this format in the developer manual.
Additionally we have now split the CLI from a single tool into two:
ratcat, responsible for sending and accepting message streams, and
ratctl, responsible for managing addresses, subscriptions and streams,
accessing peer and system diagnostics, and in future managing contact
aliases and other subscriptions as well.
In summary
----------
Overall this is a very exciting release of Irdest. Not all components
have been updated to the new APIs and the main focus of development has
been on making ratmand, ratcat, and ratctl as stable and easy to use as
possible. If you're interested in a full breakdown of the changes you
can check out the changelog [1] for this release. And as always, you
can download the new version installer from our website [2]. It's
recommended to delete any existing state from ~/.config/ratman,
~/.config/ratcat, and ~/.local/ratman before installing this version.
While we can't promise stability guarantees for the on-disk state just
yet, this is a huge step forward to a stable release of Irdest and I'm
very excited to see how future development unfolds. I'm also excited to
see what people can and will do with this new version.
Cheerio,
~ Kookie
[1]: https://git.irde.st/we/irdest/-/blob/0.6.0/CHANGELOG.md
[2]: https://irde.st/downloads/
