News News feed (Atom)

This Week in Ruma

› published on by

From the editor

I forgot to provide an update last week, so this week we've got two weeks of stuff to cover. The revamp of ruma-events I mentioned several times previously was released as 0.13.0, followed shortly by an update to 0.14.0 to bump the url crate (which is a re-exported dependency) to version 2.0. When trying to integrate the new ruma-events into ruma-client-api, Jonas quickly discovered that it wasn't actually possible, because API requests/responses that contain events need to be deserialized, and can't because the events don't have a public implementation of serde::Deserialize anymore. I felt like an idiot for not foreseeing this seemingly obvious problem, but that's the way it goes sometimes. I'll be working on addressing this next week.

A new version of ruma-api, 0.10.0, was released this week. This release contains two significant changes: The first is that the Endpoint trait has been revised to be implemented in terms of the endpoint's request type, rather than having the request be specified via an associated type. This creates a more straightforward mapping between request and response types for each endpoint, removes a bunch of unnecessary unit structs that were only used to link associated request and response types together, and lets us remove some use of the turbofish operator in ruma-client. The second change in this release is that ruma-api-macros has been merged into the repo, and now exists in a Cargo workspace. The ruma_api procedural macro is now included and re-export from ruma-api itself under a default feature. The macro has also been updated to refer to its dependencies via a hidden module in ruma-api. The result is that downstream crates like ruma-client-api no longer have to specify dependnecies on all the crates referenced by the code generated by the macro.

Another exiciting update: This past week, both Jonas and I started live streaming our development on Twitch. I made a separate post on my own blog about this, but the short version is that I intend to live stream most of my work on Ruma from now on to help people learn about both Rust and Matrix. You can find my live streams on my Twitch channel, and you can watch the recordings of past streams on my YouTube channel. I'll announce my streams in #ruma:matrix.org as well as on my Twitter feed. Jonas's live streams can be watched on his Twitch channel.

In the last update, I mentioned that Ruma finally accepts donations via Liberapay. Thank you very much to the people who have already signed on to donate to the project! I thought it wouldn't hurt to mention again in case anyone missed it. :}

This Week in Ruma

› published on by

From the editor

This week I'd like to officially welcome Jonas Platte as a member of the Ruma team. Jonas has been involved in the project for a long time and has provided many significant contributions, both via code and via discussions about the project's design. Jonas is the original author of ruma-gtk, a graphical Matrix client built on ruma-client, which later became Fest, which in turn was forked to create Fractal. Thank you very much for all your work, Jonas!

A new addition to the website this week is the how to contribute page, which offers an entry point for anyone interested in contributing to the project's software. Thank you to everyone who has expressed interest in contributing in the past (and for everyone who will in the future!) Hopefully this guide will help you get started and give you some ideas for how to help.

On several occasions I've been asked if there is a way to donate financially to the project. I'm happy to report that Ruma now accepts donations via Liberapay. Donations to the project will be divided amongst team members, which as of today consists of only Jonas and me, but will hopefully grow over time. In the past I was hesitant to accept donations because I didn't want financial interest to affect (or have the appearance of affecting) my decisions about the project. I am building Ruma because I think the world needs Matrix, not as a way to support myself. Being able to share donations to Ruma with other developers makes me much more comfortable with accepting donations. It serves both to dilute my own financial interest in the project as well as to provide benefit in a way that's more in tune with the project's goals: making the world better for all of us. For anyone that decides to support the project on Liberapay, I offer my most sincere thanks.

As for the software itself, the main update this week is that the revamp of ruma-events I've been working on for the past month is complete and has been merged into the master branch. I still need to do one quick pass over everything to be sure I didn't make any obvious mistakes, but once that's done it will be ready for a new release. As part of this revamp, a new supporting crate, ruma-events-macros also had its first release this week. Like ruma-api-macros, it's really only used as an internal dependency for the project, and doesn't have much use to other developers directly. But it could be interesting to look at if you want an example of a real-world procedural macro.

There was also a new release of ruma-api, version 0.9.0, which revises the API to support the upcoming futures 0.3, ironically by removing the dependency on futures entirely. This release also removes the library's dependency on hyper, as it's always been a goal for the foundational Ruma libraries to allow other developers to build Matrix software with other HTTP libraries if they wish.

Matrix at large

Riot, the flagship Matrix client, released version 1.3 this week, which includes the ability to edit sent messages, and to add reactions to messages. Both of these features are highly requested and bring Riot in line with other high-profile collaboration software.

Other notable news is the announcement to get Dendrite, a Matrix homeserver written in Go, into a state where it can have some practical use for Matrix users. The first goal is for it to support enough functionality to be used for Matrix bots. Read more about this announcement on the lastest This Week in Matrix.

This Week in Ruma

› published on by

From the editor

In the last update, I mentioned that the ongoing ruma-events revamp was impeded by an issue with types from ring which don't implement Clone and PartialEq. It turns out that there is a reasonable explanation for this, and so it was decided to work around it. I restructured the ruma-signatures types in question so that they don't contain ring types, but simply construct them and use them as-needed in the relevant methods.

While I was working on ruma-signatures, I decided to fill in the missing functionality—signing and verifying events. In the process of doing that, I ended up with a significantly revised API for the crate, which has now been released as version 0.5.0. In a somewhat amusing development, the new ruma-signatures API doesn't include the Signatures and SignatureSet types that were causing the issue in ruma-events in the first place. As a result, ruma-events is not only unblocked, but doesn't have depend on ruma-signatures at all.

I made a little bit more progress on ruma-events. There is only one module remaining to be updated to the new API, so we're on the home stretch for a new ruma-events release.

This Week in Ruma

› published on by

From the editor

Work continues on the major revamp of ruma-events mentioned in the last update. Only a few modules remain to be converted to the new API. It's not hard work, but it's a bit tedious, so I've been dragging on getting it done. There are also a few modules that are somewhat blocked on an issue in ring. Some of the types in ruma-events contain types from ruma-signatures which don't implement Clone and PartialEq because they contain types from ring which don't. I want all event types in ruma-events to implement these traits. I say this issue is "somewhat blocked" because I could always just modify ruma-events-macros to derive these traits conditionally and then just leave them out for the few types in question, but this would be an unfortunate workaround, so I'd prefer to have the issue solved upstream. Unfortunately, the issue hasn't received a reply from any maintainers since I opened it a few weeks ago. No hard feelings—that's how open source works sometimes.

Rust at large

The big news in Rust since the last update is that Rust 1.36 was released, and it includes stabilization of the Future trait, one of the long-awaited building blocks for first-class async support in Rust. As readers probably already know, the biggest reason for Ruma's development hiatus is waiting for async networking in Rust to mature, and this is one of the final pieces of foundational support we've been waiting for. The remaining pieces are async/await syntax, which is expected in either the next version or the one following it, and finally, waiting for important libraries like Hyper and Tokio, as well as web frameworks, to adopt the new stuff. I also consider support for impl Trait in traits to be an important feature that is not yet supported, but at this point Ruma doesn't seem to need this, so while it's a major missing component of Rust's async support, it doesn't seem likely to block Ruma development.

This Week in Ruma

› published on by

From the editor

This week was spent working on a big revamp of ruma-events, the library that defines Rust types for the "events" used in Matrix. The previous week, I did a pass through the entire library to bring it up to date with version r0.5.0 of the Matrix specification.

After some discussion in #ruma:matrix.org, I decided to make a move towards treating ruma-events as a higher-level library. Previously, ruma-events has more or less offered Rust types that are exact representations of the JSON structures used by Matrix. However, by representing events this way, it would be possible for users to easily create values that, while valid JSON, would be invalid events according to the specification.

The way we're approaching this problem is by separating serialization/deserialization of JSON from validation of events. Rather than directly implementing serde::Deserialize, event types will be converted from a string of JSON data using std::str::FromStr. This implementation will deserialize the data internally and then validate it, returning either a valid event, or a raw serde_json::Value along with an error message about why the event was invalid. The latter form is necessary because invalid events will always be present in the Matrix system, created via bugs or other problems in homeserver implementations. Once an event is in the event graph, other servers will receive these invalid events over federation, and must still persist them and deal with them as they are.

Similar to the new interface around deserialization, the serialized form of an event created by ruma-events may not directly match the structure of the Rust type. ruma-events will instead expose variants of each event to ensure they can only be created in valid states.

To reduce the boilerplate that this approach necessarily causes (because for many events there will now be two structures: the public one and the private one used for converting to and from JSON), I started a new procedural macro, ruma-events-macros, to make the most common cases require minimal code in ruma-events. It's been a lot of work, and there's still probably another week or more to go to finish implementing ruma-events using the new approach. But when it's done, the result will be a Rust library for Matrix events that leaves less room for users to accidentally do the wrong thing, while also supporting interoperation with other Matrix software that isn't so strict.