Async fn in trait... not

👋 This page was last updated ~2 years ago. Just so you know.

Async fn in trait… not

I was planning on showing the in-progress async_fn_in_trait feature in the context of my website, but it turns out, I can’t!

My website uses two databases: one local SQLite database for content, and a shared Postgres database for user credentials, preferences etc. Migrations are run on startup, and each migration implements one of the following traits:

#[async_trait::async_trait] pub trait MigrationSqlite { fn tag(&self) -> &'static str; async fn up(&self, db: &mut Transaction<Sqlite>) -> Result<()>; } #[async_trait::async_trait] pub trait MigrationPostgres { fn tag(&self) -> &'static str; async fn up(&self, db: &mut Transaction<Postgres>) -> Result<()>; }

…which can have async functions thanks to the async-trait crate.

Enabling the unstable feature works for the trait definition:

#![allow(incomplete_features)] #![feature(async_fn_in_trait)] pub trait MigrationPostgres { fn tag(&self) -> &'static str; async fn up(&self, db: &mut Transaction<Postgres>) -> Result<()>; }

…but fails later, because I’m making a Vec of migrations, to pass to:

pub async fn migrate_all_postgres( db: &PgPool, wanted_list: Vec<Box<dyn MigrationPostgres>>, ) -> Result<()> { // (cut) }

…and traits with async fns (the real thing) aren’t trait-object-safe for now:

$ cargo check Checking futile-db v0.1.0 (/home/amos/bearcove/futile/crates/futile-db) (cut) error[E0038]: the trait `MigrationPostgres` cannot be made into an object --> crates/futile-db/src/lib.rs:64:47 | 64 | async fn mk_postgres_pool(migrations: Vec<Box<dyn MigrationPostgres>>) -> Result<sqlx::PgPool> { | ^^^^^^^^^^^^^^^^^^^^^ `MigrationPostgres` cannot be made into an object | note: for a trait to be "object safe" it needs to allow building a vtable to allow the call to be resolvable dynamically; for more information visit <https://doc.rust-lang.org/reference/items/traits.html#object-safety> --> crates/futile-db/src/migrations/mod.rs:135:14 | 133 | pub trait MigrationPostgres { | ----------------- this trait cannot be made into an object... 134 | fn tag(&self) -> &'static str; 135 | async fn up(&self, db: &mut Transaction<Postgres>) -> Result<()>; | ^^ ...because method `up` references an `impl Trait` type in its return type = help: consider moving `up` to another trait error[E0038]: the trait `MigrationPostgres` cannot be made into an object --> crates/futile-db/src/migrations/mod.rs:71:26 | 71 | wanted_list: Vec<Box<dyn MigrationPostgres>>, | ^^^^^^^^^^^^^^^^^^^^^ `MigrationPostgres` cannot be made into an object | note: for a trait to be "object safe" it needs to allow building a vtable to allow the call to be resolvable dynamically; for more information visit <https://doc.rust-lang.org/reference/items/traits.html#object-safety> --> crates/futile-db/src/migrations/mod.rs:135:14 | 133 | pub trait MigrationPostgres { | ----------------- this trait cannot be made into an object... 134 | fn tag(&self) -> &'static str; 135 | async fn up(&self, db: &mut Transaction<Postgres>) -> Result<()>; | ^^ ...because method `up` references an `impl Trait` type in its return type = help: consider moving `up` to another trait (cut)

There is a very good reason for that limitation, and I hear, to lift it, something like dyn* (which is very hard to search for) is needed.

So, instead, I’ll show them what I did for hring, to get an “owned” version of the AsyncRead / AsyncWrite traits, that is io-uring-friendly:

pub trait ReadOwned { async fn read<B: IoBufMut>(&self, buf: B) -> BufResult<usize, B>; } pub trait WriteOwned { async fn write<B: IoBuf>(&self, buf: B) -> BufResult<usize, B>; async fn writev<B: IoBuf>(&self, list: Vec<B>) -> BufResult<usize, Vec<B>> { let mut out_list = Vec::with_capacity(list.len()); let mut list = list.into_iter(); let mut total = 0; while let Some(buf) = list.next() { let (res, buf) = self.write(buf).await; out_list.push(buf); match res { Ok(n) => total += n, Err(e) => { out_list.extend(list); return (Err(e), out_list); } } } (Ok(total), out_list) } async fn write_all<B: IoBuf>(&self, mut buf: B) -> BufResult<(), B> { let mut written = 0; let len = buf.bytes_init(); while written < len { let (res, slice) = self.write(buf.slice(written..len)).await; buf = slice.into_inner(); let n = match res { Ok(n) => n, Err(e) => return (Err(e), buf), }; written += n; } (Ok(()), buf) } } pub trait ReadWriteOwned: ReadOwned + WriteOwned {} impl<T> ReadWriteOwned for T where T: ReadOwned + WriteOwned {}

This all works flawlessly. Here’s the implementation for tokio-uring’s TcpStream type:

impl ReadOwned for TcpStream { async fn read<B: IoBufMut>(&self, buf: B) -> BufResult<usize, B> { TcpStream::read(self, buf).await } } impl WriteOwned for TcpStream { async fn write<B: IoBuf>(&self, buf: B) -> BufResult<usize, B> { TcpStream::write(self, buf).await } async fn writev<B: IoBuf>(&self, list: Vec<B>) -> BufResult<usize, Vec<B>> { TcpStream::writev(self, list).await } }

(It’s trivial, since the trait is closely modelled after tokio-uring’s own types).

Object trait safety is still an issue here, but we can work around it by making most functions take an &impl ReadOwned instead of a &dyn ReadOwned, which makes the function generic (which means it’ll be monomorphized, which in turns means the binary will be larger, but there won’t be dynamic dispatch at all):

/// Returns `None` on EOF, error if partially parsed message. pub(crate) async fn read_and_parse<Parser, Output>( parser: Parser, // 👇 stream: &impl ReadOwned, mut buf: RollMut, max_len: usize, ) -> eyre::Result<Option<(RollMut, Output)>> where Parser: Fn(Roll) -> IResult<Roll, Output>, { // cut }

The rest of the hring API is carefully designed to work around that limitation for now, which is… a bit of a balancing act, but it works!

Comment on /r/fasterthanlime

(JavaScript is required to see this. Or maybe my stuff broke)

Here's another article just for you:

Image decay as a service

Since I write a lot of articles about Rust, I tend to get a lot of questions about specific crates: “Amos, what do you think of oauth2-simd? Is it better than openid-sse4? I think the latter has a lot of boilerplate.”

And most of the time, I’m not sure what to responds. There’s a lot of crates out there. I could probably review one crate a day until I retire!