Designing and implementing a safer API on top of LoadLibrary

Contents

• It's refactor time!

It's refactor time!

Our complete program is now about a hundred lines, counting blank lines (see the end of part 3 for a complete listing).

While this is pretty good for a zero-dependency project (save for pretty-hex), we can do better.

First off, concerns are mixed up. In the same file, we:

• Expose LoadLibraryA / GetProcAddress
• Expose the Win32 ICMP API
• Use both of the above to ping 8.8.8.8

Second, a lot of low-level details are exposed, that really shouldn't be. I don't think main() really ought to be using transmute at all. We should build safe abstractions above all of this.

Let's start with LoadLibraryA / GetProcAddress.

At the top of src/main.rs, we'll add:

mod loadlibrary

Then we'll start writing src/loadlibrary.rs.

We might want to start with our public interface. We want to design the loadlibrary API in such a way that:

• Users of the module cannot accidentally ignore errors
• Users don't need to use any ffi/C types
• Users don't have to worry about C-style null-terminated strings
• Users don't have to use transmute to get the procs back.

First off, since the Win32 API returns a Handle when calling LoadLibraryA, we probably want to export a Library type, that will represent an opened .dll file:

// in src/loadlibrary.rs
use std::ffi::c_void;

pub struct Library {
    handle: *const c_void;
}

That's a good start!

Users of that module will see a Library type, but they won't be able to read or write to the private handle field - we're hiding implementation details.

However, nothing stops us from using convenience types internally, so let's define HModule once again. We want that one to stay private:

use std::ffi::c_void;

type HModule = *const c_void;

pub struct Library {
    handle: HModule;
}

But, we can go one step further. In our API, we only want folks to be able to hold an instance of Library if they've successfully opened a DLL.

And if they've successfully opened a DLL, the handle is not null. But currently, there's nothing in the definition of Library that really communicates that. An HModule is just a raw pointer (*const c_void), and raw pointers may be null, they're some of the least-checked types in Rust.

Luckily, Rust has a generic NonNull type, which we can use here:

use std::{
    ffi::c_void,
    ptr::NonNull,
}

type HModule = NonNull<c_void>;

pub struct Library {
    handle: HModule;
}

Thanks to this change, we can be sure that whenever we hold an instance of HModule, it really is a non-null pointer - as long as we get our FFI function signatures right - so we should be consistent with our extern "C" blocks.

Next bit, we want to be able to create a new Library, by opening a DLL. We could debate what's the proper name all day, but since it's a function that returns Self, and it's the only one, I'm going to go with the classic new:

// impl blocks don't need to be public
impl Library {
    // but fns do (they can also be private)
    pub fn new(name: &str) -> Self {
        unimplemented!()
    }
}

Good! Good.

We're taking a Rust string here, so we're satisfying the "API users shouldn't know/care about C-style strings" constraint.

We're also busy designing the public API right now, so we won't implement new just yet, instead using the unimplemented! macro, which will let our code compile. If/when Library::new() is actually called, then our program will panic with file & line information, letting us know that, woops, we forgot to implement something.

Note that new() does not take &self, it's a "static method", called like this:

fn main() {
    let l = Library::new("USER32.dll");
}

We did forget one thing though - right now, Library::new cannot fail gracefully, because its return type is Self. Our only way to do error handling with such a function signature is either to panic (which is bad, since loadlibrary is a library, and panic is a tool better left to application code), or to return an invalid instance of Library, for example one with a null handle - which we just said we don't want to do.

Typically, for a fallible operation, we'd return a Result<T, E>, where T is the return type if everything went fine, and E is the error type if something went wrong. Something like this:

pub enum Error {
    LibraryNotFound
}

impl Library {
    pub fn new(name: &str) -> Result<Self, Error> {
        unimplemented!()
    }
}

That said, for this article, we're going to assume that there's actually only two states: either the library is there/available/successfully opened, or it isn't.

In other words, we either have something, or we have nothing. And for that, we don't really need a Result<T, E> - we just need an Option<T>.

impl Library {
    pub fn new(name: &str) -> Option<Self> {
        unimplemented!()
    }
}

Good, good. We're making tremendous amounts of progress, although nothing is implemented yet. API design is really important, especially when you have a powerful type system like Rust's at your disposal.

Now, once you have an instance of Library (which must mean you've opened a DLL successfully), you may want to look up a symbol. The underlying Win32 call is GetProcAddress, so we'll take inspiration from that:

impl Library {
    // after new

    pub unsafe fn get_proc(&self, name: &str) -> Proc {
        unimplemented!()
    }
}

So far so good - this one takes &self, because it requires a valid instance of Library, and, just like new, it takes a name, which is a Rust string - again, we don't want users to worry about C strings.

Using loadlibrary might now look like this:

// only one type to import!
use loadlibrary::Library;

fn main() {
    let l = Library::new("USER32.dll");
    let msgbox = l.get_proc("MessageBoxA");
}

Hang on, that's not quite right - Library::new does not return a Library, it returns an Option<Library> - and there's no get_proc method on Option<Library>. If we try to compile, rustc rightfully complains:

As you may remember, one of our constraints was to force users to deal with errors, and that's exactly what's happening here. To get an instance of Library, folks have a lot of, uh, options to choose from.

They could use a match:

match Library::new("USER32.dll") {
    Some(l) => {
        // do something with l, a valid library
    },
    None => panic!("Could not open USER32.dll"),
}

They could use a match in a way that doesn't make the happy path have two additional levels of indentation, because match is an expression, and panic! can be used in any "arm" of the match, because the compiler knows it has flow control implications:

let l = match Library::new("USER32.dll") {
    Some(l) => l,
    None => panic!("Could not open USER32.dll"),
};

// do something with l, a valid library

They could also use an if let:

if let Some(l) = match Library::new("USER32.dll") {
    // do something with l
}

They could also use Option::expect, which panics with a custom error message on None, and unwraps on Some:

let l = Library::new("USER32.dll").expect("could not open USER32.dll");
// do something with l

Or they could be somewhat in a hurry and just use Option::unwrap, which is like expect but without the custom error message:

let l = Library::new("USER32.dll").unwrap();
// do something with l

Now, .unwrap() (along with the ? sigil) confused me when I first started reading Rust code, but all you really need to do to understand them is to follow the types!

I'll emphasize this by spelling out the types instead of relying on type inference:

let maybe_l: Option<Library> = Library::new("USER32.dll");

// at this point in the program, we don't know what's in
// maybe_l - maybe it's Some, maybe it's None.
// `Library::new()` is done executing, but we haven't dealt
// with its consequences yet.

let l: Library = maybe_l.unwrap();

// we only reach this point if `maybe_l` was `Some` - otherwise,
// we've panicked and execution has most probably stopped.

All clear? Good!

Time to get back to our Library type, in particular, our get_proc signature:

impl Library {
    pub unsafe fn get_proc(&self, name: &str) -> Proc {
        unimplemented!()
    }
}

First off, let's remember that get_proc can also fail - it can also return something or nothing, for example if we ask to look up FooBar in USER32.dll, it would definitely return nothing. So it needs to return an Option as well:

impl Library {
    pub unsafe fn get_proc(&self, name: &str) -> Option<Proc> {
        unimplemented!()
    }
}

Second, we've used a type named Proc here but, err, we haven't defined it. The problem we have is that there's no way to tell what the return type of Library::get_proc should be because:

• It's a foreign function
• The users of the loadlibrary API should specify what type they want

They might use get_proc with a different type each time:

let l = Library::new("USER32.dll");
let MessageBoxA: extern "stdcall" fn(Hwnd, *const u8, *const u8, u32) = l.get_proc("MessageBoxA");
let ShowWindow: extern "stdcall" fn(Hwnd, u32) = l.get_proc("ShowWindow");

So we can't define Proc, because, in the context of loadlibrary, we don't know what it is yet. We need to use a generic type parameter:

impl Library {
    pub unsafe fn get_proc<T>(&self, name: &str) -> Option<T> {
        unimplemented!()
    }
}

That should do it! Rust's type inference is powerful enough that we can have generic return types - it'll use the outer context to decide what T should be, and in case there's not enough information, it'll error out. For example, this won't work:

fn main {
    let l = Library::new("USER32.dll").unwrap();
    let p = unsafe { l.get_proc("ShowWindow").unwrap() };
    p();
}

And that's our full public interface! If we generate docs for the loadlibrary module, we see this:

And if we click on Library, we see this:

Now, to finish loadlibrary we're going to need a few more types, but they'll all be private. This (the screenshot above) is the full extent of what we want users to know about our library.

Of course, we should be documenting it a little more - a module-level doc comment could orient users towards the Library struct. We could document our assumptions - that holding an instance of Library means it has been successfully opened. We could warn users that the library only works on Windows. And we could justify our choices, like using Option instead of Result, and explain that we couldn't be bothered to call the Win32 API GetLastError for proper error handling.

But there's a lot to proper documentation, and that seems like a good candidate for another article.

Now, so far, both Library::new() and Library::get_proc() are stubs, because their body only contains unimplemented!(). If we use them in a program, we get something like this:

So, let's get implementing!

We know Library::new will map to LoadLibraryA, so let's declare the latter, as we did before:

use std::ffi::c_void;

extern "stdcall" {
    fn LoadLibraryA(name: *const u8) -> *const c_void;
}

Now (assuming our program links against KERNEL32.dll - which we're pretty sure it does) we should be able to call it from Library::new:

impl Library {
    pub fn new(name: &str) -> Option<Self> {
        let module = LoadLibraryA(name);

        unimplemented!()
    }

We're still using unimplemented!() because we don't know yet what the rest of the function will be, so it gets the compiler to stop complaining that we don't, currently, return an Option<Self>.

Hold up though - this doesn't compile:

Ah, of course - lucky we got LoadLibraryA's function signature right, the type checker caught this little slip-up.

So, can we turn an &str into a *const u8? Let's check the docs:

Sure we can! Should we?

No!

No no no. Step away from the docs.

C strings aren't just pointers to some bytes, they're pointers to a byte sequence that eventually ends with a null byte, otherwise it'll just keep reading and reading until it finds a null byte by accident, or reaches an invalid memory address. We found that out the hard way in part 2.

So, uh, we could always add the null byte ourselves...

impl Library {
    pub fn new(name: &str) -> Option<Self> {
        // note: here, we don't use type annotations for `c_name`
        // but it is clear from the next line that it is unambiguously
        // a `Vec<u8>`.
        let mut c_name = Vec::new();
        c_name.extend_from_slice(name.as_bytes());
        c_name.push(0x0);

        let module = LoadLibraryA(c_name.as_ptr());

        unimplemented!();
    }
}

And it would work! (I just checked).

But this is a pretty common operation when interacting with foreign code, and the Rust standard library has just the thing, in the ffi package, as expected:

Why do we want CString, and not CStr? Because:

Some C strings are valid Rust strings, but the reverse is never true.

Why is that? Well, let's say we have a C string, that we got via FFI somehow:

fn get_c_str() -> *const u8 {
    unimplemented!()
}

let c_str: *const u8 = get_c_string();

We could implement our own strlen() by reading bytes from it until we find, well, zero:

let length = {
    let mut i = 0;
    loop {
        // nota bene: pointer arithmetic is unsafe, and we
        // need to confess:
        let b = unsafe { *c_str.offset(i) };
        if b == 0 {
            // Hello$
            // ^^^^^^
            // 012345 <- length = 5
            break i;
        }
        i += 1;
    }
}

From there, we could make a byte slice:

let u8_slice = unsafe {
    std::slice::from_raw_parts(c_str, length as usize)
};

And, finally a &str:

let rust_string = std::str::from_utf8(u8_slice).expect("valid utf-8");

Note that std::str::from_utf8 returns a Result<T, E> - that part can fail, if the input is not valid utf-8. That's why only some C strings are valid Rust strings.

Note also that this code is, like, hella unsafe. rust_string refers to the same block of memory that c_str does, but the borrow checker does not know that.

If we were to free the block of memory c_str points to, our &str would become invalid. Here's how we could do it:

fn main() {
    let bytes = Box::new("Hello\0".to_owned());

    let c_str: *const u8 = bytes.as_ptr();
    let length = {
        let mut i = 0;
        loop {
            let b = unsafe { *c_str.offset(i) };
            if b == 0 {
                break i;
            }
            i += 1;
        }
    };
    let u8_slice = unsafe { std::slice::from_raw_parts(c_str, length as usize) };
    let rust_string = std::str::from_utf8(u8_slice).unwrap();

    println!("before drop = {:#?}", rust_string);
    drop(bytes);
    println!(" after drop = {:#?}", rust_string);
}

And here's what it does:

But that's to be expected! We've played fast and loose with raw pointers, and that's what happens when you're not careful with unsafe code.

So that was turning a C string into a (borrowed) Rust string - it can work, if it's valid UTF-8 (all Rust strings are valid UTF-8).

For the reverse - we already did it - we allocated a Vec<u8>, copied the contents of the Rust string, then added a null byte. Didn't I say it was impossible? Not quite: I said Rust strings can't be C strings. But they sure can be converted - and it involves allocating memory, and someone must own that memory, so we want CString, the owned variant of the CStr/CString pair.

Phew.

Let's get back to Library::new.

CString is relatively straightforward to use, but its as_ptr method returns *const c_char, which is *const i8, not *const u8, so we'll need to change LoadLibraryA's prototype:

use std::os::raw::c_char;

extern "stdcall" {
    fn LoadLibraryA(name: *const c_char) -> *const c_void;
}

Then, implement new(), finally:

impl Library {
    pub fn new(name: &str) -> Option<Self> {
        // note: this will panic if `name` contains null bytes.
        // let's just hop nobody tries `Library::new("AhAh\0GotYouGood")`
        let name = CString::new(name).expect("invalid library name");

        let module = LoadLibraryA(c_name.as_ptr());
        Some(Library { module })
    }
}

Wait, slow down, that's unsafe:

Fair enough:

impl Library {
    pub fn new(name: &str) -> Option<Self> {
        let name = CString::new(name).expect("invalid library name");

        let module = unsafe { LoadLibraryA(name.as_ptr()) }; // 🦀🦀🦀
        Some(Library { module })
    }
}

Does this work?

It doesn't! And we can thank our past self for properly designing the Library struct.

We can make it compile by changing LoadLibraryA's signature to:

extern "stdcall" {
    fn LoadLibraryA(name: *const c_char) -> HModule; // 💀💀💀
}

...but this is wrong, because the actual LoadLibraryA can return a null pointer!

Instead, we can use the type system to encode the various cases possible:

• LoadLibraryA returns a null value
• LoadLibraryA returns a valid, non-null HModule

Remember in part 3, how we used Option<&IpOptionInformation>? We can do the same here:

extern "stdcall" {
    fn LoadLibraryA(name: *const c_char) -> Option<HModule>;
}

After doing this change, we get another compile error:

Again, we're forced to agree with rustc: those aren't the same type!

We have a few options: either do explicit pattern matching:

    pub fn new(name: &str) -> Option<Self> {
        let name = CString::new(name).expect("invalid library name");

        let module = unsafe { LoadLibraryA(name.as_ptr()) };
        match module {
            Some(module) => Some(Library { module }),
            None => None,
        }
    }

...but that's a bit redundant.

Or, we could use Option::map - which lets us transform an Option<T> into an Option<U> (In this case, T is HModule, and U is Library).

    pub fn new(name: &str) -> Option<Self> {
        let name = CString::new(name).expect("invalid library name");
        let res = unsafe { LoadLibraryA(name.as_ptr()) };
        res.map(|module| Library { module })
    }

Boom, three-liner, and it's relatively readable. Now we're cooking!

Let's move on to get_proc, applying everything we've learned so far.

We're going to need GetProcAddress. Previously, we defined it like this:

type FarProc = *const c_void;

extern "stdcall" {
    fn GetProcAddress(module: HModule, name: *const u8) -> FarProc;
}

...but we can do better.

Firstly, we want to prevent calling GetProcAddress with a null HModule. That's already taken care of, because the HModule type is already a NonNull<T>.

Secondly, we'll also be using CString here, so we don't want *const u8:

// changed: u8 => c_char
fn GetProcAddress(module: HModule, name: *const c_char) -> FarProc;

Thirdly, GetProcAddress can return null, if there is no such procedure in the DLL, so, again, let's encode that:

// added: Option<T>
fn GetProcAddress(module: HModule, name: *const c_char) -> Option<FarProc>;

Back when we were designing our API, we've already decided the signature of get_proc:

impl Library {
    pub unsafe fn get_proc<T>(&self, name: &str) -> Option<T> {
        unimplemented!()
    }
}

Now all that's left is to.. implement it. First we'll need a CString:

    pub unsafe fn get_proc<T>(&self, name: &str) -> Option<T> {
        let name = CString::new(name).expect("invalid proc name");

        unimplemented!()
    }

Then call the actual Win32 API:

    pub unsafe fn get_proc<T>(&self, name: &str) -> Option<T> {
        let name = CString::new(name).expect("invalid proc name");
        let res = GetProcAddress(self.module, name.as_ptr());

        unimplemented!()
    }

Note that passing self.module to GetProcAddress works out beautifully, because they're both NonNull<HModule> already.

And finally, use Option::map, just like in new:

    pub unsafe fn get_proc<T>(&self, name: &str) -> Option<T> {
        let name = CString::new(name).expect("invalid proc name");
        let res = GetProcAddress(self.module, name.as_ptr());
        res.map(|proc| transmute_copy(&proc))
    }

What's transmute_copy? It's transmute's evil little brother. And I couldn't make it work with regular transmute, because of the generic type parameter. If you can figure out, you'll definitely be credited in this article.

Here's our complete src/loadlibrary.rs so far:

use std::{
    ffi::{c_void, CString},
    mem::transmute_copy,
    os::raw::c_char,
    ptr::NonNull,
};

type HModule = NonNull<c_void>;
type FarProc = NonNull<c_void>;

extern "stdcall" {
    fn LoadLibraryA(name: *const c_char) -> Option<HModule>;
    fn GetProcAddress(module: HModule, name: *const c_char) -> Option<FarProc>;
}

pub struct Library {
    module: HModule,
}

impl Library {
    pub fn new(name: &str) -> Option<Self> {
        let name = CString::new(name).expect("invalid library name");
        let res = unsafe { LoadLibraryA(name.as_ptr()) };
        res.map(|module| Library { module })
    }

    pub unsafe fn get_proc<T>(&self, name: &str) -> Option<T> {
        let name = CString::new(name).expect("invalid proc name");
        let res = GetProcAddress(self.module, name.as_ptr());
        res.map(|proc| transmute_copy(&proc))
    }
}

It's time to change src/main.rs's main() function to use our new wrapper.

Before, we had:

unsafe {
    let h = LoadLibraryA("IPHLPAPI.dll\0".as_ptr());
    let IcmpCreateFile: IcmpCreateFile =
        transmute(GetProcAddress(h, "IcmpCreateFile\0".as_ptr()));
    let IcmpSendEcho: IcmpSendEcho = transmute(GetProcAddress(h, "IcmpSendEcho\0".as_ptr()));
}

Now, we have:

let iphlp = loadlibrary::Library::new("IPHLPAPI.dll").unwrap();
let IcmpCreateFile: IcmpCreateFile = unsafe { iphlp.get_proc("IcmpCreateFile").unwrap() };
let IcmpSendEcho: IcmpSendEcho = unsafe { iphlp.get_proc("IcmpSendEcho").unwrap() };

And it still works!

Special thanks to @porglezomp for pointing out mistakes in the way I was using NonNull<T> and suggesting a fix!