Designing and implementing a safer API on top of LoadLibrary
👋 This page was last updated ~5 years ago. Just so you know.
It's refactor time!
Our complete program is now about a hundred lines, counting blank lines (see the end of part 3 for a complete listing).
While this is pretty good for a zero-dependency project (save for
pretty-hex
), we can do better.
First off, concerns are mixed up. In the same file, we:
- Expose
LoadLibraryA
/GetProcAddress
- Expose the Win32 ICMP API
- Use both of the above to ping 8.8.8.8
Second, a lot of low-level details are exposed, that really
shouldn't be. I don't think main()
really ought to be using
transmute
at all. We should build safe abstractions above
all of this.
Let's start with LoadLibraryA
/ GetProcAddress
.
At the top of src/main.rs
, we'll add:
mod loadlibrary
Then we'll start writing src/loadlibrary.rs
.
We might want to start with our public interface. We want to
design the loadlibrary
API in such a way that:
- Users of the module cannot accidentally ignore errors
- Users don't need to use any ffi/C types
- Users don't have to worry about C-style null-terminated strings
- Users don't have to use transmute to get the procs back.
First off, since the Win32 API returns a Handle
when calling
LoadLibraryA
, we probably want to export a Library
type, that
will represent an opened .dll file:
// in src/loadlibrary.rs use std::ffi::c_void; pub struct Library { handle: *const c_void; }
That's a good start!
Users of that module will see a Library
type, but they won't be able to
read or write to the private handle
field - we're hiding implementation
details.
However, nothing stops us from using convenience types internally, so
let's define HModule
once again. We want that one to stay private:
use std::ffi::c_void; type HModule = *const c_void; pub struct Library { handle: HModule; }
But, we can go one step further. In our API, we only want folks to be
able to hold an instance of Library
if they've successfully opened
a DLL.
And if they've successfully opened a DLL, the handle is not null. But
currently, there's nothing in the definition of Library
that really
communicates that. An HModule
is just a raw pointer (*const c_void
),
and raw pointers may be null, they're some of the least-checked types
in Rust.
Luckily, Rust has a generic NonNull
type, which we can use here:
use std::{ ffi::c_void, ptr::NonNull, } type HModule = NonNull<c_void>; pub struct Library { handle: HModule; }
Thanks to this change, we can be sure that whenever we hold an instance
of HModule
, it really is a non-null pointer - as long as we get our
FFI function signatures right - so we should be consistent with our
extern "C"
blocks.
Next bit, we want to be able to create a new Library
, by opening
a DLL. We could debate what's the proper name all day, but since
it's a function that returns Self
, and it's the only one, I'm going
to go with the classic new
:
// impl blocks don't need to be public impl Library { // but fns do (they can also be private) pub fn new(name: &str) -> Self { unimplemented!() } }
Good! Good.
We're taking a Rust string here, so we're satisfying the "API users shouldn't know/care about C-style strings" constraint.
We're also busy designing the public API right now, so we won't implement
new
just yet, instead using the unimplemented!
macro, which will let our
code compile. If/when Library::new()
is actually called, then our program
will panic with file & line information, letting us know that, woops, we
forgot to implement something.
Note that new()
does not take &self
, it's a "static method", called like this:
fn main() { let l = Library::new("USER32.dll"); }
We did forget one thing though - right now, Library::new
cannot fail gracefully,
because its return type is Self
. Our only way to do error handling with such
a function signature is either to panic (which is bad, since loadlibrary is a
library, and panic is a tool better left to application code), or to return an invalid
instance of Library
, for example one with a null handle - which we just said we
don't want to do.
Typically, for a fallible operation, we'd return a Result<T, E>
, where T
is
the return type if everything went fine, and E
is the error type if something
went wrong. Something like this:
pub enum Error { LibraryNotFound } impl Library { pub fn new(name: &str) -> Result<Self, Error> { unimplemented!() } }
That said, for this article, we're going to assume that there's actually only two states: either the library is there/available/successfully opened, or it isn't.
In other words, we either have something, or we have nothing. And for that,
we don't really need a Result<T, E>
- we just need an Option<T>
.
impl Library { pub fn new(name: &str) -> Option<Self> { unimplemented!() } }
Good, good. We're making tremendous amounts of progress, although nothing is implemented yet. API design is really important, especially when you have a powerful type system like Rust's at your disposal.
Now, once you have an instance of Library
(which must mean you've opened
a DLL successfully), you may want to look up a symbol. The underlying
Win32 call is GetProcAddress
, so we'll take inspiration from that:
impl Library { // after new pub unsafe fn get_proc(&self, name: &str) -> Proc { unimplemented!() } }
So far so good - this one takes &self
, because it requires a valid instance
of Library, and, just like new
, it takes a name, which is a Rust string - again,
we don't want users to worry about C strings.
Using loadlibrary
might now look like this:
// only one type to import! use loadlibrary::Library; fn main() { let l = Library::new("USER32.dll"); let msgbox = l.get_proc("MessageBoxA"); }
Hang on, that's not quite right - Library::new
does not return a
Library
, it returns an Option<Library>
- and there's no get_proc
method on Option<Library>
. If we try to compile, rustc rightfully complains:
As you may remember, one of our constraints was to force users to deal with
errors, and that's exactly what's happening here. To get an instance of Library
,
folks have a lot of, uh, options to choose from.
They could use a match
:
match Library::new("USER32.dll") { Some(l) => { // do something with l, a valid library }, None => panic!("Could not open USER32.dll"), }
They could use a match
in a way that doesn't make the happy path
have two additional levels of indentation, because match
is an expression,
and panic!
can be used in any "arm" of the match, because the compiler knows
it has flow control implications:
let l = match Library::new("USER32.dll") { Some(l) => l, None => panic!("Could not open USER32.dll"), }; // do something with l, a valid library
They could also use an if let
:
if let Some(l) = match Library::new("USER32.dll") { // do something with l }
They could also use Option::expect
, which panics with a custom error
message on None
, and unwraps on Some
:
let l = Library::new("USER32.dll").expect("could not open USER32.dll"); // do something with l
Or they could be somewhat in a hurry and just use Option::unwrap
, which is
like expect
but without the custom error message:
let l = Library::new("USER32.dll").unwrap(); // do something with l
Now, .unwrap()
(along with the ?
sigil) confused me when I first started
reading Rust code, but all you really need to do to understand them is to
follow the types!
I'll emphasize this by spelling out the types instead of relying on type inference:
let maybe_l: Option<Library> = Library::new("USER32.dll"); // at this point in the program, we don't know what's in // maybe_l - maybe it's Some, maybe it's None. // `Library::new()` is done executing, but we haven't dealt // with its consequences yet. let l: Library = maybe_l.unwrap(); // we only reach this point if `maybe_l` was `Some` - otherwise, // we've panicked and execution has most probably stopped.
All clear? Good!
Time to get back to our Library
type, in particular, our get_proc
signature:
impl Library { pub unsafe fn get_proc(&self, name: &str) -> Proc { unimplemented!() } }
First off, let's remember that get_proc
can also fail - it can
also return something or nothing, for example if we ask to look up
FooBar
in USER32.dll
, it would definitely return nothing. So
it needs to return an Option
as well:
impl Library { pub unsafe fn get_proc(&self, name: &str) -> Option<Proc> { unimplemented!() } }
Second, we've used a type named Proc
here but, err, we haven't defined it. The problem
we have is that there's no way to tell what the return type of Library::get_proc
should be because:
- It's a foreign function
- The users of the
loadlibrary
API should specify what type they want
They might use get_proc
with a different type each time:
let l = Library::new("USER32.dll"); let MessageBoxA: extern "stdcall" fn(Hwnd, *const u8, *const u8, u32) = l.get_proc("MessageBoxA"); let ShowWindow: extern "stdcall" fn(Hwnd, u32) = l.get_proc("ShowWindow");
So we can't define Proc
, because, in the context of loadlibrary
, we don't
know what it is yet. We need to use a generic type parameter:
impl Library { pub unsafe fn get_proc<T>(&self, name: &str) -> Option<T> { unimplemented!() } }
That should do it! Rust's type inference is powerful enough that we can have generic return types - it'll use the outer context to decide what T should be, and in case there's not enough information, it'll error out. For example, this won't work:
fn main { let l = Library::new("USER32.dll").unwrap(); let p = unsafe { l.get_proc("ShowWindow").unwrap() }; p(); }
Provided you already know about generics, this is a great error message.
And that's our full public interface! If we generate docs for the
loadlibrary
module, we see this:
And if we click on Library
, we see this:
Now, to finish loadlibrary
we're going to need a few more types,
but they'll all be private. This (the screenshot above) is the full extent
of what we want users to know about our library.
Of course, we should be documenting it a little more - a module-level
doc comment could orient users towards the Library
struct. We could
document our assumptions - that holding an instance of Library
means
it has been successfully opened. We could warn users that the library
only works on Windows. And we could justify our choices, like using Option
instead of Result
, and explain that we couldn't be bothered to call
the Win32 API GetLastError
for proper error handling.
But there's a lot to proper documentation, and that seems like a good candidate for another article.
Now, so far, both Library::new()
and Library::get_proc()
are stubs,
because their body only contains unimplemented!()
. If we use them
in a program, we get something like this:
So, let's get implementing!
We know Library::new
will map to LoadLibraryA
, so let's declare
the latter, as we did before:
use std::ffi::c_void; extern "stdcall" { fn LoadLibraryA(name: *const u8) -> *const c_void; }
Now (assuming our program links against KERNEL32.dll
- which we're pretty sure
it does) we should be able to call it from Library::new
:
impl Library { pub fn new(name: &str) -> Option<Self> { let module = LoadLibraryA(name); unimplemented!() }
We're still using unimplemented!()
because we don't know yet what the rest
of the function will be, so it gets the compiler to stop complaining that we don't,
currently, return an Option<Self>
.
Use unimplemented!()
liberally!
It's easy to search for in codebases (ie. it's easy to find out which parts of the code you haven't implemented yet) and very much "crash as early and as noisily as possible", which is great.
Think of it as a runtime-enforced // TODO:
.
Hold up though - this doesn't compile:
Ah, of course - lucky we got LoadLibraryA
's function signature right, the
type checker caught this little slip-up.
So, can we turn an &str
into a *const u8
? Let's check the docs:
Sure we can! Should we?
No!
No no no. Step away from the docs.
C strings aren't just pointers to some bytes, they're pointers to a byte sequence that eventually ends with a null byte, otherwise it'll just keep reading and reading until it finds a null byte by accident, or reaches an invalid memory address. We found that out the hard way in part 2.
So, uh, we could always add the null byte ourselves...
impl Library { pub fn new(name: &str) -> Option<Self> { // note: here, we don't use type annotations for `c_name` // but it is clear from the next line that it is unambiguously // a `Vec<u8>`. let mut c_name = Vec::new(); c_name.extend_from_slice(name.as_bytes()); c_name.push(0x0); let module = LoadLibraryA(c_name.as_ptr()); unimplemented!(); } }
And it would work! (I just checked).
But this is a pretty common operation when interacting with foreign code, and the Rust standard library has just the thing, in the ffi package, as expected:
Why do we want CString
, and not CStr
? Because:
Some C strings are valid Rust strings, but the reverse is never true.
Why is that? Well, let's say we have a C string, that we got via FFI somehow:
fn get_c_str() -> *const u8 { unimplemented!() } let c_str: *const u8 = get_c_string();
We could implement our own strlen()
by reading bytes from it until we find,
well, zero:
let length = { let mut i = 0; loop { // nota bene: pointer arithmetic is unsafe, and we // need to confess: let b = unsafe { *c_str.offset(i) }; if b == 0 { // Hello$ // ^^^^^^ // 012345 <- length = 5 break i; } i += 1; } }
Note: we could have used the for i in 0..
(a for loop iterating
through a RangeFrom),
but then we wouldn't be able to break with a value (ie. break i
);
From there, we could make a byte slice:
let u8_slice = unsafe { std::slice::from_raw_parts(c_str, length as usize) };
And, finally a &str
:
let rust_string = std::str::from_utf8(u8_slice).expect("valid utf-8");
Note that std::str::from_utf8
returns a Result<T, E>
- that part can fail,
if the input is not valid utf-8. That's why only some C strings are valid Rust
strings.
Note also that this code is, like, hella unsafe. rust_string
refers to the
same block of memory that c_str
does, but the borrow checker does not know
that.
If we were to free the block of memory c_str
points to, our &str
would become
invalid. Here's how we could do it:
fn main() { let bytes = Box::new("Hello\0".to_owned()); let c_str: *const u8 = bytes.as_ptr(); let length = { let mut i = 0; loop { let b = unsafe { *c_str.offset(i) }; if b == 0 { break i; } i += 1; } }; let u8_slice = unsafe { std::slice::from_raw_parts(c_str, length as usize) }; let rust_string = std::str::from_utf8(u8_slice).unwrap(); println!("before drop = {:#?}", rust_string); drop(bytes); println!(" after drop = {:#?}", rust_string); }
And here's what it does:
But that's to be expected! We've played fast and loose with raw pointers, and
that's what happens when you're not careful with unsafe
code.
So that was turning a C string into a (borrowed) Rust string - it can work, if it's valid UTF-8 (all Rust strings are valid UTF-8).
For the reverse - we already did it - we allocated a Vec<u8>
, copied the contents
of the Rust string, then added a null byte. Didn't I say it was impossible? Not quite:
I said Rust strings can't be C strings. But they sure can be converted - and it
involves allocating memory, and someone must own that memory, so we want CString
,
the owned variant of the CStr/CString
pair.
Phew.
Let's get back to Library::new
.
CString
is relatively straightforward to use, but its as_ptr
method
returns *const c_char
, which is *const i8
, not *const u8
, so we'll need
to change LoadLibraryA
's prototype:
use std::os::raw::c_char; extern "stdcall" { fn LoadLibraryA(name: *const c_char) -> *const c_void; }
Then, implement new()
, finally:
impl Library { pub fn new(name: &str) -> Option<Self> { // note: this will panic if `name` contains null bytes. // let's just hop nobody tries `Library::new("AhAh\0GotYouGood")` let name = CString::new(name).expect("invalid library name"); let module = LoadLibraryA(c_name.as_ptr()); Some(Library { module }) } }
Wait, slow down, that's unsafe:
Fair enough:
impl Library { pub fn new(name: &str) -> Option<Self> { let name = CString::new(name).expect("invalid library name"); let module = unsafe { LoadLibraryA(name.as_ptr()) }; // 🦀🦀🦀 Some(Library { module }) } }
Does this work?
It doesn't! And we can thank our past self for properly designing the Library
struct.
We can make it compile by changing LoadLibraryA
's signature to:
extern "stdcall" { fn LoadLibraryA(name: *const c_char) -> HModule; // 💀💀💀 }
...but this is wrong, because the actual LoadLibraryA can return a null pointer!
Instead, we can use the type system to encode the various cases possible:
- LoadLibraryA returns a null value
- LoadLibraryA returns a valid, non-null HModule
Remember in part 3, how we used Option<&IpOptionInformation>
? We can
do the same here:
extern "stdcall" { fn LoadLibraryA(name: *const c_char) -> Option<HModule>; }
After doing this change, we get another compile error:
Again, we're forced to agree with rustc: those aren't the same type!
We have a few options: either do explicit pattern matching:
pub fn new(name: &str) -> Option<Self> { let name = CString::new(name).expect("invalid library name"); let module = unsafe { LoadLibraryA(name.as_ptr()) }; match module { Some(module) => Some(Library { module }), None => None, } }
...but that's a bit redundant.
Or, we could use Option::map
- which lets us transform an Option<T>
into
an Option<U>
(In this case, T
is HModule
, and U
is Library
).
pub fn new(name: &str) -> Option<Self> { let name = CString::new(name).expect("invalid library name"); let res = unsafe { LoadLibraryA(name.as_ptr()) }; res.map(|module| Library { module }) }
Boom, three-liner, and it's relatively readable. Now we're cooking!
Let's move on to get_proc
, applying everything we've learned so far.
We're going to need GetProcAddress
. Previously, we defined it
like this:
type FarProc = *const c_void; extern "stdcall" { fn GetProcAddress(module: HModule, name: *const u8) -> FarProc; }
...but we can do better.
Firstly, we want to prevent calling GetProcAddress
with a null HModule.
That's already taken care of, because the HModule type is already a NonNull<T>
.
Secondly, we'll also be using CString
here, so we don't want *const u8
:
// changed: u8 => c_char fn GetProcAddress(module: HModule, name: *const c_char) -> FarProc;
Thirdly, GetProcAddress
can return null, if there is no such procedure
in the DLL, so, again, let's encode that:
// added: Option<T> fn GetProcAddress(module: HModule, name: *const c_char) -> Option<FarProc>;
Back when we were designing our API, we've already decided the signature of
get_proc
:
impl Library { pub unsafe fn get_proc<T>(&self, name: &str) -> Option<T> { unimplemented!() } }
Now all that's left is to.. implement it. First we'll need a CString
:
pub unsafe fn get_proc<T>(&self, name: &str) -> Option<T> { let name = CString::new(name).expect("invalid proc name"); unimplemented!() }
Then call the actual Win32 API:
pub unsafe fn get_proc<T>(&self, name: &str) -> Option<T> { let name = CString::new(name).expect("invalid proc name"); let res = GetProcAddress(self.module, name.as_ptr()); unimplemented!() }
Note that passing self.module
to GetProcAddress
works out beautifully,
because they're both NonNull<HModule>
already.
And finally, use Option::map
, just like in new
:
pub unsafe fn get_proc<T>(&self, name: &str) -> Option<T> { let name = CString::new(name).expect("invalid proc name"); let res = GetProcAddress(self.module, name.as_ptr()); res.map(|proc| transmute_copy(&proc)) }
What's transmute_copy
? It's transmute
's evil little brother. And I couldn't
make it work with regular transmute
, because of the generic type parameter.
If you can figure out, you'll definitely be credited in this article.
Here's our complete src/loadlibrary.rs
so far:
use std::{ ffi::{c_void, CString}, mem::transmute_copy, os::raw::c_char, ptr::NonNull, }; type HModule = NonNull<c_void>; type FarProc = NonNull<c_void>; extern "stdcall" { fn LoadLibraryA(name: *const c_char) -> Option<HModule>; fn GetProcAddress(module: HModule, name: *const c_char) -> Option<FarProc>; } pub struct Library { module: HModule, } impl Library { pub fn new(name: &str) -> Option<Self> { let name = CString::new(name).expect("invalid library name"); let res = unsafe { LoadLibraryA(name.as_ptr()) }; res.map(|module| Library { module }) } pub unsafe fn get_proc<T>(&self, name: &str) -> Option<T> { let name = CString::new(name).expect("invalid proc name"); let res = GetProcAddress(self.module, name.as_ptr()); res.map(|proc| transmute_copy(&proc)) } }
A note on the unsafety of get_proc
. While it seems at first glance that it
should be safe (this is part of the reason we're making a Rust wrapper over
the Win32 API, after all), it relies on the user providing a correct T
type.
Since there is an invariant to be upheld by the caller that isn't enforced
by the compiler, get_proc
must be marked unsafe, as explained in-depth
by soundlogic2236 on GitHub.
It's time to change src/main.rs
's main()
function to use our new wrapper.
Before, we had:
unsafe { let h = LoadLibraryA("IPHLPAPI.dll\0".as_ptr()); let IcmpCreateFile: IcmpCreateFile = transmute(GetProcAddress(h, "IcmpCreateFile\0".as_ptr())); let IcmpSendEcho: IcmpSendEcho = transmute(GetProcAddress(h, "IcmpSendEcho\0".as_ptr())); }
Now, we have:
let iphlp = loadlibrary::Library::new("IPHLPAPI.dll").unwrap(); let IcmpCreateFile: IcmpCreateFile = unsafe { iphlp.get_proc("IcmpCreateFile").unwrap() }; let IcmpSendEcho: IcmpSendEcho = unsafe { iphlp.get_proc("IcmpSendEcho").unwrap() };
And it still works!
Special thanks to @porglezomp for pointing out mistakes in the way I was using
NonNull<T>
and suggesting a fix!
Thanks to my sponsors: Tom Forbes, belzael, playest, Foxie Solutions, Marcus Griep, Philipp Hatt, Scott Sanderson, Olivia Crain, Philipp Gniewosz, Marc-Andre Giroux, Benjamin Röjder Delnavaz, Tiziano Santoro, Alan O'Donnell, Romet Tagobert, pinkhatbeard, notryanb, Mikkel Rasmussen, Guillaume Demonet, Olly Swanson, Chris Thackrey and 227 more
If you liked what you saw, please support my work!
Here's another article just for you:
Long story short: a couple of my articles got really popular on a bunch of sites, and someone, somewhere, went "well, let's see how much traffic that smart-ass can handle", and suddenly I was on the receiving end of a couple DDoS attacks.
It really doesn't matter what the articles were about — the attack is certainly not representative of how folks on either side of any number of debates generally behave.