Let's set aside our sup project for a while.

Don't get me wrong - it's a perfectly fine project, and, were we simply rewriting "ping" for Windows in Rust, we could (almost) stop there.

We're currently using the operating system's facility to speak ICMP, which is great for a bunch of reasons: we can be sure that whatever flaws there are in the implementation, all "native" Windows programs suffer from it as well.

As a bonus, the binary size is reasonable, (looks like 175KiB for me at time of this writing - built with cargo build --release, of course).

But we didn't come here to make sensible decisions for production-grade software. We came here to bind Win32 APIs and dig our way down the network stack, and we're all out of APIs to bind.

Using Wireshark

...would be cheating, I've decided. So let's not.

Wireshark is an extremely powerful packet analyzer, and we could probably do everything we wanted without leaving the comfort of its nifty GUI. Again, that's not why we came here. We came here to skim through specs and feel hopeless in the face of dropped packets, and oooh boyo do we have specs to read.

The OSI model

...has seen better days, as newer protocols tend to break the rules, but no networking course would be complete without it, so let's at least take a cursory look at its Wikipedia page:

Where does ICMP stand?

ICMP is a network layer protocol. There is no TCP or UDP port number associated with ICMP packets as these numbers are associated with the transport layer above. -- ICMP Wikipedia article

But later in that same article...

The ICMP packet is encapsulated in an IPv4 packet. -- ICMP Wikipedia article

Alright, we know that IPv4 packets are - usually - stuffed in Ethernet packets.

So, if we really wanted to implement the full stack, with the exception of the physical layer, we'd have to take care of three protocols, all of which handle different aspects of sending a ping:

We're going to implement the outer layers first, and then work our way in.

Two crates of rust on the wall, take one down, pass it around

As mentioned, we're going to make a new project for our network protocol implementations, that sup will hopefully, eventually, use.

Let's name it ersatz, which means:

being a usually artificial and inferior substitute or imitation -- Merriam-Webster.

Perfect.

Shell session
$ cargo new ersatz
     Created binary (application) `ersatz` package

A good start!

But we have a bit of a problem. We want to bypass Windows's ICMP, IP, and Ethernet implementation. And it turns out, it's not that easy... anymore.

From msdocs's socket page:

IPv6 and IPv4 operate differently when receiving a socket with a type of SOCK_RAW. The IPv4 receive packet includes the packet payload, the next upper-level header (for example, the IP header for a TCP or UDP packet), and the IPv4 packet header. The IPv6 receive packet includes the packet payload and the next upper-level header. The IPv6 receive packet never includes the IPv6 packet header.

Note: On Windows NT, raw socket support requires administrative privileges.

Why do raw sockets require administrative privileges? Presumably because having access to raw sockets allows you to do all sorts of fun things, like IP spoofing

When using raw sockets, we can send anything we like. We can send something malformed, and if we jiggle it just the right way, it might cause some piece of infrastructure to hack and cough and keel over for a bit.

This can sometimes be detected (some companies base most of their business model on it), and routing infrastructure is usually pretty conservative, and drops packets it feels funny about.

But the point is, when using UDP-level or TCP-level APIs, the OS can still reign over the chaos - it can check for basic assumptions. If it lets us fire away at the network interface freely, all bets are off.

So it requires administrative privileges. Unless of course, you write a kernel driver that lets you capture and inject random network data at a very low-level as a regular user, which is exactly what Npcap does.

But Npcap is a C library, and it's Windows-only, and I don't know about you but I'm all tuckered out when it comes to binding libraries - LET ME CRUNCH BITS GOSH DARN IT.

Luckily, someone took raw packet APIs from around the globe various operating systems, and packaged them neatly in one crate: rawsock.

Let's give it a shot.

Using rawsock

Shell session
$ cargo add rawsock
      Adding rawsock v0.3.0 to dependencies

Let's start simple, with a src/main.rs that has:

Rust code
fn main() -> Result<(), rawsock::Error> {
    let lib = rawsock::open_best_library()?;
    for interf in lib.all_interfaces()? {
        println!("- {:?}", interf);
    }
    Ok(())
}
Cool bear's hot tip

This will only run if you've installed npcap, otherwise you'll get a nice error.

If you haven't enabled the "Allow non-administrator users to capture network traffic", you'll need to run the app as an administrator.

Shell session
$ cargo run --quiet
- InterfaceDescription { name: "\\Device\\NPF_{2C493E35-02A0-4958-A660-8503D333247B}", description: "Oracle" }
- InterfaceDescription { name: "\\Device\\NPF_{C89ED92B-BEEE-4591-BD0B-88F490CCC54B}", description: "Microsoft" }
- InterfaceDescription { name: "\\Device\\NPF_{EB49DEF9-63DD-4F12-BB63-74D276FB95AE}", description: "NdisWan Adapter" }
- InterfaceDescription { name: "\\Device\\NPF_{64F033CC-004B-44F6-8AD0-423E923A59BE}", description: "Microsoft" }
- InterfaceDescription { name: "\\Device\\NPF_{0E89380B-814A-48FC-86C4-5C51B8040CB2}", description: "Microsoft" }
- InterfaceDescription { name: "\\Device\\NPF_{819D408C-4073-411C-B728-A3F2182C0C17}", description: "NdisWan Adapter" }
- InterfaceDescription { name: "\\Device\\NPF_{C0A1F438-0563-4EDB-8C7F-7CB44D9BF086}", description: "Microsoft" }
- InterfaceDescription { name: "\\Device\\NPF_{123B06A8-F027-4848-93E6-AD3AFF854007}", description: "NdisWan Adapter" }
- InterfaceDescription { name: "\\Device\\NPF_Loopback", description: "Adapter for loopback traffic capture" }
- InterfaceDescription { name: "\\Device\\NPF_{38A0BE18-EFE2-45F0-9749-BECDC8E65BD1}", description: "" }
- InterfaceDescription { name: "\\Device\\NPF_{6EBA42FA-5FA9-4209-B64F-DF52992236A3}", description: "Realtek Ethernet Controller" }

Frick. That's a whole lot of interfaces. Which one do we want?

Let's leave no room for guessing.

What did we learn?

There is a crate for everything.

If not, check again.

Using WMI from the command line

WMI, or Windows Management Instrumentation, lets you query various things about your system.

Ever wondered what architecture you were running on?

Fear not, the wmic command-line interface is here:

Bash
$ wmic OS Get OSArchitecture
OSArchitecture
64-bit

Yeah, the newlines are in the output, it's not a formatting error.

Cool bear's hot tip

*shrug*

The point is, everyone has different network interfaces on their computer. I have a cable one and a Wi-Fi one, and then a bunch of Virtual ones, and then there's loopback - but yours may be different!

Thankfully wmic is here to save the day.

This is going to be a two-step operation. First, we need to find the number of the "default interface".

Something I kinda glossed over in part 1 is that your computer's operating system has a routing table that maps IP address ranges to network interfaces.

If we try to reach 127.0.0.1, the OS network stack will look up the relevant address range and (if everything goes well) decide to use the Loopback interface.

In fact, heck, we can try it now:

Shell session
$ wmic Path Win32_IP4RouteTable Where "Destination='127.0.0.1'" Get
Age      Caption    Description                            Destination  Information  InstallDate  InterfaceIndex  Mask             Metric1  Metric2  Metric3  Metric4  Metric5  Name       NextHop  Protocol  Status  Type
1222294  127.0.0.1  127.0.0.1 - 255.255.255.255 - 0.0.0.0  127.0.0.1    0.0                       1               255.255.255.255  256      -1       -1       -1       -1       127.0.0.1  0.0.0.0  2                 3

We however, want to reach the internet. So the address we want to reach is, uh, apparently 0.0.0.0, which is associated to.. the default network interface.

Alrighty then!

Shell session
$ wmic Path Win32_IP4RouteTable Where "Destination='0.0.0.0'" Get
Age      Caption  Description                        Destination  Information  InstallDate  InterfaceIndex  Mask     Metric1  Metric2  Metric3  Metric4  Metric5  Name     NextHop        Protocol  Status  Type
1222511  0.0.0.0  0.0.0.0 - 0.0.0.0 - 192.168.1.254  0.0.0.0      0.0                       5               0.0.0.0  0        -1       -1       -1       -1       0.0.0.0  192.168.1.254  3                 4

Cool. Although it's a bit on the long side, and I don't feel like splitting strings right now - we've got a healthy dose of parsing ahead of us, so let's ask just for the InterfaceIndex.

Shell session
$ wmic Path Win32_IP4RouteTable Where "Destination='0.0.0.0'" Get InterfaceIndex
InterfaceIndex
5

Wonderful. Now what is this an index into exactly?

Well, Win32_NetworkAdapters, of course!

So that means, that.. with a second query, we could...?

Shell session
$ wmic Path Win32_NetworkAdapter Where "InterfaceIndex=5" Get Caption
Caption
[00000002] Intel(R) Wireless-AC 9462

Bingo! I had to redact the output a bit, because the output was a bit too revealing for my taste.

So far we've established that:

However, upon reviewing the output of our current ersatz program:

- InterfaceDescription { name: "\\Device\\NPF_{2C493E35-02A0-4958-A660-8503D333247B}", description: "Oracle" }
(etc.)

...I couldn't help but notice that the string after \\Device\\NPF_ looked a lot like a Globally unique identifier (GUID).

And, if we look at all the fields available in Win32_NetworkAdapter, we see a GUID in there:

Let's give it a shot:

Shell session
$ wmic Path Win32_NetworkAdapter Where "InterfaceIndex=5" Get GUID
GUID
{0E89380B-814A-48FC-86C4-5C51B8040CB2}

Is that in the list??

Of course it is!

If you're used to reading stuff on Medium, just pretend I included a happy dancing skeleton GIF, or maybe one from Brooklyn 99 where the dude in charge finally exclaims "BINGPOT!"

Cool bear's hot tip

Another way to write this article would've been the "top down" approach: to just explain how to do every bit correctly. But it would come a bit out of nowhere.

While writing this article, Amos actually had to look up what WMIC queries could be used. He knew WMIC from his work on the itch.io app but hadn't realized at first that it could be used to find the default network interface.

You can be sure that in Amos's research code (those article series require doing 1-2 weeks of research before writing the first one), the interface index was hard-coded. This is just sugar on top.

Anyway, the hope is that showing the "exploratory" approach helps showcasing how someone would go about figuring this stuff out, rather than give the impression that these things are self-evident.

Research is like, super important to development. Cool bear out.

What did we learn?

WMI allows one to perform queries on various aspects of a system's hardware and software - including network interface, IP routing tables, etc.

It includes its own query language, weirdly reminiscent of SQL.

Starting from Windows 2000, WMI comes preinstalled, although downloads were made available for Windows NT, 98, and even 95!

Using WMI from rust

So, how do we query WMI from Rust?

Well, we could just execute wmic and parse its output. So let's start there.

We're going to run into problems, though, because now main can not only fail because of rawsock:Error, but also because of std::io::Error.

So let's make an Error type real quick, it'll come in handy later.

In src/error.rs

Rust code
use std::fmt;

pub enum Error {
    Rawsock(rawsock::Error),
    IO(std::io::Error),
}

impl From<rawsock::Error> for Error {
    fn from(e: rawsock::Error) -> Self {
        Self::Rawsock(e)
    }
}

impl From<std::io::Error> for Error {
    fn from(e: std::io::Error) -> Self {
        Self::IO(e)
    }
}

impl fmt::Display for Error {
    fn fmt(&self, f: &mut fmt::Formatter) -> fmt::Result {
        match self {
            Self::Rawsock(e) => write!(f, "{}", e),
            Self::IO(e) => write!(f, "{}", e),
        }
    }
}

impl fmt::Debug for Error {
    fn fmt(&self, f: &mut fmt::Formatter) -> fmt::Result {
        write!(f, "{}", self)
    }
}

impl std::error::Error for Error {}

There. Now our errors can be either, and the ? sigil handles the conversion for us, using the Into trait we just implemented for both possible error types.

We just need to change src/main.rs to:

Rust code
// new!
mod error;
use error::Error;

fn main() -> Result<(), Error> {
    // ...
}

We'll make a convenience method to call wmic, since we need to call it twice, and the output needs to be parsed in a similar way:

Rust code
use std::process::Command;

fn wmic(args: &[&str]) -> Result<String, Error> {
    // create a new Command instance
    let output = Command::new("wmic").args(args).output()?;

    // Command gives us standard output and standard error as vectors of u8,
    // ie. bytes, but to us they're text, so:
    let stdout = std::str::from_utf8(&output.stdout).unwrap();
    let stderr = std::str::from_utf8(&output.stderr).unwrap();
    // Note: we could add "utf-8 parsing errors" to our error type,
    // but in this case, if wmic returns invalid utf-8, we won't be able
    // to do anything useful anyway, so we're okay with panicking instead.

    // we expect the process to be created successfully,
    // and to return a 0 exit code.
    match output.status.code() {
        Some(0) => { /* all good */ }
        _ => panic!("wmic failed:\n{}", stderr),
    }

    let response = stdout
        .split("\n") // for all lines
        .map(|s| s.trim()) // trim whitespace
        .filter(|s| !s.is_empty()) // reject empty lines
        .last() // grab the last one
        .unwrap(); // panic if there is no "last one"
    Ok(response.to_owned()) // and convert to an owned String.
}

Let's call it once, from src/main.rs:

Rust code
    // in main()
    let index = wmic(&[
        "Path",
        "Win32_IP4RouteTable",
        "Where",
        "Destination='0.0.0.0'",
        "Get",
        "InterfaceIndex",
    ])?;
    println!("interface index = {}", index);
Shell session
$ cargo run --quiet
interface index = 5

Awesome! Second wmic query coming up:

Rust code
    let query = format!("InterfaceIndex={}", index);
    let guid = wmic(&[
        "Path",
        "Win32_NetworkAdapter",
        "Where",
        &query,
        "Get",
        "GUID",
    ])?;
    println!("guid = {}", guid);
Shell session
$ cargo run --quiet
interface index = 5
guid = {0E89380B-814A-48FC-86C4-5C51B8040CB2}

And finally, let's build the complete path for Npcap and find the corresponding interface:

Rust code
    // note: the string we're building has backslashes (`\`),
    // so we use a "raw string literal", where backslashes just
    // mean backslashes, not escape sequences.
    // r#"foo\bar"# == "foo\\bar"

    let name = format!(r#"\Device\NPF_{}"#, guid);
    let interfs = lib.all_interfaces()?;
    let interf = interfs.iter().find(|i| i.name == name).unwrap();
    println!("interf = {}", interf);
Shell session
$ cargo run --quiet
interface index = 5
guid = {0E89380B-814A-48FC-86C4-5C51B8040CB2}
interf = \Device\NPF_{0E89380B-814A-48FC-86C4-5C51B8040CB2}, Microsoft

Everything's falling into place.

What did we learn?

Spawning external processes and capturing their output is deceptively simple in Rust.

Even though command-line programs tend to output text, as far as the Rust standard library is concerned, they output bytes, and it's up to us to decode them properly.

Avoiding wmic

There's a couple of things that bother me about using wmic going forward.

First of all, it's not robust at all. If someone decides to remove C:\Windows\System32\Wbem\wmic.exe (and someone eventually will), then our program will break. Even though all the information is right there!

Secondly, this is a systems-level series of articles.

If we were happy scripting our way around we might as well go ahead and write an entire HTTP server in bash. But this isn't that kind of article series.

Thirdly, when you ask WMIC for its help text, it shows the following:

Shell session
$ wmic -?

WMIC is deprecated.

[global switches] <command>

The following global switches are available:

(rest of - very long - output was cut)

So wmic is deprecated. Presumably in favor of the Powershell Get-WmiObject cmdlet.

But, for the same reasons, let's not do Powershell right now.

Because, as it turns out - there is a crate for that

From looking at its README, it seems like we're going to want to use serde as well, so let's go ahead:

Shell session
$ cargo add wmi serde
      Adding wmi v0.4.6 to dependencies
      Adding serde v1.0.102 to dependencies

And let's copy a few lines from the README, see how it goes:

Rust code
use serde::Deserialize;
use wmi::{COMLibrary, Variant, WMIConnection, WMIDateTime};

fn main() -> Result<(), Error> {
    let com_con = COMLibrary::new()?;

    // (cut)
}
Shell session
$ cargo check --quiet
error[E0277]: `?` couldn't convert the error to `error::Error`
 --> src\main.rs:8:36
  |
8 |     let com_con = COMLibrary::new()?;
  |                                    ^ the trait `std::convert::From<WMIError>` is not implemented for `error::Error`
  |
  = note: the question mark operation (`?`) implicitly performs a conversion on the error value using the `From` trait
  = help: the following implementations were found:
            <error::Error as std::convert::From<rawsock::common::err::Error>>
            <error::Error as std::convert::From<std::io::Error>>
  = note: required by `std::convert::From::from`

error: aborting due to previous error

Oh. Right. Different crate, different error type.

Let's just add it to our catch-all error type:

Rust code
use std::fmt;

pub enum Error {
    // (cut: Rawsock, IO variants)
    WMI(wmi::utils::WMIError),
}

// (cut: Rawsock, IO impls)

impl From<wmi::utils::WMIError> for Error {
    fn from(e: wmi::utils::WMIError) -> Self {
        Self::WMI(e)
    }
}

impl fmt::Display for Error {
    fn fmt(&self, f: &mut fmt::Formatter) -> fmt::Result {
        match self {
            // (cut: Rawsock, IO match arms)
            Self::WMI(e) => write!(f, "{}", e),
        }
    }
}

Ok, good. Now it compiles.

Following the README more, it looks like we should be able to list all Win32_IP4RouteTable entries with just a few lines.

Rust code
use wmi::{COMLibrary, WMIConnection, Variant};
use std::collections::HashMap;

fn main() -> Result<(), Error> {
    let com_con = COMLibrary::new()?;
    let wmi_con = WMIConnection::new(com_con.into())?;

    let results: Vec<HashMap<String, Variant>> =
        wmi_con.raw_query("SELECT * FROM Win32_IP4RouteTable")?;
    println!("{:#?}", results);

    Ok(())
}
Shell session
$ cargo run --quiet
[{"Age": I8(192995), "Metric3": I8(-1), "Description": String("0.0.0.0 - 0.0.0.0 - 192.168.1.254"), "Pro
tocol": I8(3),

Good, good. That's a mouthful though - and if we look a little further in the README, it looks like we can just deserialize WMI query results to structs? Now I want some of that, let's try this:

Rust code
use wmi::{COMLibrary, WMIConnection};
use serde::Deserialize;

fn main() -> Result<(), Error> {
    let com_con = COMLibrary::new()?;
    let wmi_con = WMIConnection::new(com_con.into())?;

    #[derive(Deserialize, Debug)]
    #[allow(non_camel_case_types, non_snake_case)]
    struct Win32_IP4RouteTable {
        InterfaceIndex: i64,
    }

    let results: Vec<Win32_IP4RouteTable> =
        wmi_con.raw_query("SELECT * FROM Win32_IP4RouteTable")?;
    println!("{:#?}", results);
    Ok(())
}
Shell session
$ cargo run --quiet
[
    Win32_IP4RouteTable {
        InterfaceIndex: 5,
    },
    Win32_IP4RouteTable {
        InterfaceIndex: 1,
    },
    Win32_IP4RouteTable {
        InterfaceIndex: 1,
    },
    Win32_IP4RouteTable {
(cut)

That's better! Although we're not filtering anything right now. We only want the default interface.

We could just change the parameter to raw_query, but it feels a little too.. untyped for my taste.

Alternatively, we could use filtered_query instead. Its signature is as follows:

Rust code
/// Query all the objects of type T, while filtering according to `filters`.
///
pub fn filtered_query<T>(&self, filters: &HashMap<String, FilterValue>) -> Result<Vec<T>, Error>
where
    T: de::DeserializeOwned,
{}

Cool, we can do that:

Rust code
fn main() -> Result<(), Error> {
    let com_con = COMLibrary::new()?;
    let wmi_con = WMIConnection::new(com_con.into())?;

    #[derive(Deserialize, Debug)]
    #[allow(non_camel_case_types, non_snake_case)]
    struct Win32_IP4RouteTable {
        InterfaceIndex: i64,
    }

    let results: Vec<Win32_IP4RouteTable> = {
        // note: we don't need to annotate `filters`' type here,
        // because we're passing it to `filtered_query` later.
        let mut filters = HashMap::new();
        filters.insert(
            "Destination".into(),
            wmi::query::FilterValue::Str("0.0.0.0"),
        );
        wmi_con.filtered_query(&filters)?
    };
    println!("{:#?}", results);
    return Ok(());
}
Shell session
$ cargo run --quiet
[
    Win32_IP4RouteTable {
        InterfaceIndex: 5,
    },
]

Perfect! Although... I don't really like having to instanciate a HashMap by hand. I really wish the language had map literals, just like we have the vec! macro.

Mhh... if only there was a crate for that.

Shell session
$ cargo add maplit
      Adding maplit v1.0.2 to dependencies
Rust code
use maplit::*;
use wmi::query::FilterValue;

// in main
let results: Vec<Win32_IP4RouteTable> = {
    wmi_con.filtered_query(&hashmap! {
        "Destination".into() => FilterValue::Str("0.0.0.0"),
    })?
};

That's a bit more readable!

Although... we want to find a single interface, and right now we've got a full Vec<T> of them.

Surely we can fix that. We can't just take [0] because it would:

  1. Panic if there was less than 1 element in the vec
  2. Take a reference to the first element, not move it out of the vec.

What we want instead is to get a drain iterator, and grab its first element.

Rust code
// note: the type inference is *fairly strong* here, note that
// we only

let route: Win32_IP4RouteTable = wmi_con
    .filtered_query(&hashmap! {
        "Destination".into() => FilterValue::Str("0.0.0.0"),
    })?
    .drain(..)
    .next()
    .expect("should have a default network interface");
println!("{:#?}", route);

Does it still work?

Shell session
$ cargo run --quiet

Win32_IP4RouteTable {
    InterfaceIndex: 5,
}

It does! Cool.

Let's go for the second query:

Rust code
// in main, after route query

#[derive(Deserialize, Debug)]
#[allow(non_camel_case_types, non_snake_case)]
struct Win32_NetworkAdapter {
    GUID: String,
}
let adapter: Win32_NetworkAdapter = wmi_con
    .filtered_query(&hashmap! {
        "InterfaceIndex".into() => FilterValue::Number(route.InterfaceIndex),
    })?
    .drain(..)
    .next()
    .expect("default network interface should exist");
println!("{:#?}", adapter);
Shell session
$ cargo run --quiet

Win32_IP4RouteTable {
    InterfaceIndex: 5,
}
Win32_NetworkAdapter {
    GUID: "{0E89380B-814A-48FC-86C4-5C51B8040CB2}",
}

Great! From there, we can build the name of the interface we want to open and be on our way.

Here's the complete listing of our program so far:

Rust code
mod error;
use error::Error;
use rawsock::open_best_library;

use maplit::*;
use serde::Deserialize;
use wmi::{query::FilterValue, COMLibrary, WMIConnection};

fn main() -> Result<(), Error> {
    let com_con = COMLibrary::new()?;
    let wmi_con = WMIConnection::new(com_con.into())?;

    #[derive(Deserialize, Debug)]
    #[allow(non_camel_case_types, non_snake_case)]
    struct Win32_IP4RouteTable {
        InterfaceIndex: i64,
    }

    let route: Win32_IP4RouteTable = wmi_con
        .filtered_query(&hashmap! {
            "Destination".into() => FilterValue::Str("0.0.0.0"),
        })?
        .drain(..)
        .next()
        .expect("should have a default network interface");
    println!("{:#?}", route);

    #[derive(Deserialize, Debug)]
    #[allow(non_camel_case_types, non_snake_case)]
    struct Win32_NetworkAdapter {
        GUID: String,
    }
    let adapter: Win32_NetworkAdapter = wmi_con
        .filtered_query(&hashmap! {
            "InterfaceIndex".into() => FilterValue::Number(route.InterfaceIndex),
        })?
        .drain(..)
        .next()
        .expect("default network interface should exist");
    println!("{:#?}", adapter);

    let lib = open_best_library()?;
    let interface_name = format!(r#"\Device\NPF_{}"#, adapter.GUID);
    lib.open_interface(&interface_name)?;

    println!("Interface opened!");
    Ok(())
}