Thanks to my sponsors: Isak Sunde Singh, Nyefan, Paige Ruten, Seth, Leigh Oliver, James Brown, Mark Old, Marcin Kołodziej, pinkhatbeard, SeniorMars, Garret Kelly, Tom Forbes, Andronik, Guillaume E, Yann Schwartz, Rufus Cable, Tyler Bloom, Jelle Besseling, Johnathan Pagnutti, Valentin Mariette and 235 more

Dec 12, 2019

16 min #networking · #rust · #win32 · #ffi · #arp · #ethernet · #parsing

Crafting ARP packets to find a remote host's MAC address

From the series Making our own ping

1

2

3

4

5

6

7

8

9

10

11

12

13

14

👋 This page was last updated ~5 years ago. Just so you know.

Alright. ALRIGHT. I know, we're all excited, but let's think about what we're doing again.

So we've managed to look at real network traffic and parse it completely. We've also taken some ICMP packets, parsed them, and then serialized them right back and we got the exact same result.

So I know what you're thinking - let's just move our way down the stack again - stuff that ICMP packet in an IP packet, then in an Ethernet frame, and then serialize the whole thing.

Right? Wrong.

We need to think about what we're doing. We (a host) are attempting to ping another host. By sending ICMP packets that are inside IPv4 packets that are inside Ethernet frames. That means we need four principal pieces of information:

The Source MAC address
The Destination MAC address
The Source IP
The Destination IP

Cool bear's hot tip

It's been a long time since we discussed what the MAC and IP addresses were for, so here's a refresher.

The destination IP address is the eventual, logical destination of our packet. It's going to get there, hopefully, after a bunch of hops. That requires the coperation (and coordination) of a bunch of hosts on the way there. The first host that needs to cooperate with us is the gateway - our router, or modem, (or box that does both), whatever connects us to the rest of the internet.

Now, we got the destination IP, because that's what the user specifies when they call ping 8.8.8.8. But everything else uhh we're going to have to go shopping for.

That's right, that means digging into Win32 APIs. But fear not! All our past work of making things easy to bind will pay off here.

Let's take a look at src/netinfo.rs:

#[repr(C)]
#[derive(CustomDebug)]
pub struct IpForwardRow {
    dest: ipv4::Addr,
    mask: ipv4::Addr,
    policy: u32,
    next_hop: ipv4::Addr,
    if_index: u32,

    #[debug(skip)]
    _other_fields: [u32; 9],
}

Mh. Interesting. What are those equal to again (for the default route)?

// in `src/netinfo.rs

pub fn default_nic_guid() -> Result<String, Error> {
    let table = VLS::new(|ptr, size| GetIpForwardTable(ptr, size, false))?;
    let entry = table
        .entries()
        .iter()
        .find(|r| r.dest == ipv4::Addr([0, 0, 0, 0]))
        .ok_or(Error::DefaultRouteMissing)?;

    println!("default route = {:#?}", entry);

    // etc.
}

$ cargo run --quiet
default route = IpForwardRow {
    dest: 0.0.0.0,
    mask: 0.0.0.0,
    policy: 0,
    next_hop: 192.168.1.254,
    if_index: 5,
}

Okay well. That doesn't immediately help us.

We have the "destination IP", the "source IP" is our IP address, which is 192.168.1.16 for me (but it's unlikely you have the same). Then we need our NIC's MAC address, which we can probably get by querying for the network adapter's information.

As for the "destination MAC", that needs to be the physical address of the host we're immediately sending the Ethernet frame to. It's not the MAC address of the Google DNS server we're pinging.

It's the MAC address of our gateway - my router. The MAC address of the host at the next hop. The MAC address that corresponds to the IP address 192.168.1.254. That MAC address.

How do we find the MAC address associated with an IP address?

With... the Address Resolution Protocol (ARP).

Now, we have two options. We could use SendARP() from IPHPLAPI.dll, or we could just roll our own.

We're gonna roll our own.

Cool bear's hot tip

This is entirely gratuitous, as our operating system keeps an ARP table handy.

If we've been able to obtain a DHCP lease, we definitely have the gateway's MAC address in our local table.

Sending an ARP request just to get that is spamming the network. But it's our network.

We will spam it if we darn well please.

Implementing a useful subset of ARP

ARP packets look like this:

Easy. Eaaaaasy. We'll have this done in no time.

Just before we jump into it though, let's find our own MAC and IP addresses. Why do we need it? Well, when we send our ARP probe, we need to include a "return address", so that other hosts on the network know where to send the reply.

Cool bear's hot tip

But why do we need a return address?

I explained waaaaay back in Part 1 that MAC addresses were especially useful if you had several hosts on the same Ethernet bus.

Then I went on to explain that routers have a different physical port for each computer that connects to it. However, my computer connects to my router over Wi-Fi, which is sort of like a bus, in the sense that every Wi-Fi device is using the same "channel" to speak to the router (so they can "hear" each other's packets - although they might not be able to make sense of them).

The physical "connection" to the router matters very little, though, because ARP is over Ethernet, not over IP, so MAC addresses are all we have to go by anyway.

It doesn't look like GetInterfaceInfo returns that information in particular:

#[repr(C)]
#[derive(Debug)]
pub struct IpAdapterIndexMap {
    index: u32,
    name: IpAdapterName,
}

Right, that's not enough.

From a cursory look at the Microsoft docs, I see two functions that could be of potential interest to us: GetIpAddrTable, for interface-to-IPv4 address mapping, and GetIpNetTable, for interface-to-Physical address mapping (aka MAC address).

Let's try the former, using the techniques we developed in Part 8.

First, we need to add them to our bind! macro invocation:

// in `src/netinfo.rs`

crate::bind! {
    library "IPHLPAPI.dll";

    fn GetIpForwardTable(table: *mut IpForwardTable, size: *mut u32, order: bool) -> u32;
    fn GetInterfaceInfo(info: *mut IpInterfaceInfo, size: *mut u32) -> u32;

    // new!
    fn GetIpAddrTable(table: *mut IpAddrTable, size: *mut u32, order: bool) -> u32;
}

Both tables have structures similar to IpForwardTable:

#[repr(C)]
#[derive(Debug)]
pub struct IpAddrTable {
    num_entries: i32,
    entries: [IpAddrRow; 1],
}

We'll add an entries() getter for it. Again, this is pretty much copy-paste from IpForwardTable:

impl IpAddrTable {
    fn entries(&self) -> &[IpAddrRow] {
        // note: `num_entries` is an u32, as per the Win32 API,
        // but slices in Rust use `usize` for indexing.
        unsafe { slice::from_raw_parts(&self.entries[0], self.num_entries as usize) }
    }
}

As for the rows, here's the C definition for the MIB_IPADDRROW_W2K struct:

typedef struct _MIB_IPADDRROW_W2K {
  DWORD          dwAddr;
  DWORD          dwIndex;
  DWORD          dwMask;
  DWORD          dwBCastAddr;
  DWORD          dwReasmSize;
  unsigned short unused1;
  unsigned short unused2;
} MIB_IPADDRROW_W2K, *PMIB_IPADDRROW_W2K;

We can see that a bunch of those are IPv4 addresses, that's easy enough to bind:

#[repr(C)]
#[derive(CustomDebug)]
pub struct IpAddrRow {
    pub addr: ipv4::Addr,
    pub index: u32,
    pub mask: ipv4::Addr,
    pub bcast_addr: ipv4::Addr,
    pub reasm_size: u32,

    #[debug(skip)]
    unused1: u16,
    #[debug(skip)]
    unused2: u16,
}

Let's try calling it:

// in `src/netinfo.rs`

pub fn default_nic_guid() -> Result<String, Error> {
    let table = VLS::new(|ptr, size| GetIpForwardTable(ptr, size, false))?;
    let entry = table
        .entries()
        .iter()
        .find(|r| r.dest == ipv4::Addr([0, 0, 0, 0]))
        .ok_or(Error::DefaultRouteMissing)?;

    let ifaces = VLS::new(|ptr, size| GetInterfaceInfo(ptr, size))?;
    let iface = ifaces
        .adapters()
        .iter()
        .find(|r| r.index == entry.if_index)
        .ok_or(Error::DefaultInterfaceMissing)?;

    // new!
    let addr_rows = VLS::new(|ptr, size| GetIpAddrTable(ptr, size, false))?;
    let addr_entries: Vec<_> = addr_rows.entries().filter(|r| r.index == entry.if_index).collect();
    println!("addr entries = {:#?}", addr_entries);

    // etc.
}

$ cargo run
addr entries = [
    IpAddrRow {
        addr: 192.168.1.16,
        index: 5,
        mask: 255.255.255.0,
        bcast_addr: 1.0.0.0,
        reasm_size: 65535,
    },
]

Hey, that's our IP!

What about our MAC address? Well, I looked at the information returned by GetIpNetTable and it didn't contain our own MAC address, just that of neighbors. For that one we'll actually need GetAdaptersInfo, which returns more information than GetInterfaceInfo.

Its C declaration is as follows:

IPHLPAPI_DLL_LINKAGE ULONG GetAdaptersInfo(
  PIP_ADAPTER_INFO AdapterInfo,
  PULONG           SizePointer
);

Interestingly, the first argument is directly to one of the entries. It actually returns a linked list of structures, so let's take a look at the entry structure:

typedef struct _IP_ADAPTER_INFO {
  struct _IP_ADAPTER_INFO *Next;
  DWORD                   ComboIndex;
  char                    AdapterName[MAX_ADAPTER_NAME_LENGTH + 4];
  char                    Description[MAX_ADAPTER_DESCRIPTION_LENGTH + 4];
  UINT                    AddressLength;
  BYTE                    Address[MAX_ADAPTER_ADDRESS_LENGTH];
  DWORD                   Index;
  UINT                    Type;
  UINT                    DhcpEnabled;
  PIP_ADDR_STRING         CurrentIpAddress;
  IP_ADDR_STRING          IpAddressList;
  IP_ADDR_STRING          GatewayList;
  IP_ADDR_STRING          DhcpServer;
  BOOL                    HaveWins;
  IP_ADDR_STRING          PrimaryWinsServer;
  IP_ADDR_STRING          SecondaryWinsServer;
  time_t                  LeaseObtained;
  time_t                  LeaseExpires;
} IP_ADAPTER_INFO, *PIP_ADAPTER_INFO;

That's uhh a mouthful.

Cool bear's hot tip

The CurrentIpAddress field above also contains our own IP address, but stored as a dot-separated string.

Eww, let's not.

Let's give it our best shot.

const MAX_ADAPTER_NAME_LENGTH: usize = 256;
const MAX_ADAPTER_DESCRIPTION_LENGTH: usize = 128;

#[repr(C)]
#[derive(CustomDebug)]
pub struct IpAdapterInfo {
    pub next: Option<NonNull<IpAdapterInfo>>,
    pub combo_index: u32,

    #[debug(skip)]
    pub adapter_name: [u8; MAX_ADAPTER_NAME_LENGTH + 4],
    #[debug(skip)]
    pub description: [u8; MAX_ADAPTER_DESCRIPTION_LENGTH + 4],

    // important: we can't use the `char` type above, as the C
    // struct references "C chars" (bytes), not "Rust chars" (Unicode
    // scalar values, 4 bytes).

    // I'm just saying it in case someone ever makes that mistake,
    // it's not like I made the mistake when first coding this up,
    // and spent quite a few puzzling minutes wondering what the heck
    // was going on. Haha. I would never.

    pub address_length: u32,
    pub address: ethernet::Addr,
    pub address_rest: u16,
    pub index: u32,
    pub type: u32,
    // ignore rest of fields
}

Hey, that's not all the fields from the C struct definition!

Well, since we have a pointer to the next entry, we don't need to cover all the fields, because the offset will always be right.

We also assumed that address will be 6 bytes (for ethernet), and we'll make sure by ... checking that address_length is 6.

Now all that's left is to bind it:

crate::bind! {
    library "IPHLPAPI.dll";

    fn GetIpForwardTable(table: *mut IpForwardTable, size: *mut u32, order: bool) -> u32;
    fn GetInterfaceInfo(info: *mut IpInterfaceInfo, size: *mut u32) -> u32;
    fn GetIpAddrTable(table: *mut IpAddrTable, size: *mut u32, order: bool) -> u32;

    // new!
    fn GetAdaptersInfo(list: *mut IpAdapterInfo, size: *mut u32) -> u32;
}

And try it out:

// in `src/netinfo.rs`
// in `default_nic_guid()`

let mut adapter_list_head = VLS::new(|ptr, size| GetAdaptersInfo(ptr, size))?;
let mut current = NonNull::new(&mut *adapter_list_head);

loop {
    if let Some(adapter) = current {
        let adapter = unsafe { adapter.as_ref() };
        if adapter.address_length == 6 && adapter.index == entry.if_index {
            println!("adapter = {:#?}", adapter);
            break;
        }
        current = adapter.next;
    } else {
        break;
    }
}

// etc.

Mhh, that doesn't compile:

$ cargo run --quiet
error[E0596]: cannot borrow data in a dereference of `vls::VLS<netinfo::IpAdapterInfo>` as mutable
  --> src\netinfo.rs:33:36
   |
33 |     let mut current = NonNull::new(&mut *adapter_list);
   |                                    ^^^^^^^^^^^^^^^^^^ cannot borrow as mutable
   |
   = help: trait `DerefMut` is required to modify through a dereference, but it is not implemented for `vls::VLS<netinfo::IpAdapterInfo>`

Oh, right, we need to implement DerefMut for VLS.

We also need to make sure that VLS handles the error code that GetAdaptersInfo returns, which is ERROR_BUFFER_OVERFLOW, whereas the others returned ERROR_INSUFFICIENT_BUFFER.

// in `src/vls.rs`

const ERROR_INSUFFICIENT_BUFFER: u32 = 122;
const ERROR_BUFFER_OVERFLOW: u32 = 111;

impl<T> VLS<T> {
    pub fn new<F>(f: F) -> Result<Self, Error>
    where
        F: Fn(*mut T, *mut u32) -> u32,
    {
        let mut size = 0;
        match f(ptr::null_mut(), &mut size) {
            ERROR_INSUFFICIENT_BUFFER => {} // good
            ERROR_BUFFER_OVERFLOW => {}     // also good
            ret => return Err(Error::Win32(ret)),
        };

        let mut v = vec![0u8; size as usize];
        match f(unsafe { mem::transmute(v.as_mut_ptr()) }, &mut size) {
            0 => {} // good
            ret => return Err(Error::Win32(ret)),
        };

        Ok(Self {
            v,
            _phantom: PhantomData::default(),
        })
    }
}

use std::ops::DerefMut;

impl<T> DerefMut for VLS<T> {
    fn deref_mut(&mut self) -> &mut T {
        unsafe { mem::transmute(self.v.as_ptr()) }
    }
}

Let's try this again shall we:

$ cargo run --quiet
addr entries = [
    IpAddrRow {
        addr: 192.168.1.16,
        index: 5,
        mask: 255.255.255.0,
        bcast_addr: 1.0.0.0,
        reasm_size: 65535,
    },
]
adapter = IpAdapterInfo {
    next: None,
    combo_index: 5,
    address_length: 6,
    address: F4-D1-08-0B-7E-BC,
    index: 5,
}

Hey, that's our MAC address!

Let's implement ARP request and replies.

We'll need a new module:

// in `src/main.rs`

mod arp;

This time we'll use derive_try_from_primitive, like we did in Part 9.

// in `src/arp.rs`

use crate::{ethernet, ipv4};
use derive_try_from_primitive::*;

#[derive(Debug, TryFromPrimitive, Clone, Copy)]
#[repr(u16)]
pub enum Operation {
    Request = 1,
    Reply = 2,
}

#[derive(Debug, TryFromPrimitive, Clone, Copy)]
#[repr(u16)]
pub enum HardwareType {
    Ethernet = 1,
}

#[derive(Debug)]
pub struct Packet {
    pub operation: Operation,
    pub sender_hw_addr: ethernet::Addr,
    pub sender_ip_addr: ipv4::Addr,
    pub target_hw_addr: ethernet::Addr,
    pub target_ip_addr: ipv4::Addr,
}

Good, good, we're getting somewhere. Note that we only support the Ethernet+IPv4 combo. We'll just ignore all the rest.

Let's write some parsers, starting with the simple ones:

// in `src/arp.rs`

use crate::parse;

impl Operation {
    pub fn parse(i: parse::Input) -> parse::Result<Option<Self>> {
        use nom::{combinator::map, error::context, number::complete::be_u16};
        context("Operation", map(be_u16, Self::try_from))(i)
    }
}

impl HardwareType {
    pub fn parse(i: parse::Input) -> parse::Result<Option<Self>> {
        use nom::{combinator::map, error::context, number::complete::be_u16};
        context("HardwareType", map(be_u16, Self::try_from))(i)
    }
}

And moving up to Packet. This'll be a whole thing, so, pay attention:

impl Packet {
    pub fn parse(i: parse::Input) -> parse::Result<Self> {
        let original_i = i;
        use nom::{number::complete::be_u8, sequence::tuple};

        let (i, (htype, ptype, _hlen, _plen)) = tuple((
            HardwareType::parse,
            ethernet::EtherType::parse,
            be_u8,
            be_u8,
        ))(i)?;

        if let Some(HardwareType::Ethernet) = htype {
            // good!
        } else {
            let msg = "arp: only Ethernet is supported".into();
            return Err(nom::Err::Error(parse::Error::custom(original_i, msg)));
        }

        if let Some(ethernet::EtherType::IPv4) = ptype {
            // good!
        } else {
            let msg = "arp: only IPv4 is supported".into();
            return Err(nom::Err::Error(parse::Error::custom(original_i, msg)));
        }

        let (i, operation) = Operation::parse(i)?;
        let operation = match operation {
            Some(operation) => operation,
            _ => {
                let msg = "arp: only Request and Reply operations are supported".into();
                return Err(nom::Err::Error(parse::Error::custom(original_i, msg)));
            }
        };

        let (i, (sender_hw_addr, sender_ip_addr)) =
            tuple((ethernet::Addr::parse, ipv4::Addr::parse))(i)?;

        let (i, (target_hw_addr, target_ip_addr)) =
            tuple((ethernet::Addr::parse, ipv4::Addr::parse))(i)?;

        let res = Self {
            operation,
            sender_hw_addr,
            sender_ip_addr,
            target_hw_addr,
            target_ip_addr,
        };
        Ok((i, res))
    }
}

Now let's write some serializers, starting again with the simple ones:

// in `src/arp.rs`

use cookie_factory as cf;
use std::io;

impl Operation {
    pub fn serialize<'a, W: io::Write + 'a>(&'a self) -> impl cf::SerializeFn<W> + 'a {
        use cf::bytes::be_u16;
        be_u16(*self as u16)
    }
}

impl HardwareType {
    pub fn serialize<'a, W: io::Write + 'a>(&'a self) -> impl cf::SerializeFn<W> + 'a {
        use cf::bytes::be_u16;
        be_u16(*self as u16)
    }
}

Now for ipv4::Addr

// in `src/ipv4.rs`

use cookie_factory as cf;
use std::io;

impl Addr {
    pub fn serialize<'a, W: io::Write + 'a>(&'a self) -> impl cf::SerializeFn<W> + 'a {
        use cf::combinator::slice;
        slice(&self.0)
    }
}

Now for ethernet::Addr and ethernet::EtherType:

// in `src/ethernet.rs`

use cookie_factory as cf;
use std::io;

impl Addr {
    pub fn serialize<'a, W: io::Write + 'a>(&'a self) -> impl cf::SerializeFn<W> + 'a {
        use cf::combinator::slice;
        slice(&self.0)
    }
}

impl EtherType {
    pub fn serialize<'a, W: io::Write + 'a>(&'a self) -> impl cf::SerializeFn<W> + 'a {
        use cf::bytes::be_u16;
        // note: I had to derive `Clone` and `Copy` to make this work
        // those weren't originally derived for `ethernet::EtherType`
        be_u16(*self as u16)
    }
}

Finally, we can write the serializer for arp::Packet:

// in `src/arp.rs`

impl Packet {
    pub fn serialize<'a, W: io::Write + 'a>(&'a self) -> impl cf::SerializeFn<W> + 'a {
        use cf::{bytes::be_u8, sequence::tuple};

        let htype = HardwareType::Ethernet.serialize();
        let ptype = ethernet::EtherType::IPv4.serialize();
        let hlen = be_u8(6);
        let plen = be_u8(4);
        tuple((
            htype,
            ptype,
            hlen,
            plen,
            self.operation.serialize(),
            self.sender_hw_addr.serialize(),
            self.sender_ip_addr.serialize(),
            self.target_hw_addr.serialize(),
            self.target_ip_addr.serialize(),
        ))
    }
}

Well that wasn't so bad.

But wait, we're not done!

So far, we've never sent anything over the network. But I'm pretty sure we can't just inject "ARP packets" into our network interface. I'm pretty sure we have to inject the exact thing we sniff from it: Ethernet frames.

Can we stuff ARP packets into Ethernet frames? Yes! It's EtherType 0x0806.

Let's make room for it:

// in `src/ethernet.rs`

#[derive(Debug, TryFromPrimitive, Clone, Copy)]
#[repr(u16)]
pub enum EtherType {
    IPv4 = 0x0800,
    // new!
    ARP = 0x0806,
}

use crate::arp;

#[derive(Debug)]
pub enum Payload {
    IPv4(ipv4::Packet),
    // new!
    ARP(arp::Packet),
    Unknown,
}

And add it to the parser for ethernet::Frame:

// in `src/ethernet.rs`

impl Frame {
    pub fn parse(i: parse::Input) -> parse::Result<Self> {
        context("Ethernet frame", |i| {
            let (i, (dst, src)) = tuple((Addr::parse, Addr::parse))(i)?;
            let (i, ether_type) = EtherType::parse(i)?;

            let (i, payload) = match ether_type {
                Some(EtherType::IPv4) => map(ipv4::Packet::parse, Payload::IPv4)(i)?,
                // new!
                Some(EtherType::ARP) => map(arp::Packet::parse, Payload::ARP)(i)?,
                None => (i, Payload::Unknown),
            };

            let res = Self {
                dst,
                src,
                ether_type,
                payload,
            };
            Ok((i, res))
        })(i)
    }
}

I like where this is going.

Let's write a serializer for ethernet::Payload

// in `src/ethernet.rs`

impl Payload {
    pub fn serialize<'a, W: io::Write + 'a>(&'a self) -> impl cf::SerializeFn<W> + 'a {
        use cf::sequence::tuple;
        move |out| match self {
            Self::ARP(ref packet) => tuple((EtherType::ARP.serialize(), packet.serialize()))(out),
            Self::IPv4(_) => unimplemented!(),
            Self::Unknown => unimplemented!(),
        }
    }
}

And finally, one for ethernet::Frame:

// in `src/ethernet.rs`

impl Frame {
    pub fn serialize<'a, W: io::Write + 'a>(&'a self) -> impl cf::SerializeFn<W> + 'a {
        use cf::sequence::tuple;
        tuple((
            self.dst.serialize(),
            self.src.serialize(),
            self.payload.serialize(),
        ))
    }
}

That ended up real neat.

For real this time

It's time... to craft network traffic!

We'll need the netinfo module to return the information we gathered earlier, so let's make a struct for it.

// in `src/netinfo.rs`

#[derive(Debug)]
pub struct NIC {
    pub guid: String,
    pub gateway: ipv4::Addr,
    pub address: ipv4::Addr,
    pub phy_address: ethernet::Addr,
}

Then let's change default_nic_guid() to default_nic(), and return the additional information. That one's a little verbose, but it's mostly code we've seen before:

// in `src/netinfo.rs`

pub fn default_nic() -> Result<NIC, Error> {
    let table = VLS::new(|ptr, size| GetIpForwardTable(ptr, size, false))?;
    let entry = table
        .entries()
        .iter()
        .find(|r| r.dest == ipv4::Addr([0, 0, 0, 0]))
        .ok_or(Error::DefaultRouteMissing)?;

    let ifaces = VLS::new(|ptr, size| GetInterfaceInfo(ptr, size))?;
    let iface = ifaces
        .adapters()
        .iter()
        .find(|r| r.index == entry.if_index)
        .ok_or(Error::DefaultInterfaceMissing)?;

    let addr_rows = VLS::new(|ptr, size| GetIpAddrTable(ptr, size, false))?;
    let address = addr_rows
        .entries()
        .iter()
        .find(|r| r.index == entry.if_index)
        .ok_or(Error::DefaultInterfaceNoIPAddr)?
        .addr;

    let mut adapter_list_head = VLS::new(|ptr, size| GetAdaptersInfo(ptr, size))?;
    let mut current = NonNull::new(&mut *adapter_list_head);
    let mut phy_address = None;
    loop {
        if let Some(adapter) = current {
            let adapter = unsafe { adapter.as_ref() };
            if adapter.address_length == 6 && adapter.index == entry.if_index {
                phy_address = Some(adapter.address);
                break;
            }
            current = adapter.next;
        } else {
            break;
        }
    }
    let phy_address = phy_address.ok_or(Error::DefaultInterfaceNoMACAddr)?;

    let name = iface.name.to_string();
    let guid_start = name.find("{").ok_or(Error::DefaultInterfaceUnidentified)?;
    let guid = &name[guid_start..];
    Ok(NIC {
        guid: guid.to_string(),
        address,
        phy_address,
        gateway: entry.next_hop,
    })
}

Phew.

Rest easy in the knowledge that the equivalent code in C would be much harder to follow still, and probably full of errors.

Note that we've introduced two new library-wide errors:

// in `src/error.rs`

use thiserror::Error;

#[derive(Debug, Error)]
pub enum Error {
    // omitted: existing errors

    #[error("ersatz could not determine the IP address of the default network interface")]
    DefaultInterfaceNoIPAddr,
    #[error("ersatz could not determine the MAC address of the default network interface")]
    DefaultInterfaceNoMACAddr,
}

We'll need to adjust main() a little:

// in `src/main.rs`

fn do_main() -> failure::Fallible<()> {
    color_backtrace::install();

    let lib = open_best_library()?;

    // new!
    let nic = netinfo::default_nic()?;
    println!("Using {:#?}", nic);

    let iface_name = format!(r#"\Device\NPF_{}"#, nic.guid);
    let iface = lib.open_interface(&iface_name)?;
    // end of new stuff

    println!("Listening for packets...");

    let start = Instant::now();
    iface.loop_infinite_dyn(&mut |packet| {
        process_packet(start.elapsed(), packet);
    })?;
    Ok(())
}

Our program now has the following output:

$ cargo run --quiet
Using NIC {
    guid: "{0E89380B-814A-48FC-86C4-5C51B8040CB2}",
    gateway: 192.168.1.254,
    address: 192.168.1.16,
    phy_address: F4-D1-08-0B-7E-BC,
}
Listening for packets...

I'm running out of ways to say "looks good", but the feeling remains the same.

The next thing we want to do is find the MAC address of 192.168.1.254.

For that, we'll need an ARP request packet:

// in `src/main.rs`
// in `do_main()`

{
    let arp_packet = arp::Packet {
        operation: arp::Operation::Request,
        sender_hw_addr: nic.phy_address,
        sender_ip_addr: nic.address,
        target_hw_addr: ethernet::Addr::zero(),
        target_ip_addr: nic.gateway,
    };
}

Since we're asking for the target's MAC address, it makes sense that we don't know it yet, so we just send an all-zero MAC address. I've added a convenience method for that:

// in `src/ethernet.rs`

impl Addr {
    pub fn zero() -> Self {
        Self([0, 0, 0, 0, 0, 0])
    }
}

Next up, we're going to slam that ARP packet into an Ethernet frame.

But uhh... chicken and egg problem here. Ethernet frames have a destination MAC address right? And we're uhh doing that ARP request precisely to know the MAC address of the gateway.

Lucky there's a broadcast MAC address, which means everyone on the local network will know about our request. Woo!

// in `src/ethernet.rs`

impl Addr {
    pub fn broadcast() -> Self {
        Self([0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF])
    }
}

// in `src/main.rs`
// in `do_main()`

{
    let arp_packet = arp::Packet {
        operation: arp::Operation::Request,
        sender_hw_addr: nic.phy_address,
        sender_ip_addr: nic.address,
        target_hw_addr: ethernet::Addr::zero(),
        target_ip_addr: nic.gateway,
    };

    let frame = ethernet::Frame {
        src: nic.phy_address,
        dst: ethernet::Addr::broadcast(),
        ether_type: None,
        payload: ethernet::Payload::ARP(arp_packet),
    };
    dbg!(&frame);
}

There, that ought to print a few things.

$ cargo run --quiet
Using NIC {
    guid: "{0E89380B-814A-48FC-86C4-5C51B8040CB2}",
    gateway: 192.168.1.254,
    address: 192.168.1.16,
    phy_address: F4-D1-08-0B-7E-BC,
}
[src\main.rs:53] &frame = Frame {
    dst: FF-FF-FF-FF-FF-FF,
    src: F4-D1-08-0B-7E-BC,
    payload: ARP(
        Packet {
            operation: Request,
            sender_hw_addr: F4-D1-08-0B-7E-BC,
            sender_ip_addr: 192.168.1.16,
            target_hw_addr: 00-00-00-00-00-00,
            target_ip_addr: 192.168.1.254,
        },
    ),
}

Yeah. Yeah it does print a few things.

Well that's good uh, that concludes our.. mh? Oh right, it doesn't.

Okay then. What? No, I'm not stalling for time. Scared? Me? No, you're scared. Haha. Ok, well, let's uh

    // after dbg!(&frame)

    struct AsHex<'a>(&'a [u8]);

    use std::fmt;
    impl<'a> fmt::Display for AsHex<'a> {
        fn fmt(&self, f: &mut fmt::Formatter) -> fmt::Result {
            for x in self.0 {
                write!(f, "{:02x} ", x)?;
            }
            Ok(())
        }
    }

    use cookie_factory as cf;
    let serialized = cf::gen_simple(frame.serialize(), Vec::new()).unwrap();
    println!("sending {}", AsHex(&serialized));
    iface.send(&serialized).unwrap();

and then uh

$ cargo run --quiet
(cut)
sending ff ff ff ff ff ff f4 d1 08 0b 7e bc 08 06 00 01 08 00 06 04 00 01 f4 d1 08 0b 7e bc c0 a8 01 10 00 00 00 00 00 00 c0 a8 01 fe
Listening for packets...

Ah.

Nothing's being printed.

Why is nothing being printed?

WHAT HAVE WE DONE WRONG OH GOD I KNEW THIS SERIES WAS A MISToh that's right, we only print ICMP packets.

Let's rejiggle process_packet a bit:

fn process_packet(now: Duration, packet: &BorrowedPacket) {
    let frame = match ethernet::Frame::parse(packet) {
        Ok((_remaining, frame)) => frame,
        Err(nom::Err::Error(e)) => {
            println!("{:?} | {:?}", now, e);
            return;
        }
        _ => unreachable!(),
    };

    match frame.payload {
        ethernet::Payload::IPv4(ref ip_packet) => match ip_packet.payload {
            ipv4::Payload::ICMP(ref icmp_packet) => println!(
                "{:?} | ({:?}) => ({:?}) | {:#?}",
                now, ip_packet.src, ip_packet.dst, icmp_packet
            ),
            _ => {}
        },
        ethernet::Payload::ARP(ref arp_packet) => {
            println!("{:?} | {:#?}", now, arp_packet);
        }
        _ => {}
    }
}

Hold me:

$ cargo run --quiet
(cut)
sending ff ff ff ff ff ff f4 d1 08 0b 7e bc 08 06 00 01 08 00 06 04 00 01 f4 d1 08 0b 7e bc c0 a8 01 10 00 00 00 00 00 00 c0 a8 01 fe
Listening for packets...
1.0006759s | Packet {
    operation: Request,
    sender_hw_addr: F4-D1-08-0B-7E-BC,
    sender_ip_addr: 192.168.1.16,
    target_hw_addr: 00-00-00-00-00-00,
    target_ip_addr: 192.168.1.254,
}
1.0007337s | Packet {
    operation: Reply,
    sender_hw_addr: 14-0C-76-6A-71-BD,
    sender_ip_addr: 192.168.1.254,
    target_hw_addr: F4-D1-08-0B-7E-BC,
    target_ip_addr: 192.168.1.16,
}

AAAAAAAAAAAAAAAAAAHHHHhhh it works!!!

For reference, it's the first time we're sending actual network traffic and getting a reply back.

Alright, alright. Settle down. Let's uhh collect our thoughts.

We'll want to use that MAC address later on, when we want to send an ICMP packet. I'm also pretty sure that iface.loop_infinite_dyn runs in a separate thread or something. So we'll want to have some way to be asynchronously notified that we've received our ARP reply.

Let's make some plumbing.

// in `src/main.rs`

use std::collections::HashMap;
use std::sync::mpsc;

pub struct PendingQueries {
    arp: HashMap<ipv4::Addr, mpsc::Sender<ethernet::Addr>>,
}

Since we're using ipv4::Addr as a key in our HashMap, we'll need to derive Hash for it.

// in `src/ipv4.rs`

// new: Hash!
#[derive(PartialEq, Eq, Clone, Copy, Hash)]
pub struct Addr(pub [u8; 4]);

Then, make our do_main look like this:

// in `src/main.rs`

use std::sync::Mutex;

fn do_main() -> failure::Fallible<()> {
    color_backtrace::install();

    let lib = open_best_library()?;

    let nic = netinfo::default_nic()?;
    println!("Using {:#?}", nic);

    let iface_name = format!(r#"\Device\NPF_{}"#, nic.guid);
    let iface = lib.open_interface(&iface_name)?;

    let pending = PendingQueries {
        arp: HashMap::new(),
    };
    let pending = Mutex::new(pending);

    use std::thread;
    thread::spawn(|| {
        make_queries(iface.as_ref(), &nic, &pending);
    });

    thread::spawn(|| {
        iface
            .loop_infinite_dyn(&mut |packet| {
                process_packet(&pending, packet);
            })
            .unwrap();
    });

    Ok(())
}

Cool bear's hot tip

We're going over this whole thread business a little quickly here, but no worries, we'll discuss that in more in-depth in Part 14.

make_queries will be in charge of doing the ARP query and waiting for the result:

fn make_queries(
    iface: &dyn rawsock::traits::DynamicInterface,
    nic: &netinfo::NIC,
    pending: &Mutex<PendingQueries>,
) {
    let arp_packet = arp::Packet {
        operation: arp::Operation::Request,
        sender_hw_addr: nic.phy_address,
        sender_ip_addr: nic.address,
        target_hw_addr: ethernet::Addr::zero(),
        target_ip_addr: nic.gateway,
    };

    // the result will be sent to `tx` (on another thread)
    // and received through `rx` (on this thread)
    let (tx, rx) = mpsc::channel();
    {
        // since `pending` is shared between threads, it's behind
        // a `Mutex`, and we need to acquire the lock before doing
        // anything with it:
        let mut pending = pending.lock().unwrap();
        // this marks that we're waiting for an ARP reply
        // that gives us the MAC address corresponding to
        // that IP address.
        pending.arp.insert(arp_packet.target_ip_addr, tx);
    }

    // send it over Ethernet, same as before
    let frame = ethernet::Frame {
        src: nic.phy_address,
        dst: ethernet::Addr::broadcast(),
        ether_type: None,
        payload: ethernet::Payload::ARP(arp_packet),
    };

    use cookie_factory as cf;
    let serialized = cf::gen_simple(frame.serialize(), Vec::new()).unwrap();
    iface.send(&serialized).unwrap();

    // this blocks until something is sent over `tx`
    let gateway_phy_address = rx.recv().unwrap();
    println!("gateway physical address: {:?}", gateway_phy_address);
}

We also need to change process_packet to read from pending and write the result to the Sender, if we're expecting an ARP reply:

// new: `pending` parameter
fn process_packet(pending: &Mutex<PendingQueries>, packet: &BorrowedPacket) {
    let frame = match ethernet::Frame::parse(packet) {
        Ok((_remaining, frame)) => frame,
        Err(nom::Err::Error(e)) => {
            println!("{:?}", e);
            return;
        }
        _ => unreachable!(),
    };

    match frame.payload {
        ethernet::Payload::IPv4(ref ip_packet) => match ip_packet.payload {
            ipv4::Payload::ICMP(ref icmp_packet) => println!(
                "({:?}) => ({:?}) | {:#?}",
                ip_packet.src, ip_packet.dst, icmp_packet
            ),
            _ => {}
        },
        ethernet::Payload::ARP(ref arp_packet) => {
            // note: we don't print ARP packets anymore
            if let arp::Operation::Reply = arp_packet.operation {
                let mut pending = pending.lock().unwrap();
                if let Some(tx) = pending.arp.remove(&arp_packet.sender_ip_addr) {
                    tx.send(arp_packet.sender_hw_addr).unwrap();
                }
            }
        }
        _ => {}
    }
}

Now, if we try to compile this, we're going to get a bunch of errors. Here's just one of them:

As always, rustc is absolutely on the money.

We're starting threads, and anything the closure we pass to std::thread::start captures is captured for 'static, ie. "the entire lifetime of the program". Sure, we could join them (wait for the thread to exit), and then they'd "stop borrowing", but the borrow checker doesn't know about that.

Now, we don't want to give the 'static lifetime to our variables, so instead we can use the crossbeam crate, which gives us scoped threads.

$ cargo add crossbeam-utils
      Adding crossbeam-utils v0.7.0 to dependencies

And instead, we can write this:

fn do_main() -> failure::Fallible<()> {
    color_backtrace::install();

    let lib = open_best_library()?;

    let nic = netinfo::default_nic()?;
    println!("Using {:#?}", nic);

    let iface_name = format!(r#"\Device\NPF_{}"#, nic.guid);
    let iface = lib.open_interface(&iface_name)?;

    let pending = PendingQueries {
        arp: HashMap::new(),
    };
    let pending = Mutex::new(pending);

    crossbeam_utils::thread::scope(|s| {
        s.spawn(|_| {
            make_queries(iface.as_ref(), &nic, &pending);
        });

        s.spawn(|_| {
            iface
                .loop_infinite_dyn(&mut |packet| {
                    process_packet(&pending, packet);
                })
                .unwrap();
        });
    })
    .unwrap();

    Ok(())
}

Which is ostensibly the same thing, except by the time scope() returns, all threads spawned inside have exited - so anything captured from the outside of the scope is only borrowed for the lifetime of the scope() invocation.

And suddenly, everything starts working:

$ cargo run --quiet
Using NIC {
    guid: "{0E89380B-814A-48FC-86C4-5C51B8040CB2}",
    gateway: 192.168.1.254,
    address: 192.168.1.16,
    phy_address: F4-D1-08-0B-7E-BC,
}
gateway physical address: 14-0C-76-6A-71-BD

Alright!

Now, when I see code like that, my refactoring senses tingle.

I can think of a few methods that would make that code shorter.

If we add the following to our ethernet package:

impl Payload {
    pub fn as_frame(self, nic: &crate::netinfo::NIC, dst: Addr) -> Frame {
        Frame {
            src: nic.phy_address,
            dst,
            ether_type: None,
            payload: self,
        }
    }

    pub fn as_broadcast_frame(self, nic: &crate::netinfo::NIC) -> Frame {
        self.as_frame(nic, Addr::broadcast())
    }
}

impl Frame {
    pub fn send(&self, iface: &dyn rawsock::traits::DynamicInterface) {
        let serialized = cf::gen_simple(self.serialize(), Vec::new()).unwrap();
        iface.send(&serialized).unwrap();
    }
}

And the following to our arp package:

impl Packet {
    pub fn request(nic: &crate::netinfo::NIC, target_ip_addr: ipv4::Addr) -> Self {
        Self {
            operation: Operation::Request,
            sender_ip_addr: nic.address,
            sender_hw_addr: nic.phy_address,
            target_ip_addr,
            target_hw_addr: ethernet::Addr::zero(),
        }
    }

    pub fn as_ethernet_payload(self) -> ethernet::Payload {
        ethernet::Payload::ARP(self)
    }
}

And we forgive cavalier error handling for the time being...

Then we can change our code to this:

fn make_queries(
    iface: &dyn rawsock::traits::DynamicInterface,
    nic: &netinfo::NIC,
    pending: &Mutex<PendingQueries>,
) {
    let gateway_ip = nic.gateway;

    let (tx, rx) = mpsc::channel();
    pending.lock().unwrap().arp.insert(gateway_ip, tx);

    arp::Packet::request(nic, gateway_ip)
        .as_ethernet_payload()
        .as_broadcast_frame(nic)
        .send(iface);

    let gateway_mac = rx.recv().unwrap();
    println!("gateway MAC: {:?}", gateway_mac);
}

And, I don't know about you, but I could look at that all damn day.

You're reading the Making our own ping series.

1

2

3

4

5

6

7

8

9

10

11

12

13

14

Comment on /r/fasterthanlime

(JavaScript is required to see this. Or maybe my stuff broke)

Here's another article just for you:

When rustc explodes

Jul 13, 2022

32 min #rust · #tracing · #investigation

One could say I have a bit of an obsession with build times.

I believe having a "tight feedback loop" is extremely valuable: when I work on a large codebase, I want to be able to make small incremental changes and check very often that things are going as expected.

Especially if I'm working on a project that needs to move quickly: say, the product for an early-stage startup, or a side-project for which I only ever get to do 1-hour work bursts at most.