Everything about rust

Welcome back to the eighteenth and final part of "Making our own executable packer".

In the last article, we had a lot of fun. We already had a "packer" executable, minipak, which joined together stage1 (a launcher), and a compressed version of whichever executable we wanted to pack.

Here's a sentence I find myself saying several times a week:

...or we could just box it.

There's two remarkable things about this sentence.

I'd like to think that my understanding of "async Rust" has increased over the past year or so. I'm 100% onboard with the basic principle: I would like to handle thousands of concurrent tasks using a handful of threads. That sounds great!

Welcome back!

In the last article, we did foundational work on minipak, our ELF packer.

It is now able to receive command-line arguments, environment variables, and auxiliary vectors. It can parse those command-line arguments into a set of options. It can make an ELF file smaller using the LZ4 compression algorithm, and pack it together with stage1, our launcher.

And we're back!

In the last article, we thanked our old code and bade it adieu, for it did not spark joy. And then we made a new, solid foundation, on which we planned to actually make an executable packer.

You're still here! Fantastic.

I have good news, and bad news. The good news is, we're actually going to make an executable packer now!

Good morning, and welcome back to "how many executables can we run with our custom dynamic loader before things get really out of control".

It's time for the Day 14 problem!

After the hassle that was Day 13, I hope this time we'll have a relatively chill time. And, at least for Part 1, that is true.

In the Day 13 problem, we're trying to take the bus.

Our input looks like this:

939
7,13,x,x,59,x,31,19

The first line indicates the earliest minute we can leave from the bus terminal, and the second line indicates the "identifier" of the buses that are active.

Time for the Day 12 problem!

In this problem, we have a ship. And we have navigation instructions:

• Action N means to move north by the given value.
• Action S means to move south by the given value.
• Action E means to move east by the given value.
• Action W means to move west by the given value.
• Action L means to turn left the given number of degrees.
• Action R means to turn right the given number of degrees.
• Action F means to move forward by the given value in the direction the ship is currently facing.

Another day, another problem.

This time the problem looks suspiciously like Conway's Game of Life, or, I guess, any old Cellular automaton.

Okay, Day 10.

Again, the problem statement is very confusing - but what it all boils down to is this. We have a list of numbers:

16
10
15
5
1
11
7
19
6
12
4

Day 9's problem statement is convoluted - the "ah maybe that's why I don't usually do Advent of Code" kind of convoluted, but let's give it a go anyway.

Time for another Advent of Code 2020 problem!

That one sounds like it's going to be fun. Our input is pretty much assembly, like this:

nop +0
acc +1
jmp +4
acc +3
jmp -3
acc -99
acc +1
jmp -4
acc +6

Another day, another Advent of Code 2020 problem.

That one seems fun! For some nerdy values of fun.

Our input is a set of rules:

light red bags contain 1 bright white bag, 2 muted yellow bags.
dark orange bags contain 3 bright white bags, 4 muted yellow bags.
bright white bags contain 1 shiny gold bag.
muted yellow bags contain 2 shiny gold bags, 9 faded blue bags.
shiny gold bags contain 1 dark olive bag, 2 vibrant plum bags.
dark olive bags contain 3 faded blue bags, 4 dotted black bags.
vibrant plum bags contain 5 faded blue bags, 6 dotted black bags.
faded blue bags contain no other bags.
dotted black bags contain no other bags.

The end of Advent of Code 2020 is fast approaching, and we're nowhere near done. Time to do Day 6!

The problem statement here is a little contrived, as uh, as the days that came before it, but that won't stop us.

Time for another day of Advent of Code 2020.

For Day 5, we're going to have to do...

It's time for Day 4 of the Advent of Code 2020!

Now, I've already had a look at the problem statement, at least for part 1, and I'm not particularly excited.

Hello all, and welcome back to Advent of Code 2020, featuring Cool Bear.

Let's get right to it.

The problem statement for Day 3 is as follows: we're given a map, that looks like this:

..##.......
#...#...#..
.#....#..#.
..#.#...#.#
.#...##..#.
..#.##.....
.#.#.#....#
.#........#
#.##...#...
#...##....#
.#..#...#.#

The Advent of Code 2020, Day 2 problem talks about passwords. Sounds familiar.

Basically, our input looks like this:

1-3 a: abcde
1-3 b: cdefg
2-9 c: ccccccccc

I was not planning on doing anything specific this December, but a lot of folks around me (on Twitter, at work) have chosen this Advent of Code to pick up Rust, and I've got big FOMO energy, so, let's see where this goes.

The Nature weekly journal of science was first published in 1869. And after one and a half century, it has finally completed one cycle of carcinization, by publishing an article about the Rust programming language.

Good morning! It is still 2020, and the world is literally on fire, so I guess we could all use a distraction.

This article continues the tradition of me getting shamelessly nerd-sniped - once by Pascal about small strings, then again by a twitch viewer about Rust enum sizes.

During a recent Rust Q&A Session on my twitch channel, someone asked a question that seemed simple: why are small string types, like SmartString or SmolStr, the same size as String, but small vec types, like SmallVec, are larger than Vec?

Learning Rust is... an experience. An emotional journey. I've rarely been more frustrated than in my first few months of trying to learn Rust.

As I've said before, I'm working on a book about lifetimes. Or maybe it's just a long series - I haven't decided the specifics yet. Like every one of my series/book things, it's long, and it starts you off way in the periphery of the subject, and takes a lot of detours to get there.

I used to be afraid of async Rust. It's easy to get into trouble!

But thanks to the work done by the whole community, async Rust is getting easier to use every week. One project I think is doing particularly great work in this area is async-std.

I write a ton of articles about rust. And in those articles, the main focus is about writing Rust code that compiles. Once it compiles, well, we're basically in the clear! Especially if it compiles to a single executable, that's made up entirely of Rust code.

I started experimenting with asynchronous Rust code back when futures 0.1 was all we had - before async/await. I was a Rust baby then (I'm at least a toddler now), so I quickly drowned in a sea of .and_then, .map_err and Either<A, B>.

Hey everyone!

This article is brought to you by a shameless nerd snipe, courtesy of Pascal.

In case you've blocked Twitter for your own good, this reads:

Since I write a lot of articles about Rust, I tend to get a lot of questions about specific crates: "Amos, what do you think of oauth2-simd? Is it better than openid-sse4? I think the latter has a lot of boilerplate."

I've been banging the same drum for years: APIs must be carefully designed.

This statement doesn't resonate the same way with everyone. In order to really understand what I mean by "careful API design", one has to have experienced both ends of the spectrum.

Hi everyone. Has it been two months since I last posted something? Yes it has!

That seems like a nice round duration, so let's break the silence with a few announcements.

I have a new website

Welcome back and thanks for joining us for the reads notes... the thirteenth installment of our series on ELF files, what they are, what they can do, what does the dynamic linker do to them, and how can we do it ourselves.

In Part 11, we spent some time clarifying mechanisms we had previously glossed over: how variables and functions from other ELF objects were accessed at runtime.

In our last installment of "Making our own executable packer", we did some code cleanups. We got rid of a bunch of unsafe code, and found a way to represent memory-mapped data structures safely.

Welcome back to the "Making our own executable packer" series, where digressions are our bread and butter.

Last time, we implemented indirect functions in a no-libc C program. Of course, we got lost on the way and accidentally implemented a couple of useful elk-powered GDB functions - with only the minimal required amount of Python code.

In the last article, we cleaned up our dynamic linker a little. We even implemented the Dynamic relocation.

But it's still pretty far away from running real-world applications.

There's a question that always comes up when people pick up the Rust programming language: why are there two string types? Why is there String, and &str?

Let's pick up where we left off: we had just taught elk to load not only an executable, but also its dependencies, and then their dependencies as well.

In order to increase fluency in a programming language, one has to read a lot of it. But how can you read a lot of it if you don't know what it means?

Up until now, we've been loading a single ELF file, and there wasn't much structure to how we did it: everyhing just kinda happened in main, in no particular order.

In our last article, we managed to load and execute a PIE (position-independent executable) compiled from the following code:

X86 Assembly
; in `samples/hello-pie.asm`

        global _start

        section .text

_start: mov rdi, 1      ; stdout fd
        lea rsi, [rel msg]
        mov rdx, 9      ; 8 chars + newline
        mov rax, 1      ; write syscall
        syscall

        xor rdi, rdi    ; return code 0
        mov rax, 60     ; exit syscall
        syscall

        section .data

msg:    db "hi there", 10

The last article, Position-independent code, was a mess. But who could blame us? We looked at the world, and found it to be a chaotic and seemingly nonsensical place. So, in order to blend in, we had to let go of a little bit of sanity.

In the last article, we found where code was hiding in our samples/hello executable, by disassembling the whole file and then looking for syscalls.

In part 1, we've looked at three executables:

• sample, an assembly program that prints "hi there" using the write system call.
• entry_point, a C program that prints the address of main using printf
• The /bin/true executable, probably also a C program (because it's part of GNU coreutils), and which just exits with code 0.

Executables have been fascinating to me ever since I discovered, as a kid, that they were just files. If you renamed a .exe to something else, you could open it in notepad! And if you renamed something else to a .exe, you'd get a neat error dialog.

So. Serializing IPv4 packets. Easy? Well, not exactly.

IPv4 was annoying to parse, because we had 3-bit integers, and 13-bit integers, and who knows what else. Serializing it is going to be exactly the same.

Alright. ALRIGHT. I know, we're all excited, but let's think about what we're doing again.

So we've managed to look at real network traffic and parse it completely. We've also taken some ICMP packets, parsed them, and then serialized them right back and we got the exact same result.

In the last part, we've finally parsed some IPv4 packets. We even found a way to filter only IPv4 packets that contain ICMP packets.

Hello and welcome to Part 11 of this series, wherein we finally use some of the code I prototyped way back when I was planning this series.

Where are we standing?

Before we move on to parsing more of our raw packets, I want to take some time to improve our error handling strategy.

Currently, the ersatz codebase contains a mix of Result<T, E>, and some methods that panic, like unwrap() and expect().

Blog posts that praise Rust are many but funding is generally in short supply.

If even a small percentage of the money Rust saves companies was put back into the ecosystem it would help secure the future of the platform tremendously.

Multiple sources of funding

Now that we've found the best way to find the "default network interface"... what can we do with that interface?

Okay, I lied.

I'm deciding - right this instant - that using wmic is cheating too. Oh, it was fair game when we were learning about Windows, but we're past that now.

Let's set aside our sup project for a while.

Don't get me wrong - it's a perfectly fine project, and, were we simply rewriting "ping" for Windows in Rust, we could (almost) stop there.

Our ping API is simple, but it's also very limited:

Rust code
pub fn ping(dest: ipv4::Addr) -> Result<(), String>

// called as:
ping(ipv4::Addr([8, 8, 8, 8])).unwrap();

We've just spent a lot of time abstracting over LoadLibrary, but we still have all the gory details of the Win32 ICMP API straight in our main.rs file! That won't do.

It's refactor time!

Our complete program is now about a hundred lines, counting blank lines (see the end of part 3 for a complete listing).

It's time to make sup, our own take on ping, use the Win32 APIs to send an ICMP echo. Earlier we discovered that Windows's ping.exe used IcmpSendEcho2Ex. But for our purposes, the simpler IcmpSendEcho will do just fine.

So, how does ping.exe actually send a ping? It seems unrealistic that ping.exe itself implements all the protocols involved in sending a ping. So it must be calling some sort of library. Also, since it ends up talking to the outside world via a NIC (network interface controller), the kernel is probably involved at some point.

It feels like an eternity since I've started using Rust, and yet I remember vividly what it felt like to bang my head against the borrow checker for the first few times.

So far, we've seen many ways to read a file from different programming languages, we've learned about syscalls, how to make those from assembly, then we've learned about memory mapping, virtual address spaces, and generally some of the mechanisms in which userland and the kernel interact.

Everybody knows how to use files. You just open up File Explorer, the Finder, or a File Manager, and bam - it's chock-full of files. There's folders and files as far as the eye can see. It's a genuine filapalooza. I have never once heard someone complain there were not enough files on their computer.

A while back, I asked on Twitter what people found confusing in Rust, and one of the top topics was "how the module system maps to files".

In my previous article, I said I needed to stop thinking of Rust generics as Java generics, because in Rust, generic types are erased.

I've been looking for this blog post everywhere, but it doesn't exist, so I guess it's my turn to write about Some Fun with Rust.

The task at hand