fasterthanlime

Aug 31, 2019 27 min #os · #golang · #strace · #kernel · #assembly

Reading files the hard way - Part 2 (x86 asm, linux kernel)

Looking at that latest mental model, it's.. a bit suspicious that every program ends up calling the same set of functions. It's almost like something different happens when calling those.

Are those even regular functions? Can we step through them with a debugger?

If we run our stdio-powered C program in gdb, and break on read, we can confirm that we indeed end up calling a read function (which is called __GI___libc_read here, but oh well):

$ gcc -g -O0 readfile-f.c -o readfile-f

$ gdb --silent ./readfile-f
Reading symbols from ./readfile-f...
(gdb) break read
Function "read" not defined.
Make breakpoint pending on future shared library load? (y or [n]) y
Breakpoint 1 (read) pending.
(gdb) r
Starting program: /home/amos/bearcove/read-files-the-hard-way/readfile-f
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".

Breakpoint 1, __GI___libc_read (fd=3, buf=0x5555555594a0, nbytes=4096) at ../sysdeps/unix/sysv/linux/read.c:25
25      ../sysdeps/unix/sysv/linux/read.c: No such file or directory.
(gdb) bt
#0  __GI___libc_read (fd=3, buf=0x5555555594a0, nbytes=4096) at ../sysdeps/unix/sysv/linux/read.c:25
#1  0x00007ffff7c88596 in _IO_new_file_underflow (fp=0x5555555592a0) at ./libio/libioP.h:947
#2  0x00007ffff7c86e00 in __GI__IO_file_xsgetn (fp=0x5555555592a0, data=<optimized out>, n=15) at ./libio/fileops.c:1321
#3  0x00007ffff7c7b709 in __GI__IO_fread (buf=0x555555559480, size=1, count=15, fp=0x5555555592a0) at ./libio/iofread.c:38
#4  0x0000555555555288 in main (argc=1, argv=0x7fffffffdb98) at readfile-f.c:16
(gdb)

Same goes for the Rust program:

$ rust-gdb --silent target/debug/readfile-rs
Reading symbols from target/debug/readfile-rs...
(gdb) break read
Breakpoint 1 at 0x1ea73: file library/std/src/fs.rs, line 872.
(gdb) r
Starting program: /home/amos/bearcove/read-files-the-hard-way/readfile-rs/target/debug/readfile-rs
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".

Breakpoint 1, std::sys::unix::fs::OpenOptions::read () at library/std/src/sys/unix/fs.rs:893
893     library/std/src/sys/unix/fs.rs: No such file or directory.
(gdb) c
Continuing.

Breakpoint 1, __GI___libc_read (fd=3, buf=0x5555555a8ba0, nbytes=220) at ../sysdeps/unix/sysv/linux/read.c:25
25      ../sysdeps/unix/sysv/linux/read.c: No such file or directory.
(gdb) bt
#0  __GI___libc_read (fd=3, buf=0x5555555a8ba0, nbytes=220) at ../sysdeps/unix/sysv/linux/read.c:25
#1  0x000055555557454f in std::sys::unix::fd::FileDesc::read_buf () at library/std/src/sys/unix/fd.rs:136
#2  std::sys::unix::fs::File::read_buf () at library/std/src/sys/unix/fs.rs:1062
#3  std::fs::{impl#5}::read_buf () at library/std/src/fs.rs:737
#4  std::io::default_read_to_end<std::fs::File> () at library/std/src/io/mod.rs:376
#5  0x000055555557297d in std::io::default_read_to_string::{closure#0}<std::fs::File> () at library/std/src/io/mod.rs:430
#6  std::io::append_to_string<std::io::default_read_to_string::{closure_env#0}<std::fs::File>> () at library/std/src/io/mod.rs:338
#7  std::io::default_read_to_string<std::fs::File> () at library/std/src/io/mod.rs:430
#8  std::fs::{impl#5}::read_to_string () at library/std/src/fs.rs:754
#9  0x000055555555c899 in readfile_rs::main () at src/main.rs:9

However, when we try to step through it... nothing. For the sake of the investigation, I cloned the glibc repository (since that's where the read function seems to live), and found this:

// in `glibc/sysdeps/unix/sysv/linux/read.c`

/* Read NBYTES into BUF from FD.  Return the number read or -1.  */
ssize_t
__libc_read (int fd, void *buf, size_t nbytes)
{
  return SYSCALL_CANCEL (read, fd, buf, nbytes);
}
libc_hidden_def (__libc_read)

libc_hidden_def (__read)
weak_alias (__libc_read, __read)
libc_hidden_def (read)
weak_alias (__libc_read, read)

The reason we can't find the source of read is because... it lives in another land entirely:

It's not just two different sets of software. They run with different privileges. The Linux kernel (and its device drivers) run in ring 0, where everything is allowed. Userland applications, however run in ring 3.

This is a classic diagram, so I had to show it here, but I don't think it's super intuitive. I prefer to think of it this way:

Because what you can do from ring 0 is a strict superset of what you can do from ring 3. Ring 3 is like a prison. Anybody from ring 0 can visit, but ring 3 can only send letters (ie. make syscalls).

So the kernel handles things like reading and writing. But it also handles things like processes. When we start our application, it runs in a process - and the kernel decides which process gets to run and when. It interrupts processes and resumes them, prioritizing the important stuff - but also, giving the illusion that a single CPU core can do multiple things at once (when it really can only do one at a time).

There's a lot the kernel is responsible for, but let's focus on files. Processes have resources associated to it - like file descriptors! When we open a file (with the open syscall), the kernel:

• Decides whether or not this is allowed
• Asks the VFS who's responsible for this particular path
• Reserves a file descriptor, which is:
  • just a number, really
  • unique per-process
• Makes a note that this number correspond to that resource
• Tells us what the number is

And in our further communication with the kernel, whenever we want to refer to that resource, we'll just use that number.

And this answers one of the questions you might have had while following this article: in the strace output for most programs, we saw a call to close (which, well, closes a file descriptor) - but in our sample C program, we never bothered calling it!

This is because the kernel, in its infinite wisdom, keeps tracks of opened file descriptors, and cleans them all up once the process exits.

"This is all getting a bit theoretical" whispers someone in the back. "I'm glad he's not showing us kernel code, but.. are we just supposed to trust that the kernel cleans up file descriptors?"

Good question! There is a command to list open file descriptors for a specific path, so we can verify that real quick with a simple C program:

// in `open.c`

#include <stdio.h>
#include <fcntl.h>

int main(int argc, char **argv) {
    int fd = open("/etc/hosts", O_RDONLY);

    printf("Our file descriptor for /etc/hosts is %d\n", fd);
    printf("Press enter to exit...\n");
    getc(stdin);
}

$ gcc open.c -o open

$ lsof /etc/hosts 2> /dev/null

$ ./open
Our file descriptor for /etc/hosts is 3
Press enter to exit...
^Z
[1]+  Stopped                 ./open

$ lsof /etc/hosts 2> /dev/null
COMMAND     PID USER   FD   TYPE DEVICE SIZE/OFF     NODE NAME
open    1917875 amos    3r   REG    8,3      220 16253092 /etc/hosts

$ fg
./open


$ lsof /etc/hosts 2> /dev/null

We need to clear up a potential source of confusion. We saw a read() function in the source code for glibc (the C library that ships with most Linux distributions), but it is distinct from the actual read syscall.

It seems like most of Unix is written in C, but can we make a syscall without using libc? Something like this:

Hey, Go is not C - does Go use libc to make syscalls? Let's find out.

This is the source code for a simple Go program that prints the contents of /etc/hosts:

package main

import (
  "io/ioutil"
  "fmt"
)

func main() {
  payload, err := ioutil.ReadFile("/etc/hosts")
  if err != nil {
    panic(err)
  }

  fmt.Printf("%s\n", string(payload))
}

It sure works, but it doesn't seem to link against libc:

$ go build main.go

$ ./main | head -3
127.0.0.1       localhost
127.0.1.1       sonic


$ ldd main
        not a dynamic executable

Trying to break on read in gdb also gives nothing. Well, can we make sure it still uses the openat and read syscalls at least? Let's strace it:

$ strace -e openat,read,write ./main
openat(AT_FDCWD, "/sys/kernel/mm/transparent_hugepage/hpage_pmd_size", O_RDONLY) = 3
read(3, "2097152\n", 20)                = 8
--- SIGURG {si_signo=SIGURG, si_code=SI_TKILL, si_pid=1923780, si_uid=1000} ---
--- SIGURG {si_signo=SIGURG, si_code=SI_TKILL, si_pid=1923780, si_uid=1000} ---
openat(AT_FDCWD, "/etc/hosts", O_RDONLY|O_CLOEXEC) = 3
read(3, "127.0.0.1\tlocalhost\n127.0.1.1\tso"..., 512) = 220
read(3, "", 292)                        = 0
write(1, "127.0.0.1\tlocalhost\n127.0.1.1\tso"..., 221127.0.0.1 localhost
127.0.1.1       sonic

# The following lines are desirable for IPv6 capable hosts
::1     ip6-localhost ip6-loopback
fe00::0 ip6-localnet
ff00::0 ip6-mcastprefix
ff02::1 ip6-allnodes
ff02::2 ip6-allrouters

) = 221
+++ exited with 0 +++

It does! So one doesn't need libc to make a syscall. What a relief.

So we've seen that pretty much all languages, no matter how many levels of abstractions they're on, have to eventually make syscalls one way or the other.

But how does one make a syscall? So far we've been using languages that either:

• Use libc to make syscalls, via wrapper functions (Node.js, Rust, C)
• Make syscalls for us (Go)

Let's try to make a syscall ourselves, in assembly.

We'll use yasm, because everything else gives me high blood pressure. Our code is going to be in readfile.asm, and we're going to build it with this makefile:

.PHONY: all

all:
	yasm -f elf64 -g dwarf2 readfile.asm
	ld readfile.o -o readfile

The yasm invocation assembles our assembly into an object file, and the ld invocation links it into a full executable.

We're going to start with a very simple program, let's put this in readfile.asm:

            global _start ; _start is our entry point - this is its declaration...

            section .text ; the text section is where we'll put executable code
_start:     xor rdi, rdi  ; ...and this is its definition. we just set rdi to 0.

This compiles and links just fine, but it segfaults when we run it:

$ make
yasm -f elf64 -g dwarf2 readfile.asm
ld readfile.o -o readfile

$ ./readfile
Segmentation fault (core dumped)

To get our program to exit cleanly, we need to.. make a syscall.

The first thing we need to make a syscall is its number. On Ubuntu for example, you can find it in a header file:

// in `/usr/include/x86_64-linux-gnu/asm/unistd_64.h`

// (cut)
#define __NR_exit 60
// (cut)

But you can also find some syscall tables online, like Filippo's, which is searchable.

The second thing we need to do is.. count our lucky stars, because on x86_64, there is a dedicated instruction to make a syscall. (It's called syscall).

So now we just need to put the syscall number 60 in the rax register, and use the syscall instruction, and we should be good:

            global _start

            section .text
_start:     mov rax, 60
            syscall

$ make
yasm -f elf64 -g dwarf2 readfile.asm
ld readfile.o -o readfile

$ ldd ./readfile
        not a dynamic executable

And just like that, we made a syscall without using libc! That wasn't so hard.

Can we re-implement our whole readfile application in assembly? Let's see.

Logging isn't going to be as easy as with Node.js, or Rust, or C, or Go. So we're going to have to lean on the debugger a little bit. Thanks to the -g dwarf2 flags we passed to yasm, we have great debug information:

$ gdb --silent ./readfile
Reading symbols from ./readfile...
(gdb) break _start
Breakpoint 1 at 0x401000: file readfile.asm, line 4.
(gdb) r
Starting program: /home/amos/bearcove/read-files-the-hard-way/readfile

Breakpoint 1, _start () at readfile.asm:4
4       _start:     mov rax, 60
(gdb) s
5                   syscall
(gdb) s
[Inferior 1 (process 1843089) exited normally]
(gdb)

So, let's try the open syscall. It needs the same parameters as in C: first a path, then a set of flags. We'll store the path in the data section.

            global _start

            section .text
_start:     mov     rax, 2      ; "open" syscall
            mov     rdi, path   ; arg 1: path
            xor     rsi, rsi    ; arg 2: flags (0 = O_RDONLY)
            syscall

            mov     rax, 60     ; "exit" syscall
            syscall

            section .data
path:       db      "/etc/hosts", 0 ; null-terminated

Stepping through this with gdb, we can make sure open succeeded, by using lsof in another terminal:

# GDB session
$ gdb --silent ./readfile
Reading symbols from ./readfile...
(gdb) starti
Starting program: /home/amos/bearcove/read-files-the-hard-way/readfile

Program stopped.
_start () at readfile.asm:4
4       _start:     mov     rax, 2      ; "open" syscall
(gdb) s
5                   mov     rdi, path   ; arg 1: path
(gdb)
6                   xor     rsi, rsi    ; arg 2: flags (0 = O_RDONLY)
(gdb)
7                   syscall
(gdb)
9                   mov     rax, 60     ; "exit" syscall
(gdb)

# In another shell
$ lsof /etc/hosts
COMMAND      PID USER   FD   TYPE DEVICE SIZE/OFF     NODE NAME
readfile 1846055 amos    3r   REG    8,3      220 16253092 /etc/hosts

We can also use strace on our resulting binary. It shows whether syscalls succeed or fail, so that works out great:

$ strace ./readfile
execve("./readfile", ["./readfile"], 0x7fffb827f0f0 /* 63 vars */) = 0
open("/etc/hosts", O_RDONLY)            = 3
exit(4202496)                           = ?
+++ exited with 0 +++

Woops, it looks like we're not exiting with status code 0, let's fix that:

            mov     rax, 60
            xor     rdi, rdi    ; <--- exit with code 0
            syscall

That's better. Now let's try reading some bytes from this file descriptor. The return value of open is stored in rax, like all other syscalls.

We're going to be using rax to make our next syscall, so we need to save it to the stack with push. Also, we need to allocate memory for our buffer - let's allocate 16 bytes on the stack.

global _start

            section .text
_start:     mov     rax, 2      ; "open"
            mov     rdi, path   ;
            xor     rsi, rsi    ; O_RDONLY
            syscall

            push rax            ; push file descriptor onto stack
            sub rsp, 16         ; reserve 16 bytes of memory

            xor     rax, rax    ; "read"
            mov     rdi, [rsp+16] ; file descriptor
            mov     rsi, rsp    ; address of buffer
            mov     rdx, 16     ; size of buffer
            syscall

            mov     rax, 60     ; "exit" syscall
            syscall

            section .data
path:       db      "/etc/hosts", 0 ; null-terminated

Running our program still prints nothing so far, but strace lets us know that everything went fine:

$ make
yasm -f elf64 -g dwarf2 readfile.asm
ld readfile.o -o readfile

$ ./readfile

$ strace ./readfile
execve("./readfile", ["./readfile"], 0x7ffc3dfee7d0 /* 60 vars */) = 0
open("/etc/hosts", O_RDONLY)            = 3
read(3, "127.0.0.1\tlocalh", 16)        = 16
exit(3)                                 = ?
+++ exited with 3 +++

$

Now let's print that buffer to stdout by using the write call.

            xor     rax, rax    ; "read"
            mov     rdi, [rsp+16] ; file descriptor
            mov     rsi, rsp    ; address of buffer
            mov     rdx, 16     ; size of buffer
            syscall

            ; `rax` contains the number of bytes read
            ; write takes the number of bytes to write via `rdx`
            mov     rdx, rax    ; number of bytes
            mov     rax, 1      ; "write"
            mov     rdi, 1      ; file descriptor (stdout)
            mov     rsi, rsp    ; address of buffer
            syscall

And we're finally seeing some output:

$ make
yasm -f elf64 -g dwarf2 readfile.asm
ld readfile.o -o readfile

$ ./readfile
127.0.0.1       localh

Finally, we just need to repeat reading and writing until read returns 0 bytes read. (We won't be doing any error checking).

Here's our final program:

            global _start

            section .text
_start:     mov     rax, 2      ; "open"
            mov     rdi, path   ;
            xor     rsi, rsi    ; O_RDONLY
            syscall

            push rax            ; push file descriptor onto stack
            sub rsp, 16         ; reserve 16 bytes of memory

read_buffer:
            xor     rax, rax    ; "read"
            mov     rdi, [rsp+16] ; file descriptor
            mov     rsi, rsp    ; address of buffer
            mov     rdx, 16     ; size of buffer
            syscall

            test    rax, rax
            ; jz means 'jump if zero'
            jz      exit

            mov     rdx, rax    ; number of bytes
            mov     rax, 1      ; "write"
            mov     rdi, 1      ; file descriptor (stdout)
            mov     rsi, rsp    ; address of buffer
            syscall

            jmp read_buffer

exit:
            mov     rax, 60     ; "exit"
            xor     rdi, rdi    ; return code 0
            syscall

            section .data
path:       db      "/etc/hosts", 0 ; null-terminated

Let's try and use strace on another program: ripgrep.

$ strace rg 'localhost' /etc/hosts
(cut)
openat(AT_FDCWD, "/etc/hosts", O_RDONLY|O_CLOEXEC) = 3
statx(3, "", AT_STATX_SYNC_AS_STAT|AT_EMPTY_PATH, STATX_ALL, {stx_mask=STATX_ALL|STATX_MNT_ID, stx_attributes=0, stx_mode=S_IFREG|0644, stx_size=220, ...}) = 0
mmap(NULL, 220, PROT_READ, MAP_SHARED, 3, 0) = 0x7fd3a694a000
write(1, "\33[0m\33[32m1\33[0m:127.0.0.1\t\33[0m\33[1"..., 511:127.0.0.1        localhost) = 51
write(1, "\n", 1
)                       = 1
write(1, "\33[0m\33[32m5\33[0m:::1     ip6-\33[0m\33"..., 665:::1     ip6-localhost ip6-loopback) = 66
write(1, "\n", 1
)                       = 1
munmap(0x7fd3a694a000, 220)             = 0
close(3)                                = 0
(cut)

We recognize the openat syscall, and also statx - but.. it doesn't use read. What's happening over here?

Well, remember when we said the kernel is an all-powerful overseer that controls everything the userland interacts with? That goes for memory too!

In an operating system like Linux, each process has its own virtual address space. Some of it is mapped to physical memory, via the Memory management unit (MMU for short).

When a process is started, a few pages are reserved for its stack. (Which we used above). When allocating memory on the heap, say, with malloc, glibc's allocator asks the kernel for more pages, and keeps track of all the allocations, so that free() works properly.

First, let's check that every process does indeed have a separate address space.

We can make a first program, write.c:

#include <stdlib.h>
#include <stdio.h>

int main(int argc, char **argv) {
  // allocate 4 bytes
  int *ptr = malloc(sizeof(int));
  // write a very specific value to it
  *ptr = 0xFEEDFACE;
  // read back the value, and print the address
  printf("Wrote %x to %p\n", *ptr, ptr);
  // wait for user input
  getc(stdin);
}

There's a good chance this program will print a different address every time, but when I ran it, it printed this:

$ gcc write.c -o write
$ ./write
Wrote feedface to 0x56459a9c7260

We can use that to write a second program, read.c:

#include <stdlib.h>
#include <stdio.h>

int main(int argc, char **argv) {
    int *ptr = (int *) 0x56459a9c7260;
    printf("Read %x to %p\n", *ptr, ptr);
}

And run it:

$ gcc read.c -o read
$ ./read
[1]   6429 segmentation fault (core dumped)   ./read

What happened? 0x56459a9c7260 was a valid address in write's virtual address space, but not in read's. Attempting to read from it is an access violation, which results in the kernel sending a signal to our process, and the default handler for that signal terminates the process.

An access violation is just one type of page fault. A page fault occurs when we try to read from or write to a (virtual) address that isn't currently mapped to physical memory.

And this is precisely the trick behind mmap. When we first mmap a file, the kernel might eagerly read the first 4K of the file into a buffer of its own, and sets up the page tables so that the (userland) process can read directly from that buffer:

But once the process reads past the first 4K, then that's a page fault!

Remember, the kernel can do anything in response to a page fault: it may decide that it's an access violation, and send a signal to the process. In this case, it chooses simply to fulfill its promise that "this virtual address range contains the contents of the file", just.. not until it's needed.

The requested part of the file is actually read, a new page mapping is set:

The kernel is of course free to "page out" parts of the file, when they haven't been accessed in a while (or as soon as it wants, really!).

Can we use that from our assembly program? Sure we can!

Since we're not sure what parameters mmap needs (and which registers to put them into), we'll use this Searchable Linux Syscall table for x86 and x86_64 by @FiloSottile.

First, as usual, we'll need to open the file:

_start:     mov     rax, 2          ; "open"
            mov     rdi, path       ;
            xor     rsi, rsi        ; O_RDONLY
            syscall

Next, we want to find the size of the file in bytes, so we can pass it to mmap. We'll use the fstat syscall for that, which takes fd (a file descriptor) in register rdi, and struct stat __user* statbuf in register rsi.

To help me write the next part, I wrote a simple C program that dumps the struct's size, along with the offset of the st_size field, and two constants:

#include <stdio.h>
#include <stddef.h>
#include <sys/stat.h>
#include <sys/mman.h>

int main() {
    printf("size of stat struct: %zu\n", sizeof(struct stat));
    printf("offset of st_size  : %zu\n", offsetof(struct stat, st_size));
    printf("PROT_READ   = 0x%x\n", PROT_READ);
    printf("MAP_PRIVATE = 0x%x\n", MAP_PRIVATE);
}

Which outputs:

size of stat struct: 144
offset of st_size  : 48
PROT_READ   = 0x1
MAP_PRIVATE = 0x2

So, it looks like we'll need to allocate 144 bytes on the stack:

            mov     rdi, rax        ; fd (returned from open)
            sub     rsp, 144        ; allocate stat struct
            mov     rsi, rsp        ; address of 'struct stat'
            mov     rax, 5          ; "fstat" syscall
            syscall

And then we can feed our file descriptor, file size, and flags to mmap. Note that we can specify an address (but NULL is fine) and an offset (but 0 is fine, since we want the whole file).

            mov     rsi, [rsp+48]   ; len = file size (from 'struct stat')
            add     rsp, 144        ; free 'struct stat'
            mov     r8, rdi         ; fd (still in rdi from last syscall)
            xor     rdi, rdi        ; address = 0
            mov     rdx, 0x1        ; protection = PROT_READ
            mov     r10, 0x2        ; flags = MAP_PRIVATE
            xor     r9, r9          ; offset = 0
            mov     rax, 9          ; "mmap" syscall
            syscall

Finally, we can write out the whole file in a single write syscall:

            mov     rdx, rsi        ; count (file size from last call)
            mov     rsi, rax        ; buffer address (returned from mmap)
            mov     rdi, 1          ; fd = stdout
            mov     rax, 1          ; "write" syscall
            syscall

And there we have it:

$ make
yasm -f elf64 -g dwarf2 readfile.asm
ld readfile.o -o readfile

$ strace ./readfile > /dev/null
execve("./readfile", ["./readfile"], 0x7ffffc6ddbf0 /* 60 vars */) = 0
open("/etc/hosts", O_RDONLY)            = 3
fstat(3, {st_mode=S_IFREG|0644, st_size=220, ...}) = 0
mmap(NULL, 220, PROT_READ, MAP_PRIVATE, 3, 0) = 0x7f11fbe5a000
write(1, "127.0.0.1\tlocalhost\n127.0.1.1\tso"..., 220) = 220
exit(0)                                 = ?
+++ exited with 0 +++

In the next part, we're going to take a look within the kernel, to see how files are organized and read - and how we can find and read them by using this knowledge, bypassing as much abstraction as we can.

Thanks to my sponsors: Rufus Cable, Marcin Kołodziej, Cole Kurkowski, Michael Alyn Miller, James Rhodes, Aalekh Patel, Guy Waldman, prairiewolf, Manuel Hutter, Scott Steele, Olly Swanson, Lennart Oldenburg, Olivia Crain, Raine Tingley, David Barsky, Joonas Koivunen, Zeeger Lubsen, Henrik Tudborg, Johan Andersson, James Brown and 227 more Rufus Cable, Marcin Kołodziej, Cole Kurkowski, Michael Alyn Miller, James Rhodes, Aalekh Patel, Guy Waldman, prairiewolf, Manuel Hutter, Scott Steele, Olly Swanson, Lennart Oldenburg, Olivia Crain, Raine Tingley, David Barsky, Joonas Koivunen, Zeeger Lubsen, Henrik Tudborg, Johan Andersson, James Brown, Johan Saf, Jelle Besseling, Jean Manguy, Victor Song, Walther, Jake Demarest-Mays, Leo Shimonaka, Foxie Solutions, xales, Alan O'Donnell, Lena Schönburg, Yves, Philipp Gniewosz, Max von Forell, jer, Borys Minaiev, Marcus Griep, Justin Ossevoort, Boris Dolgov, Yuriy Taraday, Garret Kelly, Xavier Groleau, Richard Stephens, Nicholas Bishop, old.woman.josiah, Marcus Griep, Berkus Decker, Santiago Lema, Radu Matei, Jon Gjengset, Michal Hošna, Antoine Rouaze, Chris Sims, Tobias Bahls, Mark Old, Noel, Ross Williams, Romet Tagobert, Diego Roig, genny, AdrianEddy, Chris Walker, callym, Nicolas Goy, playest, Mattia Valzelli, Tom Forbes, Yann Schwartz, Josiah Bull, Torben Clasen, Matt Jackson, SeniorMars, Jack Duvall, Jan De Landtsheer, Isak Sunde Singh, Matt Heise, Andy F, Blake Johnson, qrpth, Lucille Blumire, Matt Jadczak, Andronik, Pete Bevin, Mike Cripps, Ivo Murrell, Tyler Schmidtke, Dirkjan Ochtman, Dennis Henderson, Geoff Cant, Kristoffer Winther Balling, Chris Thackrey, Jacob Cheriathundam, Zoran Zaric, Matěj Volf, Marc-Andre Giroux, Michael, Christoph Grabo, Gran PC, Daniel Strittmatter, Romain Kelifa, Chris Emery, Colin VanDervoort, Daniel Papp, Aleksandre Khokhiashvili, Chris, Paige Ruten, villem, compwhizii, Nikolai Vincent Vaags, nyangogo, anichno, Richard Pringle, budrick, Tiziano Santoro, Neil Blakey-Milner, C J Silverio, clement, avborhanian, Thor Kamphefner, Ahmad Alhashemi, Adam Gutglick, Makoto Nakashima, Chris Biscardi, Alex Doroshenko, Miguel Piedrafita, Hadrien G., Daniel Silverstone, joseph, Julian Schmid, Guillaume E, jatescher, Mario Fleischhacker, Lyssieth, Eugene Bulkin, Josh Triplett, belzael, Corey Alexander, you got maiL, Horváth-Lázár Péter, zaurask, pinkhatbeard, Luiz Irber, Scott Sanderson, Daniel Wagner-Hall, Max Heaton, Johnathan Pagnutti, Peter Shih, Bob Ippolito, Marie Janssen, Christian Bourjau, Valentin Mariette, Mikkel Rasmussen, Morgan Rosenkranz, Moritz Lammerich, Xirvik Servers, Sam, Senyo Simpson, Joshua Roesslein, Romain Ruetschi, Ben Wishovich, Ronen Cohen, Jonathan Adams, John VanEnk, Alejandro Angulo, L0r3m1p5um, Dimitri Merejkowsky, henrique-pinto, René Ribaud, Sylvie Nightshade, Ramen, Dylan Anthony, Philipp Hatt, Paul Marques Mota, Raphaël Thériault, Steven McGuire, Michał Bartoszkiewicz, Theodore Rose, James Leitch, Jean-David Gadina, Chirag Jain, Zachary Thomas, Http 418, Niels Abildgaard, Andy Gocke, Michael Mrozek, Brandon Piña, Lev Khoroshansky, Duane Sibilly, WeblWabl, Jörn Huxhorn, Steven Pham, Ryan, Ives van Hoorne, ofrighil, Evan Relf, Dom, David Cornu, Mason Ginter, ZacJW, Yufan Lou, David Souther, Luke Konopka, Zaki, Beth Rennie, David White, David E Disch, Integer 32, LLC, Marky Mark, Tanner Muro, Enrico Zschemisch, Ronen Ulanovsky, hgranthorner, Helge Eichhorn, Sawyer Knoblich, Tabitha, Max Bruckner, Paul Schuberth, Aiden Scandella, Marco Carmosino, bbutkovic, Sarah Berrettini, Alex Krantz, Sean Bryant, notryanb, Ben Mitchell, Herman J. Radtke III, Astrid, Cass, Andrew Neth, Matthew T, Jarek Samic, std__mpa, Marcus Brito, Braidon Whatley, Guillaume Demonet, Benjamin Röjder Delnavaz, Kamran Khan, Shane Lillie, Tyler Bloom, Paul Horn, Sindre Johansen, Ernest French, Luke Yue, Geoffroy Couprie, Timothée Gerber, Marty Penner, Vincent

Comment on /r/fasterthanlime

If you liked what you saw, please support my work!

Donate on GitHub Donate on Patreon

Here's another article just for you:

So you want to live-reload Rust

Sep 26, 2020

1 hour 59 min #rust

Good morning! It is still 2020, and the world is literally on fire, so I guess we could all use a distraction.

This article continues the tradition of me getting shamelessly nerd-sniped - once by Pascal about small strings, then again by a twitch viewer about Rust enum sizes.

This time, Ana was handing out free nerdsnipes, so I got in line, and mine was:

Read more