Thanks to my sponsors: Brandon Piña, jatescher, David White, Christian Bourjau, belzael, Sean Bryant, Antoine Rouaze, Paul Horn, bbutkovic, Blake Johnson, Lennart Oldenburg, Raine Godmaire, Daniel Papp, Ronen Ulanovsky, Ronen Cohen, Hadrien G., Walther, Marcus Brito, Olly Swanson, jer and 230 more
14 min
Developing over SSH
With the previous part's VM still running, let's try connecting to our machine over SSH.
Network addresses, loopback and IP nets
Normally, to connect to a machine, you'd find its IP address. On Linux, a decade
ago, you would've used ifconfig. Nowadays you can use ip addr:
The loopback interface (lo) is local, so it's not useful to reach the box from
the outside: you can see it can be accessed over IPv4 at address 127.0.0.1 but
not just! What we're reading here is 127.0.0.1/8, which corresponds to the range
127.0.0.1 - 127.255.255.255
This is useful, as it lets us have services that listen on different loopback
IPv4 addresses, and apply different routing rules to different 127.0.0.1/8
addresses.
For IPv6 however, ::1/128 indicates a single address, ::1 (which is the
short form of 0:0:0:0:0:0:0:1):
We can use any of these addresses to connect to our VM over SSH, from our VM.
...but that's not very useful. We already had terminal access there, through the VirtualBox GUI.
What we'd like is to connect to it from the outside. We saw earlier that we had
another network interface, enp0s3. These used to be called something like
eth0 (for ethernet), but now the scheme is:
en= EtherNetp0= Bus number 0s3= Slot number 3
You can read more about the new naming scheme if you're curious.
My VM had an IPv4 address of 10.0.2.15/24 (with a range of
10.0.2.1-10.0.2.254, according to an IP subnet calculator such as this
one), and I am able to ping that address from the VM
(from the "guest"):
But not from the host (still Windows 11, here with the wonderful Windows Terminal and the latest PowerShell 7):
And that is working as designed. The guest is on its own, separate network. In fact, my other VM also has an address of 10.0.2.15 on its own network.
That's why we set up port forwarding in the previous part: it has VirtualBox listen on port 2223, and forward connections to the VM's private network.
We can make sure it's set up correctly with PowerShell (this is Windows-specific, but there's equivalents on Linux and macOS that are a web search away):
And sure enough, it accepts TCP connections on that port:
Connecting over SSH
We can connect to it with ssh (again, it's an optional Windows component now, it ships with macOS by default, and look to your Linux distribution's package manager search functionality) like so:
$ ssh amos@127.0.0.1 -p 2223
The authenticity of host '[127.0.0.1]:2223 ([127.0.0.1]:2223)' can't be established.
ED25519 key fingerprint is SHA256:zwxa3nLGjzTOLg2m3+jN91fpMH7BWVJkow89tYcygtE.
This key is not known by any other names
Are you sure you want to continue connecting (yes/no/[fingerprint])?
It asks us whether we want to trust that server key, and we do, so we can type "yes" and press enter:
$ yes
Warning: Permanently added '[127.0.0.1]:2223' (ED25519) to the list of known hosts.
Welcome to Ubuntu 22.04.1 LTS (GNU/Linux 5.15.0-53-generic x86_64)
* Documentation: https://help.ubuntu.com
* Management: https://landscape.canonical.com
* Support: https://ubuntu.com/advantage
System information as of Wed Nov 23 10:48:00 AM UTC 2022
System load: 0.0 Processes: 107
Usage of /: 14.3% of 47.93GB Users logged in: 1
Memory usage: 1% IPv4 address for enp0s3: 10.0.2.15
Swap usage: 0%
* Strictly confined Kubernetes makes edge and IoT secure. Learn how MicroK8s
just raised the bar for easy, resilient and secure K8s cluster deployment.
https://ubuntu.com/engage/secure-kubernetes-at-the-edge
0 updates can be applied immediately.
Last login: Wed Nov 23 09:59:30 2022 from ::1
amos@miles:~$ whoami
amos
(Press Ctrl-D to exit out of the SSH session. This indicates "end of file").
However, it's not very convenient to have to type that whole ssh amos@127.0.0.1 -p 2223 command every time, so we can add an entry to the host's
~/.ssh/config config file instead.
Cool bear's hot tip
~ denotes your home directory, and it'll be familiar to you if you're on Linux
or macOS. It also works in PowerShell, which means you can do something like
code ~/.ssh/config to edit it.
If the ~/.ssh directory doesn't exist yet, mkdir ~/.ssh should work on all
three OSes, again — there's a Windows
alias
for it.
The config section looks like:
# in ~/.ssh/config
Host miles
HostName 127.0.0.1
Port 2223
User amos
ForwardAgent yes
Mhh what's that ForwardAgent thingy doing here?
Well, you know how, from our host operating system, we have an SSH keypair, added to an SSH agent, that we use to authenticate with GitHub and connect to our VM?
Yes?
I mean... oh, yes okay, Windows ships with ssh-agent too nowadays, or you can use 1Password for this, so yes I guess so?
Yes it does:
$ echo $env:OS
Windows_NT
$ ssh-add -l
2048 SHA256:4txMCM8iFJaOmrB7qVAMNwSdy7KUbVvqMrcBdLd/VXo teleport:fasterthanlime (RSA-CERT)
256 SHA256:IEoy+ad7M0Mcy7lts4KLk2q0ca+i/9yyUBmd0+Cy9rY july-2020 (ED25519)
4096 SHA256:r8YfVEk6CVCO9S4ykJZew2qM+cSR/nFWLs8Ovul6hMk amos@tails (RSA)
Well look at this: if we use the VirtualBox GUI to access the VM and try to talk to GitHub, it doesn't know who we are:
But if we connect from our host to our guest, then from that SSH session, try to talk to GitHub, it does!
$ ~
❯ ssh miles
Welcome to Ubuntu 22.04.1 LTS (GNU/Linux 5.15.0-53-generic x86_64)
(cut: spammy Ubuntu banner)
Last login: Wed Nov 23 11:00:26 2022 from 10.0.2.2
amos@miles:~$ ssh -T git@github.com
Hi fasterthanlime! You've successfully authenticated, but GitHub does not provide shell access.
amos@miles:~$
Cool bear's hot tip
If this doesn't work for you, make sure ssh -T git@github.com works from the
host first.
The GitHub docs are helpful there!
Copying files
Now that we're connected to our VM, we can do something useful there!
For example, we can create a text file:
amos@miles:~$ echo "I was made inside the guest" > hello.txt
amos@miles:~$ cat hello.txt
I was made inside the guest
And then copy it from the guest to the host. This is run from the host:
~
❯ scp miles:~/hello.txt .
hello.txt 100% 28 0.0KB/s 00:00
~ took 4s
❯ cat hello.txt
I was made inside the guest
That's it. scp stands for "secure copy" and uses ssh for data transfer, uses
the same authentication mechanisms, etc.
You can also copy files to the guest, with scp ./some-host-file.txt miles:~/path-on-guest.
Installing a Rust toolchain
From the guest, let's install a rust toolchain with rustup:
amos@miles:~$ curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh
info: downloading installer
Welcome to Rust!
This will download and install the official compiler for the Rust
programming language, and its package manager, Cargo.
(cut)
Current installation options:
default host triple: x86_64-unknown-linux-gnu
default toolchain: stable (default)
profile: default
modify PATH variable: yes
1) Proceed with installation (default)
2) Customize installation
3) Cancel installation
>1
info: profile set to 'default'
info: default host triple is x86_64-unknown-linux-gnu
info: syncing channel updates for 'stable-x86_64-unknown-linux-gnu'
info: latest update on 2022-11-03, rust version 1.65.0 (897e37553 2022-11-02)
info: downloading component 'cargo'
info: downloading component 'clippy'
(cut)
info: default toolchain set to 'stable-x86_64-unknown-linux-gnu'
stable-x86_64-unknown-linux-gnu installed - rustc 1.65.0 (897e37553 2022-11-02)
Rust is installed now. Great!
To get started you may need to restart your current shell.
This would reload your PATH environment variable to include
Cargo's bin directory ($HOME/.cargo/bin).
To configure your current shell, run:
source "$HOME/.cargo/env"
Like the installer just said, we can source ~/.cargo/env to get "cargo" in our
$PATH.
Mh?
Well, it currently isn't:
amos@miles:~$ cargo
Command 'cargo' not found, but can be installed with:
sudo snap install rustup # version 1.24.3, or
sudo apt install cargo # version 0.60.0ubuntu1-0ubuntu1~22.04.1
See 'snap info rustup' for additional versions.
amos@miles:~$ which cargo
amos@miles:~$
But if we source ~/.cargo/env, then it is!
amos@miles:~$ source ~/.cargo/env
amos@miles:~$ which cargo
/home/amos/.cargo/bin/cargo
amos@miles:~$ cargo -vV
cargo 1.65.0 (4bc8f24d3 2022-10-20)
release: 1.65.0
commit-hash: 4bc8f24d3e899462e43621aab981f6383a370365
commit-date: 2022-10-20
host: x86_64-unknown-linux-gnu
libgit2: 1.5.0 (sys:0.15.0 vendored)
libcurl: 7.83.1-DEV (sys:0.4.55+curl-7.83.1 vendored ssl:OpenSSL/1.1.1q)
os: Ubuntu 22.04 (jammy) [64-bit]
Installing a web server
Now, we can use cargo to install a web server, like, say,
sfz:
amos@miles:~$ cargo install sfz
Updating crates.io index
Fetch [===> ] 16.54%, 11.34MiB/s
...because it's the first time we need the crates.io index, it has to download the whole thing. This might take a while, even with fast internet.
Wait, it has to download an index of all crates ever published on crates.io?
For now, yes. There's a sparse_index RFC I like a lot, but we're on a stable version of cargo right now, so we don't get to use it, yet.
Eventually, our command... fails with this:
Downloaded hyper v0.14.23
Downloaded chrono-tz-build v0.0.3
Downloaded 130 crates (9.0 MB) in 0.67s (largest was `brotli` at 1.4 MB)
Compiling version_check v0.9.4
Compiling proc-macro2 v1.0.47
error: linker `cc` not found
|
= note: No such file or directory (os error 2)
error: could not compile `proc-macro2` due to previous error
error: failed to compile `sfz v0.7.1`, intermediate artifacts can be found at `/tmp/cargo-installsTSzh7`
Oh no.
Never fear, we just need to ask our friendly neighborhood package manager to install a couple things.
It says cc but really, we want gcc here.
amos@miles:~$ sudo apt install gcc
[sudo] password for amos:
Reading package lists... Done
Building dependency tree... Done
Reading state information... Done
The following packages were automatically installed and are no longer required:
libflashrom1 libftdi1-2
Use 'sudo apt autoremove' to remove them.
The following additional packages will be installed:
cpp cpp-11 fontconfig-config fonts-dejavu-core gcc-11 gcc-11-base libasan6 libatomic1 libc-dev-bin libc-devtools
libc6-dev libcc1-0 libcrypt-dev libdeflate0 libfontconfig1 libgcc-11-dev libgd3 libgomp1 libisl23 libitm1 libjbig0
libjpeg-turbo8 libjpeg8 liblsan0 libmpc3 libnsl-dev libquadmath0 libtiff5 libtirpc-dev libtsan0 libubsan1 libwebp7
libxpm4 linux-libc-dev manpages-dev rpcsvc-proto
Suggested packages:
cpp-doc gcc-11-locales gcc-multilib make autoconf automake libtool flex bison gdb gcc-doc gcc-11-multilib gcc-11-doc
glibc-doc libgd-tools
The following NEW packages will be installed:
cpp cpp-11 fontconfig-config fonts-dejavu-core gcc gcc-11 gcc-11-base libasan6 libatomic1 libc-dev-bin libc-devtools
libc6-dev libcc1-0 libcrypt-dev libdeflate0 libfontconfig1 libgcc-11-dev libgd3 libgomp1 libisl23 libitm1 libjbig0
libjpeg-turbo8 libjpeg8 liblsan0 libmpc3 libnsl-dev libquadmath0 libtiff5 libtirpc-dev libtsan0 libubsan1 libwebp7
libxpm4 linux-libc-dev manpages-dev rpcsvc-proto
0 upgraded, 37 newly installed, 0 to remove and 4 not upgraded.
Need to get 48.6 MB of archives.
After this operation, 152 MB of additional disk space will be used.
Do you want to continue? [Y/n]
Since Y (yes) is capitalized here, we can simply press "Enter" to continue.
And just like that we have a cc command in $PATH:
amos@miles:~$ which cc
/usr/bin/cc
It's.. a symbolic link:
amos@miles:~$ ls -lhA /usr/bin/cc
lrwxrwxrwx 1 root root 20 Nov 23 11:19 /usr/bin/cc -> /etc/alternatives/cc
...which is managed by update-alternatives, a very Debian thing:
$ update-alternatives --list cc
/usr/bin/gcc
And, yeah. It's just gcc.
Let's try again!
$ cargo install sfz
(cut)
Compiling qstring v0.7.2
Compiling tokio-util v0.7.4
Compiling sfz v0.7.1
Finished release [optimized] target(s) in 2m 02s
Installing /home/amos/.cargo/bin/sfz
Installed package `sfz v0.7.1` (executable `sfz`)
And this time it works!
It's in $PATH, too, because we sourced ~/.cargo/env earlier:
amos@miles:~$ which sfz
/home/amos/.cargo/bin/sfz
Let's run it:
amos@miles:~$ sfz
Files served on http://127.0.0.1:5000
Cool bear's hot tip
You can exit out of this with Ctrl+C, which sends an interrupt signal
(SIGINT) to sfz.
The difference with "exiting out of an SSH session" is that sfz isn't reading the standard input, so it won't detect an EOF there. But the default signal handler will quit the (otherwise long-running) program.
And, from another terminal also in the guest, we can see that it does work:
amos@miles:~$ curl --head 0:5000
HTTP/1.1 200 OK
server: sfz/0.7.1
accept-ranges: bytes
content-type: text/html; charset=utf-8
content-length: 2166
date: Wed, 23 Nov 2022 11:26:46 GMT
Accessing our web server from the host
However, the question is... how are we going to access all this from the host?
Well, for starters, it's listening on 127.0.0.1, which is definitely not accessible from the outside.
You're right — we can have sfz listen on 0.0.0.0 instead, which means "all
network interfaces".
amos@miles:~$ sfz --bind 0.0.0.0
Files served on http://0.0.0.0:5000
...but it won't help much. Remember: the VM's enp0s3 interface is connected to
a network that's private to this VM. We can't reach it even from the host:
$ curl --verbose --head http://10.0.2.15:5000
* Trying 10.0.2.15:5000...
* connect to 10.0.2.15 port 5000 failed: Timed out
* Failed to connect to 10.0.2.15 port 5000 after 21001 ms: Timed out
* Closing connection 0
curl: (28) Failed to connect to 10.0.2.15 port 5000 after 21001 ms: Timed out
There's a couple options here: we could add another port forwarding rule in the VirtualBox network configuration. Or we could forward that port over SSH!
Let's exit out of sfz with Ctrl+C and out of our current SSH session with
Ctrl+D:
amos@miles:~$ sfz -bind 0.0.0.0
Files served on http://0.0.0.0:5000
^C
amos@miles:~$
logout
Connection to 127.0.0.1 closed.
And then log back into our VM, but asking SSH to forward something for us:
$ ssh -L 5000:localhost:5000 miles
(cut)
Last login: Wed Nov 23 11:26:31 2022 from 10.0.2.2
amos@miles:~$
This has the ssh client listen on port 5000 on the host, and whenever it accepts
connections on there, it forwards them to localhost:5000 from the perspective
of the guest.
Cool bear's hot tip
It seems odd at first that we have to specify localhost in there, because for
simple scenarios, it's what we want. But imagine we're connecting to some host
that's part of a private network, and we're simply using it as a relay to reach
a third host.
We might end up doing something like ssh -L 5000:third-host:5000 relay-host.
Now, we can run sfz again:
$ amos@miles:~$ sfz
Files served on http://127.0.0.1:5000
And access it from our.. host OS!
It's confusing because "host" means "some device on a network" and also "the operating system that runs the VM hypervisor, in which the guest OS is executed".
In this case, I mean the latter.
~
❯ curl -I http://localhost:5000
HTTP/1.1 200 OK
server: sfz/0.7.1
accept-ranges: bytes
content-type: text/html; charset=utf-8
content-length: 2166
date: Wed, 23 Nov 2022 11:42:27 GMT
In fact, we can open it in a browser:
In fact, here's our hello.txt file. Hello to you too!
VSCode's Remote SSH feature
You now know enough to develop server applications remotely on your Linux VM.
However, because this is my series and I do what I want, I'll show you why it's nice to use VSCode for this.
First off, you'll need to download / install VSCode, through the official website or some package manager.
Cool bear's hot tip
There's open-source distributions of VSCode like VSCodium (similar to Chrome -> Chromium), but they might not have the extensions we want, so, if you pick that, you're on your own.
Make sure you have the "Remote - SSH" extension installed (from the command menu, pick "Install extensions", or click on the icon from the leftmost bar that looks like a package: a grid of four squares, one of them trying to break free):
(There's also "Remote Development" group that includes "Remote - SSH", "Remote - WSL" and "Dev Containers", if you're interested in that).
Once the extension is installed, you can open the command palette (F1 on Linux/Windows, Shift+Cmd+P on macOS), start typing "Connec...", and pick "Remote-SSH: Connect to host"
Next up, VSCode will read your ~/.ssh/config and suggest picking one of the
hosts it found there:
Picking "miles" opens a new window, with another prompt asking us to select the platform, let's pick Linux:
VSCode will then download its server component on the remote machine (our guest):
And we're good to go!
We can open the built-in terminal with CmdOrCtrl+Backquote, or clicking this
little icon, then selecting the Terminal tab:
And tada, we have a terminal on our guest VM:
From there, we can run sfz again:
Cool bear's hot tip
If you get "Server error: error creating server listener: Address already in use
(os error 98)", that probably means you still have sfz running in another
terminal session.
Make sure to exit out of it first! (If everything else fails, you can try
killall sfz).
amos@miles:~$ sfz
Files served on http://127.0.0.1:5000
Now let's try opening it in a browser...
And it works!
Wait, didn't we forget something?
Forget something? Like what?
Well, when we did it with SSH in command line, we had to forward port 5000 from the host to the guest. Is that still active somehow?
Oh, no, VSCode just detected that something inside the VM is listening on port 5000, and it set up port forwarding for us, over the same SSH connection.
Making VSCode all comfy and nice
I use the excellent Iosevka font, paired with either of "GitHub Light" / "Github Dark" (following the time of day), and these extensions:
- "rust-analyzer" is an absolute must-have: it provides code intelligence,
completion and refactoring for Rust.
- "Vim" provides me with Vim keybindings inside of VS Code. It's not quite the same, but I've been using that for years (along with a couple custom bindings to switch/close tabs, also hover, jump to definition, and move to the next/previous diagnostic) and I'm very happy with it.
- "Error Lens" shows diagnostics inline. I'll occasionally still have to
expand the "Problems" pane/tab to see a bigger version, or simply run
cargo clippyin the integrated terminal, but it's good for the 90% portion of silly mistakes.
- "Better TOML" provides syntax highlighting when editing
Cargo.toml,.cargo/config.toml, etc.
I have a lot more extensions, some minor QoL things (like "FontSize Shortcuts", "TODO Highlight" or "vscode-position"), some language-specific ("x86 and x86_64 Assembly", "Nix", "systemd-unit-file"), and some bigger, like "Git Blame", "GitHub Pull Requests and Issues".
Here's another article just for you:
Recursive iterators in Rust
7 min
I've been looking for this blog post everywhere, but it doesn't exist, so I guess it's my turn to write about Some Fun with Rust.
The task at hand
Let's say you have a recursive, acyclic data structure, like so:
struct Node {
values: Vec<i32>,
children: Vec<Node>,
}
This allows you to represent a tree-like structure: