fasterthanlime

Using the Shipyard private crate registry with Docker

Thanks to my sponsors: Corey Alexander, genny, Nicholas Orta, Brandon Piña, Egor Ternovoi, Max von Forell, Ben Mitchell, Bader, Integer 32, LLC, James Rhodes, Mateusz Wykurz, Benjamin Röjder Delnavaz, Samit Basu, Richard Stephens, Henrik Tudborg, Menno Finlay-Smits, Adam Gutglick, Scott Sanderson, Christian Bourjau, budrick and 267 more Corey Alexander, genny, Nicholas Orta, Brandon Piña, Egor Ternovoi, Max von Forell, Ben Mitchell, Bader, Integer 32, LLC, James Rhodes, Mateusz Wykurz, Benjamin Röjder Delnavaz, Samit Basu, Richard Stephens, Henrik Tudborg, Menno Finlay-Smits, Adam Gutglick, Scott Sanderson, Christian Bourjau, budrick, Geoffroy Couprie, Michael Mrozek, Makoto Nakashima, Geoffrey Thomas, hgranthorner, Paul Marques Mota, notryanb, Marky Mark, Jonathan Adams, Marcus Griep, Nicholas, Malik Bougacha, C J Silverio, Hamilton Chapman, David Barsky, Mattia Valzelli, René Ribaud, Michael, Cole Tobin, Neil Blakey-Milner, Zoran Zaric, Chris Thackrey, Tyler Bloom, Marcin Kołodziej, James Leitch, ofrighil, Matt Campbell, Marc-Andre Giroux, Torben Clasen, Kai Kaufman, Senyo Simpson, Ula, Mark, SeniorMars, pinkhatbeard, Paige Ruten, Daniel Silverstone, Mikko Leppänen, Antoine PESTEL-ROPARS, Wyatt Herkamp, Marcus Griep, Michał Zalewski, Jack Maguire, Lennart Oldenburg, anichno, Mario Fleischhacker, Kristoffer Winther Balling, Tiziano Santoro, Lyssieth, ACRL, jer, Sean Bryant, Morgan Rosenkranz, Richard Pringle, Marcus Brito, Santiago Lema, Olivia Crain, Mikkel Rasmussen, Chris Biscardi, Manuel Hutter, Cole Kurkowski, Marie Janssen, Dom, WeblWabl, Thehbadger, Dominik Wagner, xales, Aalekh Patel, Matěj Volf, Joshua Roesslein, Yufan Lou, Valentin Mariette, Ives van Hoorne, Walther, qrpth, Zeeger Lubsen, Olly Swanson, jatescher, Sawyer Knoblich, John VanEnk, Julian Schmid, Taneli Kaivola, Stephan Buys, avborhanian, traxys, Marty Penner, Matt Heise, Matthew T, Vladimir, Justin Ossevoort, compwhizii, Yann Schwartz, Shane Lillie, Horváth-Lázár Péter, Mark Tomlin, Isak Sunde Singh, DaVince, Michal Hošna, Elnath, Joseph Montanaro, Michael Alyn Miller, Mark Old, Justy, Toon Willems, Seth, Alex Krantz, Chris Walker, villem, Boris Dolgov, Ronen Ulanovsky, Simon Menke, Jean-David Gadina, Aljaz Erzen, Duane Sibilly, Pete LeVasseur, Damir Vandic, Gioele Pannetto, Dimitri Merejkowsky, Colin VanDervoort, Zachary Thomas, Jelle Besseling, Beat Scherrer, Astrid, Jim, David Cornu, Josiah Bull, prairiewolf, Thor Kamphefner, Gorazd Brumen, Rufus Cable, teor, Jake Demarest-Mays, Antoine Boegli, Matt Jackson, Guillaume E, Cass, Kamran Khan, Ivo Murrell, Alex Rudy, Berkus Decker, Yuriy Taraday, Braidon Whatley, Michał Bartoszkiewicz, Matt Jadczak, Dirkjan Ochtman, Òscar Pérez, David E Disch, Eugene Bulkin, Evan Relf, Raine Godmaire, Josh Triplett, Niels Abildgaard, G, Romet Tagobert, David White, Xavier Groleau, Peter Shih, Lena Schönburg, clement, Pete Bevin, Jack Duvall, Andrew Neth, Kyle Lacy, Mathias Brossard, Elijah Voigt, Radu Matei, AdrianEddy, Ben Wishovich, Nicolas Riebesel, Brooke Tilley, Wojciech Smołka, Aiden Scandella, Zaki, Alan O'Donnell, Johnathan Pagnutti, Aleksandre Khokhiashvili, Noel, Ahmad Alhashemi, medzernik, Garret Kelly, Elendol, Sindre Johansen, callym, Sam Leonard, Paul Horn, Guilherme Neubaner, Nyefan, Ronen Cohen, bbutkovic, johbii, Adam Lassek, Luke Yue, Jan-Stefan Janetzky, Daniel Wagner-Hall, Philipp Hatt, Herman J. Radtke III, Dylan Anthony, Tyler Schmidtke, John Horowitz, Philipp Gniewosz, Jesse Luehrs, Mike English, Timothée Gerber, Sarah Berrettini, Borys Minaiev, L0r3m1p5um, Max Bruckner, Romain Kelifa, Chris, Lucille Blumire, Das Gürteltier, Dave Minter, Blake Johnson, Daniel Papp, Antoine Rouaze, Jörn Huxhorn, Luiz Ferraz, Andy Gocke, Victor Song, Guillaume Demonet, Daniel Strittmatter, Justin Smith, Beth Rennie, Diego Roig, ShikChen, Ian McLinden, Urs Metz, belzael, Chirag Jain, std__mpa, Laine Taffin Altman, Luke Konopka, Marco Carmosino, James Brown, Mason Ginter, Hadrien G., Ross Williams, Raphaël Thériault, Max Heaton, Christopher Valerio, playest, Bob Ippolito, Tobias Bahls, Jan De Landtsheer, Sylvie Nightshade, Andy F, Yves, you got maiL, Xirvik Servers, Jon Gjengset, Scott Steele, Enrico Zschemisch, old.woman.josiah, Luuk, Chris Emery, Tabitha, Tanner Muro, Steven Pham, ZacJW, e9zaktw1, Christoph Grabo, Chris Sims, Ryan, Adrián Garnier Artiñano, Geoff Cant, Andronik, Guy Waldman

Wait wait wait, so we’re not talking about nix yet?

Well, no! The service we have is pretty simple, and I want to complicate things a bit, to show how things would work in both the Dockerfile and the nix scenario.

And because I don’t like contrived examples, we’re going to do something somewhat real-world: we’re going to geo-locate visitors, and track how many visits we get from each country.

But we’re not logging IP addresses, right?

No, that would be problematic re: a bunch of privacy protection laws. I feel like the country is broad enough that it’s okay to store somewhere.

Very well, if we must.

In that particular context, it would make sense to have a cargo workspace, with both our main (web server) crate, and our “geo-locate and count visitors by country” crate in the same Git repository.

But for demonstration purposes, we’re going to make an entirely separate repository for it.

Let’s call it… locat. For location. And cat.

Let’s stub out our API:

Now’s a good time to create a repository on GitHub (or wherever), do an initial commit, and push locat to it.

Normally we’d be able to add a dependency on the crate simply by doing something like:

Or even:

But that’s not what I want to show off here.

I also don’t want to publish it to crates.io, since it’s probably too specific and I don’t want to assume the maintenance burden of it.

Instead, let’s publish it to Shipyard, a private crate registry that popped up recently.

They have a free tier, and we’re only going to publish a free crate there, so, let’s try it!

Signing up for Shipyard should give you a crate registry of your own, and the next step is to add it to the Cargo config for locat:

(My registry is named catscii, yours will probably be named something else).

Next we amend the Cargo manifest to indicate that the crate should only be published to our private registry:

Now that we have all that set up, let’s try publishing the crate with cargo publish:

That didn’t work! We need to authenticate in some way.

At the time of this writing, the proper authentication model isn’t done cooking yet, so I’m hesitant to show it here. Instead, we’ll use the hack that works on Rust stable.

First we need to generate a token over at https://shipyard.rs/tokens, and then, just so it’s shared across all our repositories, we can put it in the directory that contains both catscii and locat. In my VM, that’s just the home directory, /home/amos, or ~ for short:

This is a pretty bad workaround, since that’ll get sent with every request cargo makes, let’s make a note to try it the Rust nightly way after that.

Also: this is only half the authentication story. This takes care of making API requests to the crate registry, which is required when publishing a crate, but it doesn’t help with cloning the index, which is just a git repository.

Shipyard, like most private crate registries, provides SSH-based authentication, but this can be annoying to set up in CI. It also provides HTTPS authentication, which is annoying for other reasons.

Because we set our registry URL as https://git.shipyard.rs/REGISTRY/crate-index.git, it’s expecting HTTPs authentication, so let’s try that first.

In ~/.gitconfig, we’ll add:

And in ~/.git-credentials, we’ll add our Gitea username (find it here) and password, URL-encoded. Unless you changed it, this is the same password you signed up for Shipyard with, see their docs for details.

Cool Bear's hot tip

To get URI-encoding right, you can use encodeURIComponent("blah") in some browser dev tools.

For me, this managed to clone the repository, but it failed with:

Cool Bear's hot tip

You may also encounter:

error: unknown SSH host key The SSH host key for `ssh.shipyard.rs` is not known and cannot be validated.

You can run ssh git@ssh.shipyard.rs and accept the host key to fix it.

Running cargo login talks about crates.io, so that’s certainly wrong. Running cargo login --registry catscii does a little better but… you can tell third-party crate registries wasn’t on top of mind…

…because the URL it generates is crates.io-specific. Anyway, we have a token we generated earlier (it’s in ~/.cargo/config.toml), we can just paste it here.

Finally, publishing works:

The crate publishes successfully, and we can see it from the Shipyard UI:

However, this didn’t spark joy. Let’s try SSH authentication, too.

To reset the state a little, first let’s bump the version of our locat crate:

Then we’ll straight up:

• remove ~/.git-credentials
• remove the [credential] section from ~/.gitconfig
• delete ~/.cargo/config.toml (the ~ one, not the locat one)

And also, we’ll need to wipe the cloned crate index, to make sure it can do it again.

In locat’s cargo config, we now need to use an ssh URL:

Then we need to generate an SSH key from within the VM:

Then we need to add it to Shipyard, through the web UI.

Cool Bear's hot tip

Make sure to back up that SSH key for later, when we set up everything again from scratch!

And now, let’s try everything again!

This ended up failing for me because VS Code forwarded my host SSH agent (which is 1Password), so it only tried using the SSH keys I had on the host, as opposed to the key I just generated on the VM.

That’s fine, I can just add one of my host’s SSH keys to Shipyard instead.

I ended up having to mess with this a little more, and add back:

…otherwise nothing would work. Things are complicated by the fact that I have other SSH keys associated to another shipyard account, so I think I’m hitting an edge case you hopefully will not hit yourselves.

After that, publishing catscii@0.2.0 worked:

But keep in mind that it’s re-using the cargo login step from the “HTTP authentication” section: our token is still stored in here:

Now we can add a dependency on locat! First we need to set the registry URL:

And then, add the dependency! I wonder if cargo add will work there.

Yes it will! Here’s the resulting Cargo.toml bit:

Wonderful. Let’s do a quick cargo check just to see…

Ah. So there’s actually three halves to cargo authentication:

• Cloning the registry index (over Git: HTTPS or SSH)
• The publish API (token)
• The download API (token)

I said we’d try the nightly thing, so let’s try the nightly thing.

We only need to do that in the catscii directory. Let’s pin to a recent-ish nightly:

And set up our catscii registry:

Our next cargo check invocation downloads that nightly version:

And complains that we haven’t set the right experimental flags.

Let’s fix that:

And then… it just works!

Normally we’d have to make some code changes to get to the IP address, but since we’re deploying to fly.io, we’re behind a proxy already, and they set a header for that, so we can just use it.

We’ll add a locat field in our ServerState struct:

Initialize it:

And call it from root_get:

Even though we’re building the production version of our app using Docker, it should still run with cargo run, so let’s try that:

Trying to visit localhost:8080 from my host’s browser prints… nothing. Because the fly-client-ip header isn’t set.

We can try it with curl (in another terminal) and set an arbitrary IP there:

And the server shows:

Which makes perfect sense, since we haven’t yet connected the GeoIP database and everything. This is neat, because it lets us experiment with deploying the new code without breaking anything.

Even though our new code doesn’t do anything useful yet, let’s try deploying it to production.

We should probably make a Justfile, so we don’t forget which commands to run.

You can install just with cargo install just for now, and then:

Now tell me cool bear: do you think this’ll go smoothly?

You’re asking, so… probably not?

You’re absolutely correct. It’s time for.. more Docker golfing!

Contest number one! It says it can’t find a registry named catscii. And yet, the .cargo/config.toml is right there on disk.

Yes, but… looks at heading it’s not in the build context? We need to copy it?

Correct!

You know what else is a compile-time dependency now?

That’s right! It’s the square hole Git!

Does that mean it needs to re-download and re-install all packages from scratch on the next docker build run? You betcha!

It’s been two minutes already. I guess that was an unlucky run. How are you doing today? Is it a lovely day for you? Not necessarily the weather, but, you know, your internal weather? Have you been tending to your inner garden? It’s important, you know. Sometimes self-care might feel like you’re being selfish, but really you’re just getting yourself in a position where you can actually help others again, whereas if y- oh would you look at that, it finished.

Oh for crying out loud.

Well I guess we have time for more chit chat. What songs are you listening to right now? I’ve been trying to find Winter-themed music that isn’t offensive to my sensibilities. That mostly means Frank Sinatra, or some other jazz pianists. Teddy Wilson’s “As Times Goes By” is surprisingly g- oh, break’s over!

Oh. Uh. A little web search pointed to this, let’s try it:

No, wait, nevermind:

Okay, sure:

I’m fairly sure that defeats the purpose, but oh well.

Ok. I’ll save you the trouble. BuildKit also lets us “mount our SSH agent” (to Docker, every problem is mount-shaped), so we can do this:

The error message is exactly the same, except it includes --mount=type=ssh.

This is because, you need to tell docker build to share your SSH credentials. Otherwise any Dockerfile could just steal them. I still think the security model is kinda wonky there, but, we’ve written our own Dockerfile from start to finish and I trust it somewhat.

Let’s adjust our docker build invocation in the Justfile:

Because, again, I have multiple Shipyard accounts, and my SSH agent offers keys that are associated with the wrong account (an edge case you shouldn’t be hitting), I set up another agent quickly:

Here’s our next error:

Wait wait wait wait. I know this one. Can I try something?

Sure?

AhAh! It’s not using the right version of Rust inside Docker, because…

…because rust-toolchain.toml is not part of the build context.

Whoa, these are easy to miss, you weren’t kidding!

Yeah! But you know what? Some developers actually love technologies that make it easy to make mistakes, because then there’s always something to do! Every single one of these commits counts towards some metric, and it makes them look very productive: “saving the day” is usually remembering one of a hundred footguns and applying the same fix, over and over.

Huh. Funny how this works out.

Yes yep. Anyway:

(Also don’t forget to remove exit 251 from the Dockerfile)

Ah. So that one only worked locally because we had done cargo login. I see.

Well… this is a docker. Every problem is a mount away. Even secrets!

Because we’re going to commit that file, let’s make sure it’s encrypted with git-crypt first.

Then we can store just the value of our token in secrets/shipyard-token:

(Make sure that file has no trailing newline!)

Let’s make sure git-crypt picks it up:

Now we can mount it as a secret:

And pass it to docker build:

And finally, just deploy works!

Visiting the site, I can see this in my logs:

Nice! Now we “just” need to make it do the thing!

Comment on /r/fasterthanlime