Trying to use nix

From the series Updating fasterthanli.me for 2022

Nov 15, 2022 3 min #rust · #nix · #ftl-infra

Thanks to my sponsors: Pete LeVasseur, Chris Biscardi, Aalekh Patel, Paul Marques Mota, Kevin Anderson, psentee, Herman J. Radtke III, Colin VanDervoort, Brooke Tilley, Dominik Wagner, Zachary Thomas, Nyefan, traxys, Ross Williams, Gorazd Brumen, Josiah Bull, Makoto Nakashima, Vladimir, playest, Raphaël Thériault and 278 more

👋 This page was last updated ~3 years ago. Just so you know.

Now that my website is deployed as a container image, I wanted to give nix a try. I’m still doing it the old-fashioned way right now: with a Dockerfile, running cargo in a “builder” image, copying stuff out of there into a slimmer image (that still has an Ubuntu base, even though distroless images are a thing now).

But why?

I was mostly interested in nix because some parts of my website have pretty big native dependencies. futile itself mostly relies on sqlite3 and some JS engine (used to be quickjs, currently duktape because MSVC Windows builds). But the asset processing pipeline, salvage (which I’d like to integrate with futile at some point) has a bunch more!

Not only does it need Google Chrome headless for diagrams (which are made in the excellent diagrams.net desktop application), it then exports those to PDF, which are converted to SVG (baking text into paths, so you don’t need to have my fonts installed locally to view/print them), and then optimized further. That requires poppler and cairo, which requires glib, gobject, gio, etc.

And then there’s webp processing, avif processing, etc.

In fact I wrote a whole series about this, how I got everything to link statically (except Google Chrome, which is a standalone executable) on both Linux and Windows, that involved the Meson build system, etc.

Nowadays I don’t care so much about the Windows part, everything runs in a Linux VM anyway, but I figured nix would let me simplify a lot of this — no matter which distribution I chose to use day-to-day, all the dependencies would be statically defined in a single place, I could even apply my own patches if I wanted (which I do for the video processing pipeline — to get ffmpeg and libsvtav1 to play nice with each other), and I could link dynamically against libraries, which might speed up the link step and trim down the final executable quite a bit.

Also, apparently you can generate Docker images with nix, which sounded great.

Success (but at what cost?) with `crane`

So I started learning nix! I learned there’s quite a few ways to build Rust code with nix, a lot of folks use naersk, but after a couple tries, I gravitated towards crane instead.

A still-experimental but not-so-new thing in nix are “flakes”, I figured I’d make a flake to build futile (starting small), and got as far as this:

{
  inputs = {
    nixpkgs.url = "github:NixOS/nixpkgs/nixos-unstable";
    crane = {
      # url = "github:ipetkov/crane";
      url = "path:/home/amos/bearcove/crane";
      inputs.nixpkgs.follows = "nixpkgs";
    };
    flake-utils.url = "github:numtide/flake-utils";
    rust-overlay = {
      url = "github:oxalica/rust-overlay";
      inputs = {
        nixpkgs.follows = "nixpkgs";
        flake-utils.follows = "flake-utils";
      };
    };
  };

  outputs = { self, nixpkgs, crane, flake-utils, rust-overlay }:
    flake-utils.lib.eachDefaultSystem (system:
      let
        pkgs = import nixpkgs {
          system = system;
          overlays = [ (import rust-overlay) ];
        };

        rustToolchain = pkgs.pkgsBuildHost.rust-bin.fromRustupToolchainFile
          ./rust-toolchain.toml;
        lib = crane.lib.${system}.overrideToolchain rustToolchain;

        shipyardToken = builtins.readFile ./secrets/shipyard-token;
        craneLib = lib.appendCrateRegistries [
          (lib.registryFromDownloadUrl {
            dl = "https://crates.shipyard.rs/api/v1/crates";
            indexUrl = "https://git.shipyard.rs/bearcove/crate-index.git";
            curlOptsList = [ "-H" "user-agent: shipyard ${shipyardToken}" ];
          })
        ];
        futile = craneLib.buildPackage {
          src = ./.;
          nativeBuildInputs = with pkgs; [ pkg-config protobuf clang_14 mold ];
          installCargoArtifactsMode = "use-symlink";
        };

      in { packages.default = futile; });
}

Wait, did you add nix syntax highlighting to futile just now for this?

…maybe? Most of the work was already done.

I have no idea if this is idiomatic, it’s also complicated by the fact that I use a private crate registry, so I immediately had to go contribute to crane, and despite my best efforts, I still don’t deem it usable for now.

Building and installing a futile binary via nix still takes a long time, and the main offender is… removing references to /nix/store paths! There’s a long and interesting discussion going on about it right now, I might even get a chance to sneak some Rust in your Rust, but for now I’ve decided it’s outside the scope of this article, and we’ll revisit this all later.

It’s time to tackle the one thing I actually meant to do with this series, which is: adding GitHub login.

You're reading the Updating fasterthanli.me for 2022 series.

1

2

3

4

5

6

7

8

(JavaScript is required to see this. Or maybe my stuff broke)

Here's another article just for you:

Declarative memory management

Sep 19, 2019

35 min #rust · #memory

It feels like an eternity since I’ve started using Rust, and yet I remember vividly what it felt like to bang my head against the borrow checker for the first few times.

I’m definitely not alone in that, and there’s been quite a few articles on the subject! But I want to take some time to present the borrow checker from the perspective of its benefits, rather than as an opponent to fend with.

Trying to use nix

1

2

3

4

5

6

7

8

But why?

Success (but at what cost?) with crane

1

2

3

4

5

6

7

8

Declarative memory management

Success (but at what cost?) with `crane`