Building poppler for Windows

This article is part of the Don't shell out! series.

I know what you're thinking: haven't we strayed from the whole "content pipeline" theme in this series?

Well... fair. But compiling and distributing software is part of software engineering, and unless you're in specific circles, I see that taught a lot less than the "just write code and stuff happens" part.

Technically it's release engineering, but who's keeping track.

So we're going to think about how we could build all this for Windows. Because every single of my internal tools for my website builds for Windows just fine, in case I, you know, need to write a series on Windows, from Windows.

So my little fork of salvage that uses headless_chrome to render a .drawio file to /tmp/export.pdf? That compiles and runs on Windows!

PowerShell session
$ .\target\debug\salvage.exe 2021-11-26T13:56:29.450869Z INFO headless_chrome::browser::fetcher: Getting project dir 2021-11-26T13:56:29.454840Z INFO headless_chrome::browser::process: Launching Chrome binary at "C:\\Users\\faste\\AppData\\Roaming\\headless-chrome\\data\\win-634997\\chrome-win\\chrome.exe" 2021-11-26T13:56:29.533551Z INFO headless_chrome::browser::process: Started Chrome. PID: 18936 2021-11-26T13:56:29.712694Z INFO salvage::chrome_stuff: Navigating... 2021-11-26T13:56:30.044934Z INFO headless_chrome::browser::tab: Navigating a tab to http://localhost:5000/index.html 2021-11-26T13:56:31.049236Z INFO salvage::chrome_stuff: Navigating... done! 2021-11-26T13:56:31.061422Z INFO salvage::chrome_stuff: Got dimensions width=994 height=643 2021-11-26T13:56:31.192107Z INFO salvage::chrome_stuff: Writing pdf... pdf_path=/tmp/export.pdf 2021-11-26T13:56:31.192335Z INFO headless_chrome::browser: Dropping browser 2021-11-26T13:56:31.192442Z INFO headless_chrome::browser::process: Killing Chrome. PID: 18936 2021-11-26T13:56:31.192552Z INFO headless_chrome::browser::transport::web_socket_connection: Sending shutdown message to message handling loop 2021-11-26T13:56:31.192694Z INFO headless_chrome::browser::transport: Received shutdown message 2021-11-26T13:56:31.192785Z INFO headless_chrome::browser::transport: Shutting down message handling loop 2021-11-26T13:56:31.192886Z INFO headless_chrome::browser::transport: cleared listeners, I think 2021-11-26T13:56:31.192904Z INFO headless_chrome::browser::tab: finished tab's event handling loop 2021-11-26T13:56:31.192900Z INFO headless_chrome::browser: Finished browser's event handling loop 2021-11-26T13:56:31.198980Z INFO headless_chrome::browser::transport: dropping transport 2021-11-26T13:56:31.199044Z INFO headless_chrome::browser::transport::web_socket_connection: dropping websocket connection The application panicked (crashed). Message: called `Result::unwrap()` on an `Err` value: Os { code: 3, kind: NotFound, message: "The system cannot find the path specified." } Location: src\chrome_stuff.rs:17 Backtrace omitted. Run with RUST_BACKTRACE=1 environment variable to display it. Run with RUST_BACKTRACE=full to include source snippets. Error: 0: chrome error: JoinError::Panic(...) Location: src\main.rs:47 Backtrace omitted. Run with RUST_BACKTRACE=1 environment variable to display it. Run with RUST_BACKTRACE=full to include source snippets.

Well huh... ok maybe we can't write to /tmp/export.pdf. Let's quickly turn this:

Rust code
let pdf_path = Utf8PathBuf::from("/tmp/export.pdf");

Into this:

Rust code
let pdf_path = Utf8PathBuf::try_from(std::env::temp_dir())?.join("export.pdf");

And then... it runs!

PowerShell session
$ cargo run Compiling salvage v1.2.0 (C:\Users\faste\bearcove\salvage) Finished dev [unoptimized + debuginfo] target(s) in 2.75s Running `target\debug\salvage.exe` 2021-11-26T14:03:34.463367Z INFO headless_chrome::browser::fetcher: Getting project dir 2021-11-26T14:03:34.468192Z INFO headless_chrome::browser::process: Launching Chrome binary at "C:\\Users\\faste\\AppData\\Roaming\\headless-chrome\\data\\win-634997\\chrome-win\\chrome.exe" 2021-11-26T14:03:34.550763Z INFO headless_chrome::browser::process: Started Chrome. PID: 14984 2021-11-26T14:03:34.853213Z INFO salvage::chrome_stuff: Navigating... 2021-11-26T14:03:35.188660Z INFO headless_chrome::browser::tab: Navigating a tab to http://localhost:5000/index.html 2021-11-26T14:03:37.092808Z INFO salvage::chrome_stuff: Navigating... done! 2021-11-26T14:03:37.104829Z INFO salvage::chrome_stuff: Got dimensions width=994 height=643 2021-11-26T14:03:37.241291Z INFO salvage::chrome_stuff: Writing pdf... pdf_path=C:\Users\faste\AppData\Local\Temp\export.pdf 2021-11-26T14:03:37.241867Z INFO salvage::chrome_stuff: Writing pdf... done! pdf_path=C:\Users\faste\AppData\Local\Temp\export.pdf 2021-11-26T14:03:37.241957Z INFO headless_chrome::browser: Dropping browser 2021-11-26T14:03:37.242089Z INFO headless_chrome::browser::process: Killing Chrome. PID: 14984 2021-11-26T14:03:37.242193Z INFO headless_chrome::browser::transport::web_socket_connection: Sending shutdown message to message handling loop 2021-11-26T14:03:37.242320Z INFO headless_chrome::browser::transport: Received shutdown message 2021-11-26T14:03:37.242398Z INFO headless_chrome::browser::transport: Shutting down message handling loop 2021-11-26T14:03:37.242484Z INFO headless_chrome::browser::transport: cleared listeners, I think 2021-11-26T14:03:37.242521Z INFO headless_chrome::browser::tab: finished tab's event handling loop 2021-11-26T14:03:37.242516Z INFO headless_chrome::browser: Finished browser's event handling loop

Isn't it amazing that this was the only change we needed to make?

Heck yeah!

This goes to show how great of a job crate maintainers have been doing!

Our diagram looks just as fine on Windows:

Shell session
$ start $env:TMP\export.pdf

Our pdftocairo executable however... it's a different story. Well, to be fair, I have no idea what to expect, so let's find out:

PowerShell session
$ cargo build Updating crates.io index Downloaded anyhow v1.0.47 Downloaded cairo-rs v0.14.9 (cut) Downloaded glib-sys v0.14.0 Downloaded proc-macro-error v1.0.4 Downloaded 19 crates (615.9 KB) in 1.39s Compiling proc-macro2 v1.0.32 Compiling unicode-xid v0.2.2 (cut) Compiling poppler-sys-rs v0.18.0 The following warnings were emitted during compilation: warning: Could not run `"pkg-config" "--libs" "--cflags" "glib-2.0" "glib-2.0 >= 2.48"` error: failed to run custom build command for `glib-sys v0.14.0` Caused by: process didn't exit successfully: `C:\Users\faste\bearcove\pdftocairo\target\debug\build\glib-sys-0231afba2b42ecdf\build-script-build` (exit code: 1) --- stdout cargo:rerun-if-env-changed=GLIB_2.0_NO_PKG_CONFIG cargo:rerun-if-env-changed=PKG_CONFIG_x86_64-pc-windows-msvc cargo:rerun-if-env-changed=PKG_CONFIG_x86_64_pc_windows_msvc cargo:rerun-if-env-changed=HOST_PKG_CONFIG cargo:rerun-if-env-changed=PKG_CONFIG cargo:rerun-if-env-changed=PKG_CONFIG_PATH_x86_64-pc-windows-msvc cargo:rerun-if-env-changed=PKG_CONFIG_PATH_x86_64_pc_windows_msvc cargo:rerun-if-env-changed=HOST_PKG_CONFIG_PATH cargo:rerun-if-env-changed=PKG_CONFIG_PATH cargo:rerun-if-env-changed=PKG_CONFIG_LIBDIR_x86_64-pc-windows-msvc cargo:rerun-if-env-changed=PKG_CONFIG_LIBDIR_x86_64_pc_windows_msvc cargo:rerun-if-env-changed=HOST_PKG_CONFIG_LIBDIR cargo:rerun-if-env-changed=PKG_CONFIG_LIBDIR cargo:rerun-if-env-changed=PKG_CONFIG_SYSROOT_DIR_x86_64-pc-windows-msvc cargo:rerun-if-env-changed=PKG_CONFIG_SYSROOT_DIR_x86_64_pc_windows_msvc cargo:rerun-if-env-changed=HOST_PKG_CONFIG_SYSROOT_DIR cargo:rerun-if-env-changed=PKG_CONFIG_SYSROOT_DIR cargo:warning=Could not run `"pkg-config" "--libs" "--cflags" "glib-2.0" "glib-2.0 >= 2.48"` The pkg-config command could not be found. Most likely, you need to install a pkg-config package for your OS. If you've already installed it, ensure the pkg-config command is one of the directories in the PATH environment variable. If you did not expect this build to link to a pre-installed system library, then check documentation of the glib-sys crate for an option to build the library from source, or disable features or dependencies that require pkg-config. warning: build failed, waiting for other jobs to finish... error: build failed

Well, system-deps wasn't lying, it really does only support pkg-config!

pkg-config can be built for windows, and in fact, we can install it with scoop:

PowerShell session
$ scoop install pkg-config Installing 'pkg-config' (0.26-1) [64bit] Loading pkg-config_0.26-1_win32.zip from cache Checking hash of pkg-config_0.26-1_win32.zip ... ok. Loading glib_2.28.8-1_win32.zip from cache Checking hash of glib_2.28.8-1_win32.zip ... ok. Loading gettext-runtime_0.18.1.1-2_win32.zip from cache Checking hash of gettext-runtime_0.18.1.1-2_win32.zip ... ok. Extracting pkg-config_0.26-1_win32.zip ... done. Extracting glib_2.28.8-1_win32.zip ... done. Extracting gettext-runtime_0.18.1.1-2_win32.zip ... done. Linking ~\scoop\apps\pkg-config\current => ~\scoop\apps\pkg-config\0.26-1 Creating shim for 'pkg-config'. 'pkg-config' (0.26-1) was installed successfully!

But of course, it can't find anything:

PowerShell session
$ pkg-config --cflags --libs cairo Package cairo was not found in the pkg-config search path. Perhaps you should add the directory containing `cairo.pc' to the PKG_CONFIG_PATH environment variable No package 'cairo' found

Where does it even look?

Well bear, you know what RTFM means?

Read the.. fine manual?

Yes! But after a couple searches, I cannot for the life of me figure out where pkg-config looks for .pc files on Windows by default.

So what do we do then?

Well, even if I had found something, the docs always lie. So what we do is spy on the process instead.

But this is Windows, we don't have strace. What we do have, is Process Monitor.

And now we have our answer! It looks in C:\Users\faste\scoop\apps\pkg-config\current\share\pkgconfig, which is a symbolic link to the currently installed version:

PowerShell session
$ Get-Item C:\Users\faste\scoop\apps\pkg-config\current\ Directory: C:\Users\faste\scoop\apps\pkg-config Mode LastWriteTime Length Name ---- ------------- ------ ---- l-r-- 26/11/2021 15:11 current -> C:\Users\faste\scoop\apps\pkg-config\0.26-1

So, yay, there's a global path! But nothing stable like /usr/lib64/pkgconfig on Fedora, or /usr/lib/x86_64-linux-gnu/pkgconfig on Ubuntu.

I'm sure if we install our .pc files somewhere though, pkg-config will be able to find them, right?

Well, our build scripts right now are bash, so... unless we install bash, we can't run them as-is, but we can try to run similar instructions from a more Windows-y shell, like PowerShell?

Let's try with pcre:

PowerShell session
$ curl -f -L -o pcre.tar.bz2 https://sourceforge.net/projects/pcre/files/pcre/8.45/pcre-8.45.tar.bz2/download % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 605 100 605 0 0 605 0 0:00:01 --:--:-- 0:00:01 880 100 331 100 331 0 0 331 0 0:00:01 0:00:01 --:--:-- 331 100 1541k 100 1541k 0 0 1541k 0 0:00:01 0:00:01 --:--:-- 9883k

So far so good! Apparently Windows 11 ships the real curl, not just an alias?

PowerShell session
$ Get-Command curl CommandType Name Version Source ----------- ---- ------- ------ Application curl.exe 7.55.1.0 C:\Windows\system32\curl.exe

Next we need to extract the archive. Apparently tar also ships with Windows 11!

PowerShell session
$ tar xjf .\pcre.tar.bz2 $ cd .\pcre-8.45\

Now, what build system is PCRE using again? Autotools? Right. Well...

...that's not gonna work.

Luckily, PCRE also comes with a CMakeLists.txt, so we can "just" use cmake!

Visual Studio 2022 Build Tools ships with cmake, it's just a bit hidden, but if we start the "x64 Native Tools Command Prompt for VS 2022", it's right there in the PATH:

PowerShell session
$ Get-Command cmake CommandType Name Version Source ----------- ---- ------- ------ Application cmake.exe 3.21.2108… C:\Program Files (x86)\Microsoft Visual Studio\2022\BuildTools\Common7\IDE\CommonExtensions\Microsoft\CMake\CMake\bin\cmake.exe

So, let's make a build folder and see what happens!

PowerShell session
$ mkdir build $ cmake -S . -B build -- Building for: Visual Studio 17 2022 -- Selecting Windows SDK version 10.0.19041.0 to target Windows 10.0.22000. -- The C compiler identification is MSVC 19.30.30705.0 -- The CXX compiler identification is MSVC 19.30.30705.0 (cut) -- Could NOT find BZip2 (missing: BZIP2_LIBRARIES BZIP2_INCLUDE_DIR) -- Could NOT find ZLIB (missing: ZLIB_LIBRARY ZLIB_INCLUDE_DIR) -- Could not find OPTIONAL package Readline -- Could not find OPTIONAL package Editline -- Looking for dirent.h -- Looking for dirent.h - not found (cut) -- -- -- PCRE-8.45 configuration summary: -- -- Install prefix .................. : C:/Program Files (x86)/PCRE -- C compiler ...................... : C:/Program Files (x86)/Microsoft Visual Studio/2022/BuildTools/VC/Tools/MSVC/14.30.30705/bin/Hostx64/x64/cl.exe -- C++ compiler .................... : C:/Program Files (x86)/Microsoft Visual Studio/2022/BuildTools/VC/Tools/MSVC/14.30.30705/bin/Hostx64/x64/cl.exe -- C compiler flags ................ : /DWIN32 /D_WINDOWS /W3 -- C++ compiler flags .............. : /DWIN32 /D_WINDOWS /W3 /GR /EHsc -- -- Build 8 bit PCRE library ........ : ON (cut) -- Install MSVC .pdb files ..........: OFF -- -- Configuring done -- Generating done -- Build files have been written to: C:/Users/faste/bearcove/poppler-build/pcre-8.45/build

Ah, forgot to specify a prefix, let's do that now:

PowerShell session
$ cmake -S . -B build -DCMAKE_INSTALL_PREFIX=C:/Users/faste/bearcove/poppler-build/prefix (output omitted)

And.. let's build?

PowerShell session
$ cmake --build build --parallel (Get-CimInstance Win32_ComputerSystem).NumberOfLogicalProcessors Microsoft (R) Build Engine version 17.0.0+c9eb9dd64 for .NET Framework Copyright (C) Microsoft Corporation. All rights reserved. Checking Build System Building Custom Rule C:/Users/faste/bearcove/poppler-build/pcre-8.45/CMakeLists.txt pcre_byte_order.c pcre_chartables.c pcre_compile.c (cut) pcre_stringpiece_unittest.vcxproj -> C:\Users\faste\bearcove\poppler-build\pcre-8.45\build\Debug\pcre_stringpiece_unittest.exe pcrecpp_unittest.vcxproj -> C:\Users\faste\bearcove\poppler-build\pcre-8.45\build\Debug\pcrecpp_unittest.exe Building Custom Rule C:/Users/faste/bearcove/poppler-build/pcre-8.45/CMakeLists.txt

And install?

PowerShell session
$ cmake --install build -- Install configuration: "Release" CMake Error at build/cmake_install.cmake:39 (file): file INSTALL cannot find "C:/Users/faste/bearcove/poppler-build/pcre-8.45/build/Release/pcre.lib": No error.

Oh, huh, it's trying to install a release build but we made a debug build.

Fine, let's fix our mistakes...

PowerShell session
$ cmake -S . -B build -DCMAKE_BUILD_TYPE=Release -DCMAKE_INSTALL_PREFIX=C:/Users/faste/bearcove/poppler-build/prefix (cut) $ cmake --build build --config Release --parallel (Get-CimInstance Win32_ComputerSystem).NumberOfLogicalProcessors (cut) $ cmake --install .\build\ --config Release -- Installing: C:/Users/faste/bearcove/poppler-build/prefix/lib/pcre.lib -- Installing: C:/Users/faste/bearcove/poppler-build/prefix/lib/pcreposix.lib -- Installing: C:/Users/faste/bearcove/poppler-build/prefix/lib/pcrecpp.lib -- Installing: C:/Users/faste/bearcove/poppler-build/prefix/bin/pcregrep.exe (cut) -- Installing: C:/Users/faste/bearcove/poppler-build/prefix/lib/pkgconfig/libpcre.pc -- Installing: C:/Users/faste/bearcove/poppler-build/prefix/lib/pkgconfig/libpcrecpp.pc -- Installing: C:/Users/faste/bearcove/poppler-build/prefix/lib/pkgconfig/libpcreposix.pc -- Installing: C:/Users/faste/bearcove/poppler-build/prefix/bin/pcre-config

Oh look, it installed .pc files!

Now I guess we can get pkg-config to find those!

PowerShell session
$ pkg-config --static --cflags libpcre -DPCRE_STATIC -IC:/Users/faste/bearcove/poppler-build/prefix/include $ pkg-config --static --libs libpcre -LC:/Users/faste/bearcove/poppler-build/prefix/lib -lpcre

We can!

Mhh but those look like command-line flags for GCC/Clang. You just said the flags for MSVC looked different! (And then gave no example).

Well, time for an example then, because on Windows, pkg-config definitely has a setting for that:

PowerShell session
$ pkg-config --msvc-syntax --static --libs libpcre /libpath:C:/Users/faste/bearcove/poppler-build/prefix/lib pcre.lib

Look, it's different! Using /flag instead of -flag / --flag. It doesn't seem to change anything for --cflags though, so uhhh I'm curious how everything will work out.

Let's maybe try to make a quick project using libpcre from Rust with system-deps, to see how it'll handle that.

PowerShell session
$ cargo new pcre-test Created binary (application) `pcre-test` package $ cd pcre-test $ cargo add -B system-deps Updating 'https://github.com/rust-lang/crates.io-index' index Adding system-deps v6.0.0 to build-dependencies
Rust code
// in `pcre-test/build.rs` fn main() { system_deps::Config::new().probe().unwrap(); }
TOML markup
// in `pcre-test/Cargo.toml` [package.metadata.system-deps] libpcre = { version = "8.45" }
Rust code
// in `pcre-test/src/main.rs` use std::ffi::CStr; extern "C" { fn pcre_version() -> *const i8; } fn main() { let version = unsafe { CStr::from_ptr(pcre_version()) }; dbg!(&version); }

If we try cargo run, it fails!

PowerShell session
$ cargo run Compiling serde v1.0.130 Compiling smallvec v1.7.0 Compiling unicode-segmentation v1.8.0 Compiling pkg-config v0.3.22 Compiling version-compare v0.1.0 Compiling cfg-expr v0.9.0 Compiling heck v0.3.3 Compiling toml v0.5.8 Compiling system-deps v6.0.0 Compiling pcre-test v0.1.0 (C:\Users\faste\bearcove\pcre-test) error: failed to run custom build command for `pcre-test v0.1.0 (C:\Users\faste\bearcove\pcre-test)` Caused by: process didn't exit successfully: `C:\Users\faste\bearcove\pcre-test\target\debug\build\pcre-test-128e1f1f3d1e876c\build-script-build` (exit code: 101) --- stdout cargo:rerun-if-env-changed=LIBPCRE_NO_PKG_CONFIG cargo:rerun-if-env-changed=PKG_CONFIG_x86_64-pc-windows-msvc cargo:rerun-if-env-changed=PKG_CONFIG_x86_64_pc_windows_msvc cargo:rerun-if-env-changed=HOST_PKG_CONFIG cargo:rerun-if-env-changed=PKG_CONFIG cargo:rerun-if-env-changed=PKG_CONFIG_PATH_x86_64-pc-windows-msvc cargo:rerun-if-env-changed=PKG_CONFIG_PATH_x86_64_pc_windows_msvc cargo:rerun-if-env-changed=HOST_PKG_CONFIG_PATH cargo:rerun-if-env-changed=PKG_CONFIG_PATH cargo:rerun-if-env-changed=PKG_CONFIG_LIBDIR_x86_64-pc-windows-msvc cargo:rerun-if-env-changed=PKG_CONFIG_LIBDIR_x86_64_pc_windows_msvc cargo:rerun-if-env-changed=HOST_PKG_CONFIG_LIBDIR cargo:rerun-if-env-changed=PKG_CONFIG_LIBDIR cargo:rerun-if-env-changed=PKG_CONFIG_SYSROOT_DIR_x86_64-pc-windows-msvc cargo:rerun-if-env-changed=PKG_CONFIG_SYSROOT_DIR_x86_64_pc_windows_msvc cargo:rerun-if-env-changed=HOST_PKG_CONFIG_SYSROOT_DIR cargo:rerun-if-env-changed=PKG_CONFIG_SYSROOT_DIR --- stderr thread 'main' panicked at 'called `Result::unwrap()` on an `Err` value: PkgConfig(`"pkg-config" "--libs" "--cflags" "libpcre" "libpcre >= 8.45"` did not exit successfully: exit code: 1 --- stderr Package libpcre was not found in the pkg-config search path. Perhaps you should add the directory containing `libpcre.pc' to the PKG_CONFIG_PATH environment variable No package 'libpcre' found Package libpcre was not found in the pkg-config search path. Perhaps you should add the directory containing `libpcre.pc' to the PKG_CONFIG_PATH environment variable No package 'libpcre' found )', build.rs:2:40 note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace

...as expected!

Let's just set PKG_CONFIG_PATH and...

PowerShell session
$ $env:PKG_CONFIG_PATH = "C:/Users/faste/bearcove/poppler-build/prefix/lib/pkgconfig" $ cargo run Compiling pcre-test v0.1.0 (C:\Users\faste\bearcove\pcre-test) Finished dev [unoptimized + debuginfo] target(s) in 0.39s Running `target\debug\pcre-test.exe` [src\main.rs:9] &version = "8.45 2021-06-15"

It works perfectly! I'm really happy about that. There was hardly a mention of windows in the system-deps docs, it was honestly a 50/50 between "just works" and "woops sorry we assumed Linux all along".

For learning purposes, I did scoop uninstall pkg-config, and cargo clean && cargo run failed - it does need the real thing in path, they haven't reimplemented pkg-config as a Rust library or anything.

Should they?

I don't know, can you think of an argument for doing that?

Well, you wouldn't need to install pkg-config on Windows. It's a pretty nonstandard tool to have on there...

And can you think of an argument against doing that?

I suppose the Rust implementation would have to have the same hardcoded paths as the "real" pkg-config on Linux (or shell out to it to discover them?), and there's always the risk the implementations could drift apart...

Exactly! So, should they?

Ehhhhhhhhhhhhhhhhh.

Exactly.

So, at this point, in this article about checks notes vector graphics, all we need to do is build all those libraries and we should be good, right?

But to be completely honest with y'all, I don't feel like porting my build scripts to Windows, and maintaining them so that they work on both Windows and Linux.

This feels like a lot of work, /especially/ since I discovered that Meson can download and compile dependencies.

A crash course in meson wraps

Alright, let's start simple! We'll make a new folder, poppler-meson, and add this file:

meson.build file
# in `poppler-meson/meson.build` project('poppler-meson', 'c', version: '1.0.0', meson_version: '>= 0.60.1' ) glib_dep = dependency('glib-2.0')

This should only pull in glib and its dependencies. Right now, our project fails to configure:

PowerShell session
$ meson setup build --wipe The Meson build system Version: 0.60.1 Source dir: C:\Users\faste\bearcove\poppler-meson Build dir: C:\Users\faste\bearcove\poppler-meson\build Build type: native build Project name: poppler-meson Project version: 1.0.0 Activating VS 17.0.1 C compiler for the host machine: cl (msvc 19.30.30705 "Microsoft (R) C/C++ Optimizing Compiler Version 19.30.30705 for x64") C linker for the host machine: link link 14.30.30705.0 Host machine cpu family: x86_64 Host machine cpu: x86_64 Found pkg-config: C:\Users\faste\scoop\shims\pkg-config.EXE (0.26) Found CMake: C:\Program Files (x86)\Microsoft Visual Studio\2022\BuildTools\Common7\IDE\CommonExtensions\Microsoft\CMake\CMake\bin\cmake.EXE (3.21.21080301) Traceback (most recent call last): (cut) Build type: native build Project name: poppler-meson Project version: 1.0.0 Activating VS 17.0.1 C compiler for the host machine: cl (msvc 19.30.30705 "Microsoft (R) C/C++ Optimizing Compiler Version 19.30.30705 for x64") C linker for the host machine: link link 14.30.30705.0 Host machine cpu family: x86_64 Host machine cpu: x86_64 Found pkg-config: C:\Users\faste\scoop\shims\pkg-config.EXE (0.26) Found CMake: C:\Program Files (x86)\Microsoft Visual Studio\2022\BuildTools\Common7\IDE\CommonExtensions\Microsoft\CMake\CMake\bin\cmake.EXE (3.21.21080301) Run-time dependency glib-2.0 found: NO (tried pkgconfig and cmake) meson.build:6:0: ERROR: Dependency "glib-2.0" not found, tried pkgconfig and cmake A full log can be found at C:\Users\faste\bearcove\poppler-meson\build\meson-logs\meson-log.txt

And that's because we added a dependency on glib-2.0, but didn't give meson any information on where to find it. So, it tried to find it with pkg-config and cmake, but it didn't. On Fedora, with glib2-devel installed, this would probably work.

But we're on Windows now! So meson needs additional info. Luckily, meson now comes with "wraptool", which lets us pick from a list of libraries that have been "wrapped" so they can be built with meson instead of whatever their maintainers thought they should be built with (autotools, cmake, a home-grown organic set of Makefiles, etc.)

PowerShell session
$ meson wrap search glib-2.0 Dependency glib-2.0 found in wrap glib

Hey there it is! Let's add it:

PowerShell session
$ meson wrap install glib Installed glib version 2.70.1 revision 1

Just like that, wraptool added the subprojects/glib.wrap file:

meson .wrap file
[wrap-file] directory = glib-2.70.1 source_url = https://download.gnome.org/sources/glib/2.70/glib-2.70.1.tar.xz source_filename = glib-2.70.1.tar.xz source_hash = f9b7bce7f51753a1f43853bbcaca8bf09e15e994268e29cfd7a76f65636263c0 [provide] dependency_names = gthread-2.0, gobject-2.0, gmodule-no-export-2.0, gmodule-export-2.0, gmodule-2.0, glib-2.0, gio-2.0, gio-win32-2.0, gio-unix-2.0 program_names = glib-genmarshal, glib-mkenums, glib-compile-schemas, glib-compile-resources, gio-querymodules, gdbus-codegen

(With hashes and everything!)

Running meson setup again goes further! It actually crates subprojects/libpcre.wrap, since it's required by glib:

meson .wrap file
[wrap-redirect] filename = glib-2.70.1\subprojects\libpcre.wrap

But unfortunately, it eventually fails, because the default mirror for pcre sources hates me, apparently:

PowerShell session
$ meson setup --wipe build --prefix C:\Users\faste\bearcove\poppler-meson\prefix\ --default-library static (cut) glib| Run-time dependency libpcre found: NO (tried pkgconfig and cmake) glib| Library pcred found: NO glib| Run-time dependency libpcre found: NO (tried pkgconfig and cmake) glib| Looking for a fallback subproject for the dependency libpcre glib| Using subprojects\glib-2.70.1\subprojects\libpcre.wrap glib| Downloading libpcre source from https://ftp.pcre.org/pub/pcre/pcre-8.37.tar.bz2 glib| <urlopen error [WinError 10060] A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond>glib| A fallback URL could be specified using source_fallback_url key in the wrap file subprojects\glib-2.70.1\meson.build:2002:2: ERROR: could not get https://ftp.pcre.org/pub/pcre/pcre-8.37.tar.bz2 is the internet available?

Luckily, there's other mirrors we can pick from! By replacing the contents of subprojects/libpcre.wrap with a slightly tuned version of subprojects/glib-2.70.1/subprojects/libpcre.wrap...

meson .wrap file
[wrap-file] directory = pcre-8.37 source_url = https://ftp.halifax.rwth-aachen.de/osdn/sfnet/p/pc/pcre/pcre/8.37/pcre-8.37.tar.bz2 source_filename = pcre-8.37.tar.bz2 source_hash = 51679ea8006ce31379fb0860e46dd86665d864b5020fc9cd19e71260eef4789d patch_filename = pcre_8.37-2_patch.zip patch_url = https://wrapdb.mesonbuild.com/v2/pcre_8.37-2/get_patch patch_hash = 6b80f72385e1bf06721e26fbc83aced576e9c0d3182d86a55dd173a04050fe26 [provide] libpcre = pcre_dep

...we can keep going.

PowerShell session
$ meson setup --wipe build --prefix C:\Users\faste\bearcove\poppler-meson\prefix\ --default-library static (cut) Build targets in project: 341 poppler-meson 1.0.0 Subprojects glib : YES 3 warnings libffi : YES libpcre : YES proxy-libintl : YES zlib : YES User defined options default_library: static prefix : C:\Users\faste\bearcove\poppler-meson\prefix\ Found ninja-1.10.2 at C:\Users\faste\scoop\shims\ninja.EXE Visual Studio environment is needed to run Ninja. It is recommended to use Meson wrapper: C:\Users\faste\AppData\Local\Programs\Python\Python310\Scripts\meson compile -C build

Look at that, it pulled in zlib, libffi and proxy-libintl as well!

It did! How convenient.

A short meson compile -C build and many MSVC warnings later... it fails to build. Well, it turns out glib static builds on Windows are kind of a work-in-progress, so I had to learn about a few more meson features...

Some C defines were missing, but luckily, we can override them at the project level. We can even have the project default to a static build.

As for the glib dependency, we can also set things there, like ask for it to be statically linked with static: true, and set some default options, like: "don't bother with tests".

meson.build file
# in poppler-meson/meson.build project('poppler-meson', 'c', version: '1.0.0', meson_version: '>= 0.60.1', default_options: ['c_args=-DG_INTL_STATIC_COMPILATION=1 -DFFI_STATIC_BUILD=1', 'default_library=static'], ) glib_dep = dependency('glib-2.0', static: true, default_options: ['tests=false'])

And because glib really insists on including DllMain in its multiple libraries (gobject, gio, etc.), even when they're built to a static .a library on Windows, I had to bring in a little patch.

Which is a thing meson lets you do! Usually to "add meson build files", but what's a little crime among friends?

meson .wrap file
# in `subprojects/glib.wrap` [wrap-file] directory = glib-2.70.1 source_url = https://download.gnome.org/sources/glib/2.70/glib-2.70.1.tar.xz source_filename = glib-2.70.1.tar.xz source_hash = f9b7bce7f51753a1f43853bbcaca8bf09e15e994268e29cfd7a76f65636263c0 # 👇 new! patch_directory = glib-2.70.1 [provide] dependency_names = gthread-2.0, gobject-2.0, gmodule-no-export-2.0, gmodule-export-2.0, gmodule-2.0, glib-2.0, gio-2.0, gio-win32-2.0, gio-unix-2.0 program_names = glib-genmarshal, glib-mkenums, glib-compile-schemas, glib-compile-resources, gio-querymodules, gdbus-codegen

The patch is actually not a patch, more like an overlay, so I had to create subprojects/packagefiles/glib-2.70.1/glib/glib-init.c with my already-patched version of this source file. Handy!

And finally, it builds:

PowerShell session
$ meson compile -C build Activating VS 17.0.1 ninja: Entering directory `C:/Users/faste/bearcove/poppler-meson/build' (cut) [361/361] Linking target subprojects/glib-2.70.1/fuzzing/fuzz_inet_socket_address_new_from_string.exe

And installs:

PowerShell session
$ meson install -C build ninja: Entering directory `C:\Users\faste\bearcove\poppler-meson\build' ninja: no work to do. Installing subprojects\libffi\src\libffi.a to C:\Users\faste\bearcove\poppler-meson\prefix\lib (cut) Installing C:\Users\faste\bearcove\poppler-meson\subprojects\glib-2.70.1\m4macros/gsettings.m4 to C:\Users\faste\bearcove\poppler-meson\prefix\share/aclocal
Cool bear's hot tip

Oddly enough, meson install wouldn't find ninja the way meson compile did, and a scoop install ninja fixed it there.

Another option would've been meson devenv -C build, which enters a "dev environment", with C/C++ compilers/linkers (like cl.exe, link.exe) and build tools (cmake.exe, ninja.exe) in PATH.

So let's look at what meson/ninja has actually installed in our prefix.

PowerShell session
$ dir prefix\lib\pkgconfig Directory: C:\Users\faste\bearcove\poppler-meson\prefix\lib\pkgconfig Mode LastWriteTime Length Name ---- ------------- ------ ---- -a--- 01/12/2021 17:34 751 gio-2.0.pc -a--- 01/12/2021 17:34 311 gio-windows-2.0.pc -a--- 01/12/2021 17:34 428 glib-2.0.pc -a--- 01/12/2021 17:34 278 gmodule-2.0.pc -a--- 01/12/2021 17:34 278 gmodule-export-2.0.pc -a--- 01/12/2021 17:34 295 gmodule-no-export-2.0.pc -a--- 01/12/2021 17:34 294 gobject-2.0.pc -a--- 01/12/2021 17:34 262 gthread-2.0.pc -a--- 01/12/2021 17:34 248 libffi.pc -a--- 01/12/2021 17:34 223 zlib.pc

That is great, we definitely needed a bunch of these.

Our static libraries are all installed in prefix\lib\pkgconfig as well:

PowerShell session
$ dir prefix\lib Directory: C:\Users\faste\bearcove\poppler-meson\prefix\lib Mode LastWriteTime Length Name ---- ------------- ------ ---- -a--- 01/12/2021 18:26 127230 libffi.a -a--- 01/12/2021 18:26 13584544 libgio-2.0.a -a--- 01/12/2021 18:26 6860652 libglib-2.0.a -a--- 01/12/2021 18:26 77578 libgmodule-2.0.a -a--- 01/12/2021 18:26 1561240 libgobject-2.0.a -a--- 01/12/2021 18:26 19220 libgthread-2.0.a -a--- 01/12/2021 18:26 28104 libintl.a -a--- 01/12/2021 18:26 309480 libz.a

I didn't immediately noticed something was wrong, because it had been a while since I had to think about MSVC idiosyncracies, but...

PowerShell session
$ cd glib-test $ cargo run Compiling glib-test v0.1.0 (C:\Users\faste\bearcove\glib-test) error: linking with `link.exe` failed: exit code: 1181 | (cut) = note: LINK : fatal error LNK1181: cannot open input file 'glib-2.0.lib' error: could not compile `glib-test` due to previous error

...that's not how libraries should be named for MSVC. They shouldn't be named libfoobar.a, they should be named foobar.lib.

Luckily, a single PowerShell command lets us solve that (quickly adds powershell to the tags of this article):

PowerShell session
$ Get-ChildItem .\prefix\lib\lib*.a | Rename-Item -NewName { $_.Name -replace "^lib", "" -replace ".a$", ".lib" } $ Get-ChildItem .\prefix\lib\*.lib Directory: C:\Users\faste\bearcove\poppler-meson\prefix\lib Mode LastWriteTime Length Name ---- ------------- ------ ---- -a--- 01/12/2021 18:26 127230 ffi.lib -a--- 01/12/2021 18:26 13584544 gio-2.0.lib -a--- 01/12/2021 18:26 6860652 glib-2.0.lib -a--- 01/12/2021 18:26 77578 gmodule-2.0.lib -a--- 01/12/2021 18:26 1561240 gobject-2.0.lib -a--- 01/12/2021 18:26 19220 gthread-2.0.lib -a--- 01/12/2021 18:26 28104 intl.lib -a--- 01/12/2021 18:26 309480 z.lib

Tada! Wish zsh would let me do that this easily. Maybe nushell can?

Anyway, now it works:

PowerShell session
$ cargo run (cut) LINK : warning LNK4098: defaultlib 'MSVCRTD' conflicts with use of other libs; use /NODEFAULTLIB:library (cut)

Nope, nope, go back, meson defaults to a debug build, we want a release build for sure.

PowerShell session
$ meson setup build --prefix C:\Users\faste\bearcove\poppler-meson\prefix\ --buildtype release --wipe $ meson compile -C build $ meson install -C build $ # don't forget to rename libfoobar.a to foobar.lib again...

Ok surely now it'll work:

PowerShell session
$ cargo run (cut) = note: glib-2.0.lib(gutils.c.obj) : error LNK2019: unresolved external symbol __imp_CoTaskMemFree referenced in function g_build_home_dir glib-2.0.lib(gutils.c.obj) : error LNK2019: unresolved external symbol SHGetKnownFolderPath referenced in function g_build_home_dir glib-2.0.lib(gmessages.c.obj) : error LNK2019: unresolved external symbol __imp_MessageBoxW referenced in function g_log_writer_default glib-2.0.lib(gwin32.c.obj) : error LNK2019: unresolved external symbol __imp_CommandLineToArgvW referenced in function g_win32_get_command_line glib-2.0.lib(giowin32.c.obj) : error LNK2019: unresolved external symbol __imp_PeekMessageA referenced in function g_io_win32_check glib-2.0.lib(giowin32.c.obj) : error LNK2019: unresolved external symbol __imp_PostMessageA referenced in function g_io_win32_msg_write glib-2.0.lib(giowin32.c.obj) : error LNK2019: unresolved external symbol __imp_IsWindow referenced in function g_io_channel_win32_new_messages glib-2.0.lib(gpoll.c.obj) : error LNK2019: unresolved external symbol __imp_MsgWaitForMultipleObjectsEx referenced in function g_poll C:\Users\faste\bearcove\glib-test\target\debug\deps\glib_test.exe : fatal error LNK1120: 8 unresolved externals error: could not compile `glib-test` due to previous error

Err almost. Just missing a couple libraries...

Rust code
// in `glib-test/build.rs` fn main() { system_deps::Config::new().probe().unwrap(); let target = std::env::var("TARGET").unwrap(); if target.contains("msvc") { println!("cargo:rustc-link-lib=ole32"); println!("cargo:rustc-link-lib=user32"); println!("cargo:rustc-link-lib=shell32"); } }
PowerShell session
$ cargo run Compiling glib-test v0.1.0 (C:\Users\faste\bearcove\glib-test) Finished dev [unoptimized + debuginfo] target(s) in 0.78s Running `target\debug\glib-test.exe` [src\main.rs:9] &home_dir = "C:\\Users\\faste"

YES! Yes! Finally!

The source for this sample program, by the way, is:

Rust code
// in `glib-test/src/main.rs` use std::ffi::CStr; extern "C" { fn g_get_home_dir() -> *const i8; } fn main() { let home_dir = unsafe { CStr::from_ptr(g_get_home_dir()) }; dbg!(&home_dir); }

And the glib-2.0 dependency is specified like so:

TOML markup
[package.metadata.system-deps] "glib" = { name = "glib-2.0", version = "2.70.1" }

And I include it so you can find out that the system dep's key cannot include dots, but that you can save it by specifying "name".

Okay! So that's glib. I mean, that's glib.

What else do we need... well, cairo's a big one. And good news, cairo, alongside autotools, actually ships its own meson.build file!

But that still leaves poppler itself. And it doesn't ship with its own meson build files. It ships with CMake, and in theory, I could try to port everything to CMake, and if I did, I would use Izzy's wonderful CMake articles, like Everything you never wanted to know about CMake, and How to Find Packages With CMake.

This time, I chose to go the opposite route: everything except poppler already had meson build files, so it seemed like a good opportunity for me to simply write meson.build files for poppler! Which is exactly what I did.

This article was made possible thanks to my patrons: Alexander Payne, Fredrik Østrem, David Barsky, Yufan Lou, Stephen Molyneaux, Barret Rennie, Thomas Corbin, MW, Jacob Cheriathundam, Michael Watzko, Embark Studios, Eugene Bulkin, Marcus Griep, Petar Radosevic, Tool Army, Tully, Santiago Lema, Spencer Gilbert, Jörn Huxhorn, Garrett Ward, DEX, Christian Oudard, Ronen Cohen, Thor Kamphefner, Kamran Khan, Cole Kurkowski, Arjen Laarhoven, Vicente Bosch, Chirag Jain, Ville Mattila, Marie Janssen, Vladyslav Batyrenko, Cameron Clausen, spike grobstein, Jon Gjengset, Paul Marques Mota, Jakub Fijałkowski, Mitchell Hamilton, Brad Luyster, Max von Forell, Jake S, Dimitri Merejkowsky, Chris Biscardi, René Ribaud, Alex Doroshenko, Vincent, Steven McGuire, Chad Birch, Chris Emery, Bob Ippolito, John Van Enk, metabaron, Isak Sunde Singh, Philipp Gniewosz, Mads Johansen, lukvol, Ives van Hoorne, Jan De Landtsheer, Daniel Strittmatter, Evgeniy Dubovskoy, Alex Rudy, Shane Lillie, Romet Tagobert, Douglas Creager, Corey Alexander, Molly Howell, knutwalker, Zachary Dremann, Sebastian Ziebell, Julien Roncaglia, Amber Kowalski, T, queenfartbutt, Paul Kline, Kristoffer Ström, Astrid Bek, Yoh Deadfall, Justin Ossevoort, Tomáš Duda, Jeremy Banks, Rasmus Larsen, Torben Clasen, C J Silverio, Walther, Pete Bevin, Shane Sveller, Clara Schultz, jer, Wonwoo Choi, Hawken Rives, João Veiga, Richard Pringle, Adam Perry, Benjamin Röjder Delnavaz, Matt Jadczak, Jonathan Knapp, Maximilian, Seth Stadick, brianloveswords, Sean Bryant, Ember, Sebastian Zimmer, Makoto Nakashima, Geoff Cant, Geoffroy Couprie, Michael Alyn Miller, o0Ignition0o, Zaki, Raphael Gaschignard, Romain Ruetschi, Ignacio Vergara, Pascal, Jane Lusby, Nicolas Goy, Ted Mielczarek, Aurora.

This article is part 4 of the Don't shell out! series.

Read the next part

If you liked this article, please support my work on Patreon!

Become a Patron

Looking for the homepage?
Another article: Small strings in Rust