Building poppler for Windows
fasterthanlime
This article is part of the Don't shell out! series.
- A crash course in meson wraps
Contents
I know what you're thinking: haven't we strayed from the whole "content pipeline" theme in this series?
Well... fair. But compiling and distributing software is part of software engineering, and unless you're in specific circles, I see that taught a lot less than the "just write code and stuff happens" part.
Technically it's release engineering, but who's keeping track.
So we're going to think about how we could build all this for Windows. Because every single of my internal tools for my website builds for Windows just fine, in case I, you know, need to write a series on Windows, from Windows.
So my little fork of salvage that uses headless_chrome to render a .drawio
file to /tmp/export.pdf? That compiles and runs on Windows!
$ .\target\debug\salvage.exe
2021-11-26T13:56:29.450869Z INFO headless_chrome::browser::fetcher: Getting project dir
2021-11-26T13:56:29.454840Z INFO headless_chrome::browser::process: Launching Chrome binary at "C:\\Users\\faste\\AppData\\Roaming\\headless-chrome\\data\\win-634997\\chrome-win\\chrome.exe"
2021-11-26T13:56:29.533551Z INFO headless_chrome::browser::process: Started Chrome. PID: 18936
2021-11-26T13:56:29.712694Z INFO salvage::chrome_stuff: Navigating...
2021-11-26T13:56:30.044934Z INFO headless_chrome::browser::tab: Navigating a tab to http://localhost:5000/index.html
2021-11-26T13:56:31.049236Z INFO salvage::chrome_stuff: Navigating... done!
2021-11-26T13:56:31.061422Z INFO salvage::chrome_stuff: Got dimensions width=994 height=643
2021-11-26T13:56:31.192107Z INFO salvage::chrome_stuff: Writing pdf... pdf_path=/tmp/export.pdf
2021-11-26T13:56:31.192335Z INFO headless_chrome::browser: Dropping browser
2021-11-26T13:56:31.192442Z INFO headless_chrome::browser::process: Killing Chrome. PID: 18936
2021-11-26T13:56:31.192552Z INFO headless_chrome::browser::transport::web_socket_connection: Sending shutdown message to message handling loop
2021-11-26T13:56:31.192694Z INFO headless_chrome::browser::transport: Received shutdown message
2021-11-26T13:56:31.192785Z INFO headless_chrome::browser::transport: Shutting down message handling loop
2021-11-26T13:56:31.192886Z INFO headless_chrome::browser::transport: cleared listeners, I think
2021-11-26T13:56:31.192904Z INFO headless_chrome::browser::tab: finished tab's event handling loop
2021-11-26T13:56:31.192900Z INFO headless_chrome::browser: Finished browser's event handling loop
2021-11-26T13:56:31.198980Z INFO headless_chrome::browser::transport: dropping transport
2021-11-26T13:56:31.199044Z INFO headless_chrome::browser::transport::web_socket_connection: dropping websocket connection
The application panicked (crashed).
Message: called `Result::unwrap()` on an `Err` value: Os { code: 3, kind: NotFound, message: "The system cannot find the path specified." }
Location: src\chrome_stuff.rs:17
Backtrace omitted.
Run with RUST_BACKTRACE=1 environment variable to display it.
Run with RUST_BACKTRACE=full to include source snippets.
Error:
0: chrome error: JoinError::Panic(...)
Location:
src\main.rs:47
Backtrace omitted.
Run with RUST_BACKTRACE=1 environment variable to display it.
Run with RUST_BACKTRACE=full to include source snippets.
Well huh... ok maybe we can't write to /tmp/export.pdf. Let's quickly turn this:
let pdf_path = Utf8PathBuf::from("/tmp/export.pdf");
Into this:
let pdf_path = Utf8PathBuf::try_from(std::env::temp_dir())?.join("export.pdf");
And then... it runs!
$ cargo run
Compiling salvage v1.2.0 (C:\Users\faste\bearcove\salvage)
Finished dev [unoptimized + debuginfo] target(s) in 2.75s
Running `target\debug\salvage.exe`
2021-11-26T14:03:34.463367Z INFO headless_chrome::browser::fetcher: Getting project dir
2021-11-26T14:03:34.468192Z INFO headless_chrome::browser::process: Launching Chrome binary at "C:\\Users\\faste\\AppData\\Roaming\\headless-chrome\\data\\win-634997\\chrome-win\\chrome.exe"
2021-11-26T14:03:34.550763Z INFO headless_chrome::browser::process: Started Chrome. PID: 14984
2021-11-26T14:03:34.853213Z INFO salvage::chrome_stuff: Navigating...
2021-11-26T14:03:35.188660Z INFO headless_chrome::browser::tab: Navigating a tab to http://localhost:5000/index.html
2021-11-26T14:03:37.092808Z INFO salvage::chrome_stuff: Navigating... done!
2021-11-26T14:03:37.104829Z INFO salvage::chrome_stuff: Got dimensions width=994 height=643
2021-11-26T14:03:37.241291Z INFO salvage::chrome_stuff: Writing pdf... pdf_path=C:\Users\faste\AppData\Local\Temp\export.pdf
2021-11-26T14:03:37.241867Z INFO salvage::chrome_stuff: Writing pdf... done! pdf_path=C:\Users\faste\AppData\Local\Temp\export.pdf
2021-11-26T14:03:37.241957Z INFO headless_chrome::browser: Dropping browser
2021-11-26T14:03:37.242089Z INFO headless_chrome::browser::process: Killing Chrome. PID: 14984
2021-11-26T14:03:37.242193Z INFO headless_chrome::browser::transport::web_socket_connection: Sending shutdown message to message handling loop
2021-11-26T14:03:37.242320Z INFO headless_chrome::browser::transport: Received shutdown message
2021-11-26T14:03:37.242398Z INFO headless_chrome::browser::transport: Shutting down message handling loop
2021-11-26T14:03:37.242484Z INFO headless_chrome::browser::transport: cleared listeners, I think
2021-11-26T14:03:37.242521Z INFO headless_chrome::browser::tab: finished tab's event handling loop
2021-11-26T14:03:37.242516Z INFO headless_chrome::browser: Finished browser's event handling loop
Isn't it amazing that this was the only change we needed to make?
Heck yeah!
This goes to show how great of a job crate maintainers have been doing!
Our diagram looks just as fine on Windows:
$ start $env:TMP\export.pdf
Our pdftocairo executable however... it's a different story. Well, to be fair,
I have no idea what to expect, so let's find out:
$ cargo build
Updating crates.io index
Downloaded anyhow v1.0.47
Downloaded cairo-rs v0.14.9
(cut)
Downloaded glib-sys v0.14.0
Downloaded proc-macro-error v1.0.4
Downloaded 19 crates (615.9 KB) in 1.39s
Compiling proc-macro2 v1.0.32
Compiling unicode-xid v0.2.2
(cut)
Compiling poppler-sys-rs v0.18.0
The following warnings were emitted during compilation:
warning: Could not run `"pkg-config" "--libs" "--cflags" "glib-2.0" "glib-2.0 >= 2.48"`
error: failed to run custom build command for `glib-sys v0.14.0`
Caused by:
process didn't exit successfully: `C:\Users\faste\bearcove\pdftocairo\target\debug\build\glib-sys-0231afba2b42ecdf\build-script-build` (exit code: 1)
--- stdout
cargo:rerun-if-env-changed=GLIB_2.0_NO_PKG_CONFIG
cargo:rerun-if-env-changed=PKG_CONFIG_x86_64-pc-windows-msvc
cargo:rerun-if-env-changed=PKG_CONFIG_x86_64_pc_windows_msvc
cargo:rerun-if-env-changed=HOST_PKG_CONFIG
cargo:rerun-if-env-changed=PKG_CONFIG
cargo:rerun-if-env-changed=PKG_CONFIG_PATH_x86_64-pc-windows-msvc
cargo:rerun-if-env-changed=PKG_CONFIG_PATH_x86_64_pc_windows_msvc
cargo:rerun-if-env-changed=HOST_PKG_CONFIG_PATH
cargo:rerun-if-env-changed=PKG_CONFIG_PATH
cargo:rerun-if-env-changed=PKG_CONFIG_LIBDIR_x86_64-pc-windows-msvc
cargo:rerun-if-env-changed=PKG_CONFIG_LIBDIR_x86_64_pc_windows_msvc
cargo:rerun-if-env-changed=HOST_PKG_CONFIG_LIBDIR
cargo:rerun-if-env-changed=PKG_CONFIG_LIBDIR
cargo:rerun-if-env-changed=PKG_CONFIG_SYSROOT_DIR_x86_64-pc-windows-msvc
cargo:rerun-if-env-changed=PKG_CONFIG_SYSROOT_DIR_x86_64_pc_windows_msvc
cargo:rerun-if-env-changed=HOST_PKG_CONFIG_SYSROOT_DIR
cargo:rerun-if-env-changed=PKG_CONFIG_SYSROOT_DIR
cargo:warning=Could not run `"pkg-config" "--libs" "--cflags" "glib-2.0" "glib-2.0 >= 2.48"`
The pkg-config command could not be found.
Most likely, you need to install a pkg-config package for your OS.
If you've already installed it, ensure the pkg-config command is one of the
directories in the PATH environment variable.
If you did not expect this build to link to a pre-installed system library,
then check documentation of the glib-sys crate for an option to
build the library from source, or disable features or dependencies
that require pkg-config.
warning: build failed, waiting for other jobs to finish...
error: build failed
Well, system-deps wasn't lying, it really does only support pkg-config!
pkg-config can be built for windows, and in fact, we can install it with
scoop:
$ scoop install pkg-config Installing 'pkg-config' (0.26-1) [64bit] Loading pkg-config_0.26-1_win32.zip from cache Checking hash of pkg-config_0.26-1_win32.zip ... ok. Loading glib_2.28.8-1_win32.zip from cache Checking hash of glib_2.28.8-1_win32.zip ... ok. Loading gettext-runtime_0.18.1.1-2_win32.zip from cache Checking hash of gettext-runtime_0.18.1.1-2_win32.zip ... ok. Extracting pkg-config_0.26-1_win32.zip ... done. Extracting glib_2.28.8-1_win32.zip ... done. Extracting gettext-runtime_0.18.1.1-2_win32.zip ... done. Linking ~\scoop\apps\pkg-config\current => ~\scoop\apps\pkg-config\0.26-1 Creating shim for 'pkg-config'. 'pkg-config' (0.26-1) was installed successfully!
But of course, it can't find anything:
$ pkg-config --cflags --libs cairo Package cairo was not found in the pkg-config search path. Perhaps you should add the directory containing `cairo.pc' to the PKG_CONFIG_PATH environment variable No package 'cairo' found
Where does it even look?
Well bear, you know what RTFM means?
Read the.. fine manual?
Yes! But after a couple searches, I cannot for the life of me figure
out where pkg-config looks for .pc files on Windows by default.
So what do we do then?
Well, even if I had found something, the docs always lie. So what we do is spy on the process instead.
But this is Windows, we don't have strace. What we do have, is Process
Monitor.
And now we have our answer! It looks in C:\Users\faste\scoop\apps\pkg-config\current\share\pkgconfig, which
is a symbolic link to the currently installed version:
$ Get-Item C:\Users\faste\scoop\apps\pkg-config\current\
Directory: C:\Users\faste\scoop\apps\pkg-config
Mode LastWriteTime Length Name
---- ------------- ------ ----
l-r-- 26/11/2021 15:11 current -> C:\Users\faste\scoop\apps\pkg-config\0.26-1
So, yay, there's a global path! But nothing stable like /usr/lib64/pkgconfig
on Fedora, or /usr/lib/x86_64-linux-gnu/pkgconfig on Ubuntu.
I'm sure if we install our .pc files somewhere though, pkg-config will be
able to find them, right?
Well, our build scripts right now are bash, so... unless we install bash, we
can't run them as-is, but we can try to run similar instructions from a more
Windows-y shell, like PowerShell?
Let's try with pcre:
$ curl -f -L -o pcre.tar.bz2 https://sourceforge.net/projects/pcre/files/pcre/8.45/pcre-8.45.tar.bz2/download
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 605 100 605 0 0 605 0 0:00:01 --:--:-- 0:00:01 880
100 331 100 331 0 0 331 0 0:00:01 0:00:01 --:--:-- 331
100 1541k 100 1541k 0 0 1541k 0 0:00:01 0:00:01 --:--:-- 9883k
So far so good! Apparently Windows 11 ships the real curl, not just an alias?
$ Get-Command curl CommandType Name Version Source ----------- ---- ------- ------ Application curl.exe 7.55.1.0 C:\Windows\system32\curl.exe
Next we need to extract the archive. Apparently tar also ships with Windows
11!
$ tar xjf .\pcre.tar.bz2 $ cd .\pcre-8.45\
Now, what build system is PCRE using again? Autotools? Right. Well...
...that's not gonna work.
Luckily, PCRE also comes with a CMakeLists.txt, so we can "just" use cmake!
Visual Studio 2022 Build Tools ships with cmake, it's just a bit hidden, but if we start the "x64 Native Tools Command Prompt for VS 2022", it's right there in the PATH:
$ Get-Command cmake CommandType Name Version Source ----------- ---- ------- ------ Application cmake.exe 3.21.2108… C:\Program Files (x86)\Microsoft Visual Studio\2022\BuildTools\Common7\IDE\CommonExtensions\Microsoft\CMake\CMake\bin\cmake.exe
So, let's make a build folder and see what happens!
$ mkdir build $ cmake -S . -B build -- Building for: Visual Studio 17 2022 -- Selecting Windows SDK version 10.0.19041.0 to target Windows 10.0.22000. -- The C compiler identification is MSVC 19.30.30705.0 -- The CXX compiler identification is MSVC 19.30.30705.0 (cut) -- Could NOT find BZip2 (missing: BZIP2_LIBRARIES BZIP2_INCLUDE_DIR) -- Could NOT find ZLIB (missing: ZLIB_LIBRARY ZLIB_INCLUDE_DIR) -- Could not find OPTIONAL package Readline -- Could not find OPTIONAL package Editline -- Looking for dirent.h -- Looking for dirent.h - not found (cut) -- -- -- PCRE-8.45 configuration summary: -- -- Install prefix .................. : C:/Program Files (x86)/PCRE -- C compiler ...................... : C:/Program Files (x86)/Microsoft Visual Studio/2022/BuildTools/VC/Tools/MSVC/14.30.30705/bin/Hostx64/x64/cl.exe -- C++ compiler .................... : C:/Program Files (x86)/Microsoft Visual Studio/2022/BuildTools/VC/Tools/MSVC/14.30.30705/bin/Hostx64/x64/cl.exe -- C compiler flags ................ : /DWIN32 /D_WINDOWS /W3 -- C++ compiler flags .............. : /DWIN32 /D_WINDOWS /W3 /GR /EHsc -- -- Build 8 bit PCRE library ........ : ON (cut) -- Install MSVC .pdb files ..........: OFF -- -- Configuring done -- Generating done -- Build files have been written to: C:/Users/faste/bearcove/poppler-build/pcre-8.45/build
Ah, forgot to specify a prefix, let's do that now:
$ cmake -S . -B build -DCMAKE_INSTALL_PREFIX=C:/Users/faste/bearcove/poppler-build/prefix (output omitted)
And.. let's build?
$ cmake --build build --parallel (Get-CimInstance Win32_ComputerSystem).NumberOfLogicalProcessors Microsoft (R) Build Engine version 17.0.0+c9eb9dd64 for .NET Framework Copyright (C) Microsoft Corporation. All rights reserved. Checking Build System Building Custom Rule C:/Users/faste/bearcove/poppler-build/pcre-8.45/CMakeLists.txt pcre_byte_order.c pcre_chartables.c pcre_compile.c (cut) pcre_stringpiece_unittest.vcxproj -> C:\Users\faste\bearcove\poppler-build\pcre-8.45\build\Debug\pcre_stringpiece_unittest.exe pcrecpp_unittest.vcxproj -> C:\Users\faste\bearcove\poppler-build\pcre-8.45\build\Debug\pcrecpp_unittest.exe Building Custom Rule C:/Users/faste/bearcove/poppler-build/pcre-8.45/CMakeLists.txt
And install?
$ cmake --install build -- Install configuration: "Release" CMake Error at build/cmake_install.cmake:39 (file): file INSTALL cannot find "C:/Users/faste/bearcove/poppler-build/pcre-8.45/build/Release/pcre.lib": No error.
Oh, huh, it's trying to install a release build but we made a debug build.
Fine, let's fix our mistakes...
$ cmake -S . -B build -DCMAKE_BUILD_TYPE=Release -DCMAKE_INSTALL_PREFIX=C:/Users/faste/bearcove/poppler-build/prefix (cut) $ cmake --build build --config Release --parallel (Get-CimInstance Win32_ComputerSystem).NumberOfLogicalProcessors (cut) $ cmake --install .\build\ --config Release -- Installing: C:/Users/faste/bearcove/poppler-build/prefix/lib/pcre.lib -- Installing: C:/Users/faste/bearcove/poppler-build/prefix/lib/pcreposix.lib -- Installing: C:/Users/faste/bearcove/poppler-build/prefix/lib/pcrecpp.lib -- Installing: C:/Users/faste/bearcove/poppler-build/prefix/bin/pcregrep.exe (cut) -- Installing: C:/Users/faste/bearcove/poppler-build/prefix/lib/pkgconfig/libpcre.pc -- Installing: C:/Users/faste/bearcove/poppler-build/prefix/lib/pkgconfig/libpcrecpp.pc -- Installing: C:/Users/faste/bearcove/poppler-build/prefix/lib/pkgconfig/libpcreposix.pc -- Installing: C:/Users/faste/bearcove/poppler-build/prefix/bin/pcre-config
Oh look, it installed .pc files!
Now I guess we can get pkg-config to find those!
$ pkg-config --static --cflags libpcre -DPCRE_STATIC -IC:/Users/faste/bearcove/poppler-build/prefix/include $ pkg-config --static --libs libpcre -LC:/Users/faste/bearcove/poppler-build/prefix/lib -lpcre
We can!
Mhh but those look like command-line flags for GCC/Clang. You just said the flags for MSVC looked different! (And then gave no example).
Well, time for an example then, because on Windows, pkg-config definitely
has a setting for that:
$ pkg-config --msvc-syntax --static --libs libpcre /libpath:C:/Users/faste/bearcove/poppler-build/prefix/lib pcre.lib
Look, it's different! Using /flag instead of -flag / --flag. It doesn't
seem to change anything for --cflags though, so uhhh I'm curious how
everything will work out.
Let's maybe try to make a quick project using libpcre from Rust with
system-deps, to see how it'll handle that.
$ cargo new pcre-test
Created binary (application) `pcre-test` package
$ cd pcre-test
$ cargo add -B system-deps
Updating 'https://github.com/rust-lang/crates.io-index' index
Adding system-deps v6.0.0 to build-dependencies
// in `pcre-test/build.rs`
fn main() {
system_deps::Config::new().probe().unwrap();
}
// in `pcre-test/Cargo.toml`
[package.metadata.system-deps]
libpcre = { version = "8.45" }
// in `pcre-test/src/main.rs`
use std::ffi::CStr;
extern "C" {
fn pcre_version() -> *const i8;
}
fn main() {
let version = unsafe { CStr::from_ptr(pcre_version()) };
dbg!(&version);
}
If we try cargo run, it fails!
$ cargo run Compiling serde v1.0.130 Compiling smallvec v1.7.0 Compiling unicode-segmentation v1.8.0 Compiling pkg-config v0.3.22 Compiling version-compare v0.1.0 Compiling cfg-expr v0.9.0 Compiling heck v0.3.3 Compiling toml v0.5.8 Compiling system-deps v6.0.0 Compiling pcre-test v0.1.0 (C:\Users\faste\bearcove\pcre-test) error: failed to run custom build command for `pcre-test v0.1.0 (C:\Users\faste\bearcove\pcre-test)` Caused by: process didn't exit successfully: `C:\Users\faste\bearcove\pcre-test\target\debug\build\pcre-test-128e1f1f3d1e876c\build-script-build` (exit code: 101) --- stdout cargo:rerun-if-env-changed=LIBPCRE_NO_PKG_CONFIG cargo:rerun-if-env-changed=PKG_CONFIG_x86_64-pc-windows-msvc cargo:rerun-if-env-changed=PKG_CONFIG_x86_64_pc_windows_msvc cargo:rerun-if-env-changed=HOST_PKG_CONFIG cargo:rerun-if-env-changed=PKG_CONFIG cargo:rerun-if-env-changed=PKG_CONFIG_PATH_x86_64-pc-windows-msvc cargo:rerun-if-env-changed=PKG_CONFIG_PATH_x86_64_pc_windows_msvc cargo:rerun-if-env-changed=HOST_PKG_CONFIG_PATH cargo:rerun-if-env-changed=PKG_CONFIG_PATH cargo:rerun-if-env-changed=PKG_CONFIG_LIBDIR_x86_64-pc-windows-msvc cargo:rerun-if-env-changed=PKG_CONFIG_LIBDIR_x86_64_pc_windows_msvc cargo:rerun-if-env-changed=HOST_PKG_CONFIG_LIBDIR cargo:rerun-if-env-changed=PKG_CONFIG_LIBDIR cargo:rerun-if-env-changed=PKG_CONFIG_SYSROOT_DIR_x86_64-pc-windows-msvc cargo:rerun-if-env-changed=PKG_CONFIG_SYSROOT_DIR_x86_64_pc_windows_msvc cargo:rerun-if-env-changed=HOST_PKG_CONFIG_SYSROOT_DIR cargo:rerun-if-env-changed=PKG_CONFIG_SYSROOT_DIR --- stderr thread 'main' panicked at 'called `Result::unwrap()` on an `Err` value: PkgConfig(`"pkg-config" "--libs" "--cflags" "libpcre" "libpcre >= 8.45"` did not exit successfully: exit code: 1 --- stderr Package libpcre was not found in the pkg-config search path. Perhaps you should add the directory containing `libpcre.pc' to the PKG_CONFIG_PATH environment variable No package 'libpcre' found Package libpcre was not found in the pkg-config search path. Perhaps you should add the directory containing `libpcre.pc' to the PKG_CONFIG_PATH environment variable No package 'libpcre' found )', build.rs:2:40 note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
...as expected!
Let's just set PKG_CONFIG_PATH and...
$ $env:PKG_CONFIG_PATH = "C:/Users/faste/bearcove/poppler-build/prefix/lib/pkgconfig"
$ cargo run
Compiling pcre-test v0.1.0 (C:\Users\faste\bearcove\pcre-test)
Finished dev [unoptimized + debuginfo] target(s) in 0.39s
Running `target\debug\pcre-test.exe`
[src\main.rs:9] &version = "8.45 2021-06-15"
It works perfectly! I'm really happy about that. There was hardly a mention of
windows in the system-deps docs, it was honestly a 50/50 between "just works"
and "woops sorry we assumed Linux all along".
For learning purposes, I did scoop uninstall pkg-config, and cargo clean && cargo run failed - it does need the real thing in path, they haven't
reimplemented pkg-config as a Rust library or anything.
Should they?
I don't know, can you think of an argument for doing that?
Well, you wouldn't need to install pkg-config on Windows. It's a pretty nonstandard tool to have on there...
And can you think of an argument against doing that?
I suppose the Rust implementation would have to have the same hardcoded paths as the "real" pkg-config on Linux (or shell out to it to discover them?), and there's always the risk the implementations could drift apart...
Exactly! So, should they?
Ehhhhhhhhhhhhhhhhh.
Exactly.
So, at this point, in this article about checks notes vector graphics, all we need to do is build all those libraries and we should be good, right?
But to be completely honest with y'all, I don't feel like porting my build scripts to Windows, and maintaining them so that they work on both Windows and Linux.
This feels like a lot of work, /especially/ since I discovered that Meson can download and compile dependencies.
A crash course in meson wraps
Alright, let's start simple! We'll make a new folder, poppler-meson, and
add this file:
# in `poppler-meson/meson.build`
project('poppler-meson', 'c',
version: '1.0.0',
meson_version: '>= 0.60.1'
)
glib_dep = dependency('glib-2.0')
This should only pull in glib and its dependencies. Right now, our project
fails to configure:
$ meson setup build --wipe The Meson build system Version: 0.60.1 Source dir: C:\Users\faste\bearcove\poppler-meson Build dir: C:\Users\faste\bearcove\poppler-meson\build Build type: native build Project name: poppler-meson Project version: 1.0.0 Activating VS 17.0.1 C compiler for the host machine: cl (msvc 19.30.30705 "Microsoft (R) C/C++ Optimizing Compiler Version 19.30.30705 for x64") C linker for the host machine: link link 14.30.30705.0 Host machine cpu family: x86_64 Host machine cpu: x86_64 Found pkg-config: C:\Users\faste\scoop\shims\pkg-config.EXE (0.26) Found CMake: C:\Program Files (x86)\Microsoft Visual Studio\2022\BuildTools\Common7\IDE\CommonExtensions\Microsoft\CMake\CMake\bin\cmake.EXE (3.21.21080301) Traceback (most recent call last): (cut) Build type: native build Project name: poppler-meson Project version: 1.0.0 Activating VS 17.0.1 C compiler for the host machine: cl (msvc 19.30.30705 "Microsoft (R) C/C++ Optimizing Compiler Version 19.30.30705 for x64") C linker for the host machine: link link 14.30.30705.0 Host machine cpu family: x86_64 Host machine cpu: x86_64 Found pkg-config: C:\Users\faste\scoop\shims\pkg-config.EXE (0.26) Found CMake: C:\Program Files (x86)\Microsoft Visual Studio\2022\BuildTools\Common7\IDE\CommonExtensions\Microsoft\CMake\CMake\bin\cmake.EXE (3.21.21080301) Run-time dependency glib-2.0 found: NO (tried pkgconfig and cmake) meson.build:6:0: ERROR: Dependency "glib-2.0" not found, tried pkgconfig and cmake A full log can be found at C:\Users\faste\bearcove\poppler-meson\build\meson-logs\meson-log.txt
And that's because we added a dependency on glib-2.0, but didn't give meson
any information on where to find it. So, it tried to find it with pkg-config
and cmake, but it didn't. On Fedora, with glib2-devel installed, this would
probably work.
But we're on Windows now! So meson needs additional info. Luckily, meson now comes with "wraptool", which lets us pick from a list of libraries that have been "wrapped" so they can be built with meson instead of whatever their maintainers thought they should be built with (autotools, cmake, a home-grown organic set of Makefiles, etc.)
$ meson wrap search glib-2.0 Dependency glib-2.0 found in wrap glib
Hey there it is! Let's add it:
$ meson wrap install glib Installed glib version 2.70.1 revision 1
Just like that, wraptool added the subprojects/glib.wrap file:
[wrap-file] directory = glib-2.70.1 source_url = https://download.gnome.org/sources/glib/2.70/glib-2.70.1.tar.xz source_filename = glib-2.70.1.tar.xz source_hash = f9b7bce7f51753a1f43853bbcaca8bf09e15e994268e29cfd7a76f65636263c0 [provide] dependency_names = gthread-2.0, gobject-2.0, gmodule-no-export-2.0, gmodule-export-2.0, gmodule-2.0, glib-2.0, gio-2.0, gio-win32-2.0, gio-unix-2.0 program_names = glib-genmarshal, glib-mkenums, glib-compile-schemas, glib-compile-resources, gio-querymodules, gdbus-codegen
(With hashes and everything!)
Running meson setup again goes further! It actually creates
subprojects/libpcre.wrap, since it's required by glib:
[wrap-redirect] filename = glib-2.70.1\subprojects\libpcre.wrap
But unfortunately, it eventually fails, because the default mirror for pcre sources hates me, apparently:
$ meson setup --wipe build --prefix C:\Users\faste\bearcove\poppler-meson\prefix\ --default-library static (cut) glib| Run-time dependency libpcre found: NO (tried pkgconfig and cmake) glib| Library pcred found: NO glib| Run-time dependency libpcre found: NO (tried pkgconfig and cmake) glib| Looking for a fallback subproject for the dependency libpcre glib| Using subprojects\glib-2.70.1\subprojects\libpcre.wrap glib| Downloading libpcre source from https://ftp.pcre.org/pub/pcre/pcre-8.37.tar.bz2 glib| <urlopen error [WinError 10060] A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond>glib| A fallback URL could be specified using source_fallback_url key in the wrap file subprojects\glib-2.70.1\meson.build:2002:2: ERROR: could not get https://ftp.pcre.org/pub/pcre/pcre-8.37.tar.bz2 is the internet available?
Luckily, there's other mirrors we can pick from! By replacing the contents of
subprojects/libpcre.wrap with a slightly tuned version of
subprojects/glib-2.70.1/subprojects/libpcre.wrap...
[wrap-file] directory = pcre-8.37 source_url = https://ftp.halifax.rwth-aachen.de/osdn/sfnet/p/pc/pcre/pcre/8.37/pcre-8.37.tar.bz2 source_filename = pcre-8.37.tar.bz2 source_hash = 51679ea8006ce31379fb0860e46dd86665d864b5020fc9cd19e71260eef4789d patch_filename = pcre_8.37-2_patch.zip patch_url = https://wrapdb.mesonbuild.com/v2/pcre_8.37-2/get_patch patch_hash = 6b80f72385e1bf06721e26fbc83aced576e9c0d3182d86a55dd173a04050fe26 [provide] libpcre = pcre_dep
...we can keep going.
$ meson setup --wipe build --prefix C:\Users\faste\bearcove\poppler-meson\prefix\ --default-library static
(cut)
Build targets in project: 341
poppler-meson 1.0.0
Subprojects
glib : YES 3 warnings
libffi : YES
libpcre : YES
proxy-libintl : YES
zlib : YES
User defined options
default_library: static
prefix : C:\Users\faste\bearcove\poppler-meson\prefix\
Found ninja-1.10.2 at C:\Users\faste\scoop\shims\ninja.EXE
Visual Studio environment is needed to run Ninja. It is recommended to use Meson wrapper:
C:\Users\faste\AppData\Local\Programs\Python\Python310\Scripts\meson compile -C build
Look at that, it pulled in zlib, libffi and proxy-libintl as well!
It did! How convenient.
A short meson compile -C build and many MSVC warnings later... it fails to
build. Well, it turns out glib static builds on Windows are kind of a
work-in-progress,
so I had to learn about a few more meson features...
Some C defines were missing, but luckily, we can override them at the project level. We can even have the project default to a static build.
As for the glib dependency, we can also set things there, like ask for it to
be statically linked with static: true, and set some default options, like:
"don't bother with tests".
# in poppler-meson/meson.build
project('poppler-meson', 'c',
version: '1.0.0',
meson_version: '>= 0.60.1',
default_options: ['c_args=-DG_INTL_STATIC_COMPILATION=1 -DFFI_STATIC_BUILD=1', 'default_library=static'],
)
glib_dep = dependency('glib-2.0', static: true, default_options: ['tests=false'])
And because glib really insists on including DllMain in its multiple libraries
(gobject, gio, etc.), even when they're built to a static .a library on
Windows, I had to bring in a little patch.
Which is a thing meson lets you do! Usually to "add meson build files", but what's a little crime among friends?
# in `subprojects/glib.wrap` [wrap-file] directory = glib-2.70.1 source_url = https://download.gnome.org/sources/glib/2.70/glib-2.70.1.tar.xz source_filename = glib-2.70.1.tar.xz source_hash = f9b7bce7f51753a1f43853bbcaca8bf09e15e994268e29cfd7a76f65636263c0 # 👇 new! patch_directory = glib-2.70.1 [provide] dependency_names = gthread-2.0, gobject-2.0, gmodule-no-export-2.0, gmodule-export-2.0, gmodule-2.0, glib-2.0, gio-2.0, gio-win32-2.0, gio-unix-2.0 program_names = glib-genmarshal, glib-mkenums, glib-compile-schemas, glib-compile-resources, gio-querymodules, gdbus-codegen
The patch is actually not a patch, more like an overlay, so I had to create
subprojects/packagefiles/glib-2.70.1/glib/glib-init.c with my already-patched
version of this source file. Handy!
And finally, it builds:
$ meson compile -C build Activating VS 17.0.1 ninja: Entering directory `C:/Users/faste/bearcove/poppler-meson/build' (cut) [361/361] Linking target subprojects/glib-2.70.1/fuzzing/fuzz_inet_socket_address_new_from_string.exe
And installs:
$ meson install -C build ninja: Entering directory `C:\Users\faste\bearcove\poppler-meson\build' ninja: no work to do. Installing subprojects\libffi\src\libffi.a to C:\Users\faste\bearcove\poppler-meson\prefix\lib (cut) Installing C:\Users\faste\bearcove\poppler-meson\subprojects\glib-2.70.1\m4macros/gsettings.m4 to C:\Users\faste\bearcove\poppler-meson\prefix\share/aclocal
Oddly enough, meson install wouldn't find ninja the way meson compile did,
and a scoop install ninja fixed it there.
Another option would've been meson devenv -C build, which enters a "dev
environment", with C/C++ compilers/linkers (like cl.exe, link.exe) and build
tools (cmake.exe, ninja.exe) in PATH.
So let's look at what meson/ninja has actually installed in our prefix.
$ dir prefix\lib\pkgconfig
Directory: C:\Users\faste\bearcove\poppler-meson\prefix\lib\pkgconfig
Mode LastWriteTime Length Name
---- ------------- ------ ----
-a--- 01/12/2021 17:34 751 gio-2.0.pc
-a--- 01/12/2021 17:34 311 gio-windows-2.0.pc
-a--- 01/12/2021 17:34 428 glib-2.0.pc
-a--- 01/12/2021 17:34 278 gmodule-2.0.pc
-a--- 01/12/2021 17:34 278 gmodule-export-2.0.pc
-a--- 01/12/2021 17:34 295 gmodule-no-export-2.0.pc
-a--- 01/12/2021 17:34 294 gobject-2.0.pc
-a--- 01/12/2021 17:34 262 gthread-2.0.pc
-a--- 01/12/2021 17:34 248 libffi.pc
-a--- 01/12/2021 17:34 223 zlib.pc
That is great, we definitely needed a bunch of these.
Our static libraries are all installed in prefix\lib\pkgconfig as well:
$ dir prefix\lib
Directory: C:\Users\faste\bearcove\poppler-meson\prefix\lib
Mode LastWriteTime Length Name
---- ------------- ------ ----
-a--- 01/12/2021 18:26 127230 libffi.a
-a--- 01/12/2021 18:26 13584544 libgio-2.0.a
-a--- 01/12/2021 18:26 6860652 libglib-2.0.a
-a--- 01/12/2021 18:26 77578 libgmodule-2.0.a
-a--- 01/12/2021 18:26 1561240 libgobject-2.0.a
-a--- 01/12/2021 18:26 19220 libgthread-2.0.a
-a--- 01/12/2021 18:26 28104 libintl.a
-a--- 01/12/2021 18:26 309480 libz.a
I didn't immediately noticed something was wrong, because it had been a while since I had to think about MSVC idiosyncracies, but...
$ cd glib-test $ cargo run Compiling glib-test v0.1.0 (C:\Users\faste\bearcove\glib-test) error: linking with `link.exe` failed: exit code: 1181 | (cut) = note: LINK : fatal error LNK1181: cannot open input file 'glib-2.0.lib' error: could not compile `glib-test` due to previous error
...that's not how libraries should be named for MSVC. They shouldn't be named
libfoobar.a, they should be named foobar.lib.
Luckily, a single PowerShell command lets us solve that (quickly adds powershell to the tags of this article):
$ Get-ChildItem .\prefix\lib\lib*.a | Rename-Item -NewName { $_.Name -replace "^lib", "" -replace ".a$", ".lib" }
$ Get-ChildItem .\prefix\lib\*.lib
Directory: C:\Users\faste\bearcove\poppler-meson\prefix\lib
Mode LastWriteTime Length Name
---- ------------- ------ ----
-a--- 01/12/2021 18:26 127230 ffi.lib
-a--- 01/12/2021 18:26 13584544 gio-2.0.lib
-a--- 01/12/2021 18:26 6860652 glib-2.0.lib
-a--- 01/12/2021 18:26 77578 gmodule-2.0.lib
-a--- 01/12/2021 18:26 1561240 gobject-2.0.lib
-a--- 01/12/2021 18:26 19220 gthread-2.0.lib
-a--- 01/12/2021 18:26 28104 intl.lib
-a--- 01/12/2021 18:26 309480 z.lib
Tada! Wish zsh would let me do that this easily. Maybe nushell can?
Anyway, now it works:
$ cargo run (cut) LINK : warning LNK4098: defaultlib 'MSVCRTD' conflicts with use of other libs; use /NODEFAULTLIB:library (cut)
Nope, nope, go back, meson defaults to a debug build, we want a release build for sure.
$ meson setup build --prefix C:\Users\faste\bearcove\poppler-meson\prefix\ --buildtype release --wipe $ meson compile -C build $ meson install -C build $ # don't forget to rename libfoobar.a to foobar.lib again...
Ok surely now it'll work:
$ cargo run
(cut)
= note: glib-2.0.lib(gutils.c.obj) : error LNK2019: unresolved external symbol __imp_CoTaskMemFree referenced in function g_build_home_dir
glib-2.0.lib(gutils.c.obj) : error LNK2019: unresolved external symbol SHGetKnownFolderPath referenced in function g_build_home_dir
glib-2.0.lib(gmessages.c.obj) : error LNK2019: unresolved external symbol __imp_MessageBoxW referenced in function g_log_writer_default
glib-2.0.lib(gwin32.c.obj) : error LNK2019: unresolved external symbol __imp_CommandLineToArgvW referenced in function g_win32_get_command_line
glib-2.0.lib(giowin32.c.obj) : error LNK2019: unresolved external symbol __imp_PeekMessageA referenced in function g_io_win32_check
glib-2.0.lib(giowin32.c.obj) : error LNK2019: unresolved external symbol __imp_PostMessageA referenced in function g_io_win32_msg_write
glib-2.0.lib(giowin32.c.obj) : error LNK2019: unresolved external symbol __imp_IsWindow referenced in function g_io_channel_win32_new_messages
glib-2.0.lib(gpoll.c.obj) : error LNK2019: unresolved external symbol __imp_MsgWaitForMultipleObjectsEx referenced in function g_poll
C:\Users\faste\bearcove\glib-test\target\debug\deps\glib_test.exe : fatal error LNK1120: 8 unresolved externals
error: could not compile `glib-test` due to previous error
Err almost. Just missing a couple libraries...
// in `glib-test/build.rs`
fn main() {
system_deps::Config::new().probe().unwrap();
let target = std::env::var("TARGET").unwrap();
if target.contains("msvc") {
println!("cargo:rustc-link-lib=ole32");
println!("cargo:rustc-link-lib=user32");
println!("cargo:rustc-link-lib=shell32");
}
}
$ cargo run
Compiling glib-test v0.1.0 (C:\Users\faste\bearcove\glib-test)
Finished dev [unoptimized + debuginfo] target(s) in 0.78s
Running `target\debug\glib-test.exe`
[src\main.rs:9] &home_dir = "C:\\Users\\faste"
YES! Yes! Finally!
The source for this sample program, by the way, is:
// in `glib-test/src/main.rs`
use std::ffi::CStr;
extern "C" {
fn g_get_home_dir() -> *const i8;
}
fn main() {
let home_dir = unsafe { CStr::from_ptr(g_get_home_dir()) };
dbg!(&home_dir);
}
And the glib-2.0 dependency is specified like so:
[package.metadata.system-deps]
"glib" = { name = "glib-2.0", version = "2.70.1" }
And I include it so you can find out that the system dep's key cannot include dots, but that you can save it by specifying "name".
Okay! So that's glib. I mean, that's glib.
What else do we need... well, cairo's a big one. And good news, cairo,
alongside autotools, actually ships its own meson.build file!
But that still leaves poppler itself. And it doesn't ship with its own meson
build files. It ships with CMake, and in theory, I could try to port
everything to CMake, and if I did, I would use Izzy's wonderful CMake articles,
like Everything you never wanted to know about
CMake,
and How to Find Packages With
CMake.
This time, I chose to go the opposite route: everything except poppler already
had meson build files, so it seemed like a good opportunity for me to simply
write meson.build files for poppler! Which is exactly what I did.
This article is part 4 of the Don't shell out! series.
If you liked what you saw, please support my work!
