In our last installment of
"Making our own executable packer", we did some code cleanups. We got rid of
a bunch of unsafe
code, and found a way to represent memory-mapped data
structures safely.
But that article was merely a break in our otherwise colorful saga of "trying
to get as many executables to run with our own dynamic loader". The last thing
we got running was the ifunc-nolibc
program.
Shell session
$ ./target/debug/elk run ./samples/ifunc-nolibc
Loading "/home/amos/ftl/elf-series/samples/ifunc-nolibc"
Hello, regular user!
$ sudo ./target/debug/elk run ./samples/ifunc-nolibc
Loading "/home/amos/ftl/elf-series/samples/ifunc-nolibc"
Hello, root!
It was an interesting
article, because even though
we discovered elk
, our dynamic loader, isn't quite up to the task of loading
libc, it can load C programs that are compiled with -nostartfiles -nodefaultlibs
.
In other words, we're able to use gcc as a fancy assembler - which is great,
because I'm much more comfortable writing cursed C code than I am writing
nasm. More importantly, most of the programs we're going to try and run are
also written in C, so it's easier for me to figure out which C constructs
correspond to various parts of readelf -a
's output.
In the current state of things, if we try to run a dynamically-linked C program,
elk
errors out pretty early:
Shell session
$ ./target/debug/elk run /usr/bin/ls
Loading "/usr/bin/ls"
Loading "/usr/lib/libcap.so.2.45"
Loading "/usr/lib/libc-2.32.so"
Fatal error: Could not read symbols from ELF object: Parsing error: String("Unknown SymType 6 (0x6)"):
input: 16 00 19 00 10 00 00 00 00 00 00 00 04 00 00 00 00 00 00 00
But that's just the tip of the iceberg. There are many more things missing
from elk
for it to be able to load and run a program that links against
glibc.
So, instead of repeatedly banging our heads against a wall repeatedly trying
to run real-world executables and seeing what's missing, let's try to be proactive
and build a sample program, and add features to delf
and elk
as we progress.
That way, we can be sure that we understand what is supposed to happen before we
figure out a way to actually make it happen.
Our sample program this time will be a little more involved than before, so let's
give it a whole directory:
Shell session
$ cd samples/
$ mkdir chimera
$ cd chimera
Let's start simple:
C code
// in `elk/samples/chimera/chimera.c`
void ftl_exit(int code) {
__asm__ (
" \
mov %[code], %%edi \n\
mov $60, %%rax \n\
syscall"
:
: [code] "r" (code)
);
}
void _start(void) {
ftl_exit(21);
}
In case you need a refresher: the abomination code above is GCC inline assembly, which
we need to make an exit
syscall, because we're not linking against libc.
We discussed GCC inline assembly to some degree in part 9.
We're going to be building multiple .c files, so let's whip up a quick Makefile:
makefile
# in `elk/samples/chimera/Makefile`
CFLAGS := -fPIC
LDFLAGS := -nostartfiles -nodefaultlibs -L. -Wl,-rpath='$$ORIGIN'
all: chimera
chimera: chimera.c
gcc -c chimera.c ${CFLAGS}
gcc chimera.o -o chimera ${LDFLAGS}
clean:
rm -f chimera *.o *.so
We're only using GNU make as a history lesson. Back when C was considered a
reasonable language to write anything in, make was also considered a
reasonable build system.
Luckily, since then, much better tools have
appeared, and GNU make is barely used anymore. It was weaponized for a
while
but that proved unwieldy and the whole arrangement quickly died down.
So, we're indulging here, not because it's a good tool for the job, but
because we're trying to understand what life was like back then — long, long
ago — so it's only fair that we try and use era-appropriate software.
Let's walk through it piece by piece:
Here we're defining a "simply expanded" GNU make variable (not an environment
variable) containing flags used for compilation. -fPIC
tells gcc to
generate position-independent code, which is needed because we compile and
link separately, so it can't guess in advance whether the code is going to end
up in a position-independent object or not.
makefile
LDFLAGS := -nostartfiles -nodefaultlibs -L. -Wl,-rpath='$$ORIGIN'
Same here, but for linker flags. The first two flags are familiar, -L
adds
.
(the current directory) to the library search path, so if we end up linking
against libfoo.so
or libbar.so
it'll know where to find them, and finally,
-Wl,XXX
is a way to pass arguments to the actual linker, GNU ld
, and we've
learned what -rpath
does in Part 5.
Note: the $
(dollar sign) is doubled so GNU make doesn't think we're accessing
a GNU make variable. The whole thing is single-quoted so bash doesn't think we
want to expand a bash variable.
That's right. It's a double freaking escape. That's what people used to have to
deal with, back in the days. Good thing that's over.
This is our first "target", and since it's the first, it's the one that's going to
get run when we invoke GNU make simply as make
. It depends on chimera
, which
is the name of the executable we want to build, which means that, if a file named
chimera
doesn't exist, it'll run this target:
makefile
chimera: chimera.c
gcc -c chimera.c ${CFLAGS}
gcc chimera.o -o chimera ${LDFLAGS}
It's very important for commands inside targets to be indented with a single tab.
Not two spaces, not four spaces. One tab.
If you get it wrong, GNU make will hit you with this wonderful error message until
you comply or give up:
Shell session
Makefile:9: *** missing separator. Stop.
This target will also be run if chimera.c
is newer than chimera
(or whatever
GNU make uses to determine out-of-dateness these days).
As for the commands, well, -c
will run the C assembler and generate an ELF object file,
chimera.o
, and the second invocation drives the GNU linker to make a real live,
position-independent executable.
makefile
clean:
rm -f chimera *.o *.so
Finally, this target removes every object / executable / library file whenever we invoke
make clean
.
Archaic tooling hype! Let's build it.
Shell session
$ make
gcc -c chimera.c -fPIC
gcc chimera.o -o chimera -nostartfiles -nodefaultlibs -L. -Wl,-rpath='$ORIGIN'
$
Woo! Did that work?
Shell session
$ ./chimera; echo $?
21
Wonderful.
Can we run it through elk?
Shell session
$ ../../target/debug/elk run ./chimera; echo $?
Loading "/home/amos/ftl/elk/samples/chimera/chimera"
Found RPATH entry "/home/amos/ftl/elk/samples/chimera"
21
Sure we can! Why couldn't we? There's nothing particularly interesting about
the chimera
executable. Nothing we don't support, at least.
So let's bring a dynamic library into the mix - we'll call it libfoo
.
C code
// in `elk/samples/chimera/foo.c`
int number = 21;
Now remember, this is C, where the default visibility for symbols is
"weeeeeeee", so we can definitely use it from chimera.c
C code
// in `elk/samples/chimera/chimera.c`
// omitted: `ftl_exit`
extern int number;
void _start() {
ftl_exit(number);
}
Let's change up our Makefile
so it builds and links against libfoo:
makefile
# in `elk/samples/chimera/Makefile`
# omitted: everything else
# 👇 now depends on `libfoo.so` target
chimera: chimera.c libfoo.so
gcc -c chimera.c ${CFLAGS}
# 👇 new!
gcc chimera.o -o chimera -lfoo ${LDFLAGS}
# 👇 new target
libfoo.so: foo.c
gcc -c foo.c ${CFLAGS}
gcc foo.o -shared -o libfoo.so ${LDFLAGS}
Shell session
$ make
gcc -c foo.c -fPIC
gcc foo.o -shared -o libfoo.so -nostartfiles -nodefaultlibs -L. -Wl,-rpath='$ORIGIN'
gcc -c chimera.c -fPIC
gcc chimera.o -o chimera -lfoo -nostartfiles -nodefaultlibs -L. -Wl,-rpath='$ORIGIN'
...and voilà !
Shell session
$ ./chimera; echo $?
21
But will it blend^W run through elk?
Shell session
$ ../../target/debug/elk run ./chimera; echo $?
Loading "/home/amos/ftl/elk/samples/chimera/chimera"
Found RPATH entry "/home/amos/ftl/elk/samples/chimera"
Loading "/home/amos/ftl/elk/samples/chimera/libfoo.so"
Found RPATH entry "/home/amos/ftl/elk/samples/chimera"
Fatal error: unimplemented relocation: GlobDat
0
No! It doesn't. Looks like time has caught up with us, and we need
to implement more relocations. Very well then!
First off, what's the relocation to?
Shell session
$ readelf -r ./chimera
Relocation section '.rela.dyn' at offset 0x358 contains 1 entry:
Offset Info Type Sym. Value Sym. Name + Addend
000000003ff8 000100000006 R_X86_64_GLOB_DAT 0000000000000000 number + 0
Right! The only symbol libfoo.so
exports. No surprise there.
Let's look at the "Relocation Types" table again, from the System V ABI:
Name | Value | Field | Calculation |
None | 0 | none | none |
64 | 1 | word64 | S + A |
PC32 | 2 | word32 | S + A - P |
GOT32 | 3 | word32 | G + A |
PLT32 | 4 | word32 | L + A - P |
COPY | 5 | none | none |
GLOB_DAT | 6 | wordclass | S |
JUMP_SLOT | 7 | wordclass | S |
RELATIVE | 8 | wordclass | B + A |
Now, I'm ready to bet GLOB_DAT
stands for "global data". No big mystery there -
number
lives in the address space for libfoo.so
, so any references to it
must be relocated, because we don't know where it'll be loaded in advance.
But wait.. we already had a sample program that uses a variable from another library,
the hello-dl
program from Dynamic symbol resolution.
Back then it used a Copy
relocation, not a GlobDat
one.
What determines which type of relocation we get: Copy
or GlobDat
?
Turns out, in this case, it depends on whether we pass -fPIC
to GCC.
Let's compare what we get "without -fPIC" and "with -fPIC" step by step.