Distributing software is a tricky thing. If you want to share the functionality with someone you may share the source with them and give them the build instructions and it would all play out good. But that may not be the case if the source is a different language and the consumer is using a different language. For example, there are a lot of libraries that use common functionalities like openssl for crypto operations or network libraries or packages like zlib just because they are reliable and have been tested for years on speed and correctness. You might not want to translate it in the language of your choice.

How do you accomplish something like that. Calling a library written in some other language that your source. There is a concept called FFI or Foreign Function Interface which is used for this exact thing.

You might have heard about JNI or Java Native Interfaces which is used to call subroutines in such native libraries. This came to me as a surprise, a lot of the core components of Java is written in C and is called within the language using JNI bindings. You would have seen the following signature in some Java classes when traversing the Go-To definitions.

public native String foo()

For example, This is what FileInputStream.open looks like

JNIEXPORT void JNICALL
Java_java_io_FileInputStream_open0(JNIEnv *env, jobject this, jstring path) {
    fileOpen(env, this, path, fis_fd, O_RDONLY);
}

That’s the calling convention for JNI, the java.io.FileInputStream.open0() would be translated to Java_java_io_FileInputStream_open0().

Java is hardwired to look for these symbols when someone calls the Java name of these functions and most of it is present in libjava.so and libjvm.so files. These are distributable Shared Objects that contains the implementation of these native functions and is platform dependant. This may come as a surprise to some people (or I may be making a fool out of myself), Java isn’t platform independant, atleast not all the core components. The bytecode generated by Java compiler is platform independant. The VM on which the bytecode executes has to be written and compiled for all platforms and architectures.

That looks something like this jdk/src/java.base/ .

Shared objects

Shared objects or dynamic libraries are an interesting concept and sometimes a pain. Let’s go through an example.

#include <stdio.h>
int main() {
    printf("hello world");
    return 0;
}

This program is just calling a function called printf. But, wait, I can call the functions that actually exists, otherwise the compiler goes batshit crazy, unless it’s javascript ofcourse, which will break at runtime. This code compiles, so this function must exist, but where, it’s not present in my source file.

Its present in the /usr/include/stdio.h file.

extern int printf (const char *__restrict __format, ...);

Wait, it’s only the declaration, where is the implementation. When I build this this code and run the linked dependency tools ldd, it shows this,

$ gcc hello.c -o hello
$ ldd hello
        linux-vdso.so.1 (0x00007fff715ea000)
        libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f1f085a1000)
        /lib64/ld-linux-x86-64.so.2 (0x00007f1f08b94000)

Notice, there is something called, libc present in here. Linux exports all these implementations in this shared object. For example, notice this file in the glibc repository, glibc/printf.c. If libc is not present on the target host, the simple hello world program won’t work since it would not find the printf subroutine.

Now, if the code is in a binary file, how do people and compilers know if they using it correctly. That’s where the header files come in. Header files are basically signatures that the compiler can rely on to check if the code is correct syntactically. This is generally exported as a public include folder in C projects.

There is a very nice explanation on what shared objects are in this SO post. TL;DR, it’s a binary which contains implementation of the corresponding headers. Shared object naming convention is specific to link. For other platforms, they are called dylib for MacOS and dll for windows. They are not quite the same but, they behave in a similar fashion and this can be backed up by the C API dlopen which is used to load dynamic libraries.

Now, that we have vague idea of what shared objects represent, let’s move to “why this waste of internet resources, this article”.

Interoperability with Rust

Rust is popular because it boasts a good interoperability with the C APIs. Which means, it’s simple in Rust to call C APIs with minimal efforts. Let’s see a minimal API.

The C code we would call.

We need to have some functionality that we want to call from other languages. Let’s write a toy project which exposes such API.

The project structure looks like this.

.
├── Makefile
└── src
   ├── include
   │  └── shared.h
   └── shared
      └── shared.c

This is a fairly standard structure for C projects. To have definitions for all public APIs in the include folder.

The shared.h file.

#ifndef SHARED_H
#define SHARED_H
#endif

struct key_spec {
    char key[16];
    const char *type;
};

struct key_spec* get_key();

The shared.c file that contains the actual implementation. This is a very advanced key generator ! almost cryptographically secure.

#include <shared.h>
#include <stdlib.h>

struct key_spec *get_key() {
    struct key_spec *ks = (struct key_spec*) malloc(sizeof(struct key_spec*));
    for (int i = 0; i < 16; i++) {
        ks->key[i] = i + 32;
    }

    ks->type = (const char*)"dummy\0";
    return ks;
}

How to build it into a shared object (notice it does not have a int main(). Create a Makefile with

build:
        gcc -shared -Isrc/include src/shared/shared.c -o libshared.so

And that’s it. Just run make and it should dump a shared object file.

$ make
$ file libshared.so
libshared.so: ELF 64-bit LSB shared object, x86-64, version 1 (SYSV),
dynamically linked, BuildID[sha1]=65587972f8df8f099b66363f0cc44f96f43c2828, not stripped

Interfacing rust with the shared object

Now that we have a shared object which has out advanced key generator, we need to tell Rust, how does it look, function definitions, fields etc. There is just the tool for it, bindgen. It’s a rust language project that generates FFI bindings (the interface in the target language, i.e. Rust for it’s compiler to understand. It’s basically a header file but in rust.)

Install the bindgen crate via cargo install bindgen. Make sure you have some form of a C compiler present, I am using gcc here.

Generating the header-ish files for rust

Let’s ask rust to generate some code.

$ cargo init --bin test-rs
$ cd test-rs
$ bindgen cproject/src/include/shared.hpp -o src/shared.rs

This should generate a rust source file.

/* automatically generated by rust-bindgen 0.55.1 */

#[repr(C)]
#[derive(Debug, Copy, Clone)]
pub struct key_spec {
    pub key: [::std::os::raw::c_char; 16usize],
    pub type_: *const ::std::os::raw::c_char,
}

/* omitting tests */

extern "C" {
    pub fn get_key() -> *mut key_spec;
}

Notice, it has a similar structure of our header file and defintions. Alright, it looks nice and usable in rust. Let’s move on.

Let’s write the main.rs file.

mod shared;

fn main() {
    unsafe {
        let k = shared::get_key();
        println!("the key: {:#?}", k.as_ref());
    };
}

Since, this is a call from a different library, rust can not guarantee it would not do something funny, so we have to write it inside the unsafe block.

That’s all code we need.

Building it in rust

Before we rush and do a cargo run, we need to tell the compiler what are we trying to do.

If you try to build it now, the linker will throw a huge error saying, it can’t find

$ cargo build
error: linking with `cc` failed: exit code: 1
...
error: linking with `cc` failed: exit code: 1
  |
  = note: "cc" "-Wl,--as-needed"...
  = note: test-rs/target/debug/deps/test_rs-b3e83acc1bd66527.3yrtf1vyhbvxamca.rcgu.o: In function `test_rs::main':
          test-rs/src/main.rs:5: undefined reference to `get_key'
          collect2: error: ld returned 1 exit status

which is logical as we just told rust about the definition, we never told rust where to look for the actual implementation is, notice it’s a linker ld error saying it can’t find get_key function.

There are 2 ways to do it.

Using linker flags, link and search paths

We can do it the old school way using the -L and the -l options in ld.

The rust compiler, just like the gcc can take some linker flags and pass it on to the linker in the linking step.

$ env RUSTFLAGS="-Lcproject/ -lshared" cargo build

This will tell ld to look for libshared.so file in the search path cproject (since we build the shared object in that project folder).

The all rust way

We can tell rust by specifying a links key in package section in Cargo.toml.

This naming convention is same as dlopen, omit the leading lib from the shared object name. So, libshared.so becomes shared.

After adding this, cargo build will ask you to have a custom build script, i.e. build.rs. This very specific use case is present in the bindgen manual

use std::env;

fn main() {
    let project_dir = env::var("CARGO_MANIFEST_DIR").unwrap();

    println!("cargo:rustc-link-search={}", project_dir); // the "-L" flag
    println!("cargo:rustc-link-lib=shared"); // the "-l" flag
}

We tell cargo to use this build.rs file by specifying the build key in the package section in Cargo.toml.

[package]
name = test-rs
...
links = "shared"
build = "build.rs"

Now, when we run, we should get a successful build. Run a cargo clean to make sure old artefacts are removed.

$ cargo clean && cargo build

Let’s celebrate our victory

We are ready to fly..

$ cargo build
$ ./target/debug/test-rs
./target/debug/test-rs: error while loading shared libraries: libshared.so:
cannot open shared object file: No such file or directory

But wait, this is trivial, remember LD_LIBRARY_PATH. All binaries that have dynamic dependencies should be told where to find those dependencies. eg:

$ ldd $HOME/.cargo/bin/cargo
        linux-vdso.so.1 (0x00007ffe401f4000)
        libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2 (0x00007f6ed709c000)
        librt.so.1 => /lib/x86_64-linux-gnu/librt.so.1 (0x00007f6ed6e94000)
        libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007f6ed6c75000)
        libgcc_s.so.1 => /lib/x86_64-linux-gnu/libgcc_s.so.1 (0x00007f6ed6a5d000)
        libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f6ed666c000)
        libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007f6ed62ce000)
        /lib64/ld-linux-x86-64.so.2 (0x00007f6ed7e2c000)

Notice, how all the shared objects are mapped to a physical location in the memory which is usually part of /lib and /lib64. For an exhustive list look at /etc/ld.so.conf.d/.

We can set the environment variable LD_LIBRARY_PATH and then it should run.

$ env LD_LIBRARY_PATH=cproject target/debug/test-rs
the key: Some(
    key_spec {
        key: [
            32,
            33,
            ...
            45,
            46,
            47,
        ],
        type_: 0x00007fbafa2f2665,
    },
)

Rejoice!!

Something interesting that I observed

The original intent of this excercise was to get Rust to use C++ library. So, the initial version of the C source was in C++. Then, I decided to move to something simpler. But I forgot to rename the header file to a .h extension from a .hpp extension. Due to that, bindgen was compiling those headers into mangled link_names causing a lot of pain to me.

When generating the ffi bindings for the headers named as .hpp instead of .h, I was getting

extern "C" {
    #[link_name = "\u{1}_Z7get_keyv"]
    pub fn get_key() -> *mut key_spec;
}

Notice the link_name, an additional attribute.The cargo builds kept failing due to the following errors when I was trying to use the above generated code with a C version of the shared object build, i.e. using gcc.

"cc" "-Wl,--as-needed" ... "-L" "cproject" ... "-l" "shared" ...

It seems to be looking at the correct locations, but it’s not able to find the mangled name.

Notice the link_name, it’s not get_key but _Z7get_keyv. This is called Name Mangling, compiler’s way of embedding meta data for the linker. Let’s see what name is present in the gcc version of the shared object that the linker is trying to look in.

$ objdump -d libshared.so | grep get_key
000000000000060a <get_key>:
 627:   eb 18                   jmp    641 <get_key+0x37>
 645:   7e e2                   jle    629 <get_key+0x1f>

This seems to be plain and simple, no mangling at all, as expected.

We can explicitly tell bindgen to not use those compiler provided mangled link names. by specifying

bindgen --distrust-clang-mangling <header file> -o src/shared.rs

Observing bindgen with C++

Mangling is a very prominent feature in C++. Which means, g++ should be able to produce the same results as bindgen when asserting the project to be a c++ project by the .hpp extension.

Let’s compile the source shared object with g++ instead of gcc.

$ g++ -shared -Isrc/include src/shared/shared.c -o libshared.so
$ file libshared.so
libshared.so: ELF 64-bit LSB shared object, x86-64, version 1 (SYSV),
dynamically linked, BuildID[sha1]=002b495798ab8683e9596c1e2a85104dc5e48fb6, not stripped

$ objdump -d libshared.so | grep get_key
000000000000061a <_Z7get_keyv>:
 63b:   7f 1a                   jg     657 <_Z7get_keyv+0x3d>
 655:   eb e0                   jmp    637 <_Z7get_keyv+0x1d>

So, the shared object built with g++ does produce the expected mangled name.

$ c++filt _Z7get_keyv
get_key()

And is able to reverse properly too.

Thanks to /u/boomshroom for pointing this .h to .hpp error out.

References

Some interesting things that I came across that may help

• Linux Executables: From Assembly to C and Rust
• Before Main: How Executables Work on Linux
• Rust bindgen tutorial

Discussion thread: here