% Custom Allocators
Allocating memory isn't always the easiest thing to do, and while Rust generally takes care of this by default it often becomes necessary to customize how allocation occurs. The compiler and standard library currently allow switching out the default global allocator in use at compile time. The design is currently spelled out in RFC 1183 but this will walk you through how to get your own allocator up and running.
Default Allocator
The compiler currently ships two default allocators: alloc_system and
alloc_jemalloc (some targets don't have jemalloc, however). These allocators
are normal Rust crates and contain an implementation of the routines to
allocate and deallocate memory. The standard library is not compiled assuming
either one, and the compiler will decide which allocator is in use at
compile-time depending on the type of output artifact being produced.
Binaries generated by the compiler will use alloc_jemalloc by default (where
available). In this situation the compiler "controls the world" in the sense of
it has power over the final link. Primarily this means that the allocator
decision can be left up the compiler.
Dynamic and static libraries, however, will use alloc_system by default. Here
Rust is typically a 'guest' in another application or another world where it
cannot authoritatively decide what allocator is in use. As a result it resorts
back to the standard APIs (e.g. malloc and free) for acquiring and releasing
memory.
Switching Allocators
Although the compiler's default choices may work most of the time, it's often necessary to tweak certain aspects. Overriding the compiler's decision about which allocator is in use is done simply by linking to the desired allocator:
#![feature(alloc_system)]
extern crate alloc_system;
fn main() {
let a = Box::new(4); // allocates from the system allocator
println!("{}", a);
}
In this example the binary generated will not link to jemalloc by default but instead use the system allocator. Conversely to generate a dynamic library which uses jemalloc by default one would write:
#![feature(alloc_jemalloc)]
#![crate_type = "dylib"]
extern crate alloc_jemalloc;
pub fn foo() {
let a = Box::new(4); // allocates from jemalloc
println!("{}", a);
}
# fn main() {}
Writing a custom allocator
Sometimes even the choices of jemalloc vs the system allocator aren't enough and
an entirely new custom allocator is required. In this you'll write your own
crate which implements the allocator API (e.g. the same as alloc_system or
alloc_jemalloc). As an example, let's take a look at a simplified and
annotated version of alloc_system
# // only needed for rustdoc --test down below
# #![feature(lang_items)]
// The compiler needs to be instructed that this crate is an allocator in order
// to realize that when this is linked in another allocator like jemalloc should
// not be linked in
#![feature(allocator)]
#![allocator]
// Allocators are not allowed to depend on the standard library which in turn
// requires an allocator in order to avoid circular dependencies. This crate,
// however, can use all of libcore.
#![no_std]
// Let's give a unique name to our custom allocator
#![crate_name = "my_allocator"]
#![crate_type = "rlib"]
// Our system allocator will use the in-tree libc crate for FFI bindings. Note
// that currently the external (crates.io) libc cannot be used because it links
// to the standard library (e.g. `#![no_std]` isn't stable yet), so that's why
// this specifically requires the in-tree version.
#![feature(libc)]
extern crate libc;
// Listed below are the five allocation functions currently required by custom
// allocators. Their signatures and symbol names are not currently typechecked
// by the compiler, but this is a future extension and are required to match
// what is found below.
//
// Note that the standard `malloc` and `realloc` functions do not provide a way
// to communicate alignment so this implementation would need to be improved
// with respect to alignment in that aspect.
#[no_mangle]
pub extern fn __rust_allocate(size: usize, _align: usize) -> *mut u8 {
unsafe { libc::malloc(size as libc::size_t) as *mut u8 }
}
#[no_mangle]
pub extern fn __rust_deallocate(ptr: *mut u8, _old_size: usize, _align: usize) {
unsafe { libc::free(ptr as *mut libc::c_void) }
}
#[no_mangle]
pub extern fn __rust_reallocate(ptr: *mut u8, _old_size: usize, size: usize,
_align: usize) -> *mut u8 {
unsafe {
libc::realloc(ptr as *mut libc::c_void, size as libc::size_t) as *mut u8
}
}
#[no_mangle]
pub extern fn __rust_reallocate_inplace(_ptr: *mut u8, old_size: usize,
_size: usize, _align: usize) -> usize {
old_size // this api is not supported by libc
}
#[no_mangle]
pub extern fn __rust_usable_size(size: usize, _align: usize) -> usize {
size
}
# // only needed to get rustdoc to test this
# fn main() {}
# #[lang = "panic_fmt"] fn panic_fmt() {}
# #[lang = "eh_personality"] fn eh_personality() {}
# #[lang = "eh_unwind_resume"] extern fn eh_unwind_resume() {}
# #[no_mangle] pub extern fn rust_eh_register_frames () {}
# #[no_mangle] pub extern fn rust_eh_unregister_frames () {}
After we compile this crate, it can be used as follows:
extern crate my_allocator;
fn main() {
let a = Box::new(8); // allocates memory via our custom allocator crate
println!("{}", a);
}
Custom allocator limitations
There are a few restrictions when working with custom allocators which may cause compiler errors:
-
Any one artifact may only be linked to at most one allocator. Binaries, dylibs, and staticlibs must link to exactly one allocator, and if none have been explicitly chosen the compiler will choose one. On the other hand rlibs do not need to link to an allocator (but still can).
-
A consumer of an allocator is tagged with
# and an#[allocator]crate cannot transitively depend on a crate which needs an allocator (e.g. circular dependencies are not allowed). This basically means that allocators must restrict themselves to libcore currently.
commit 6ba9520