Buck2 is a new open source build system developed by Meta (Facebook) which we already looked at before in some depth, see A Tour Around Buck2, Meta’s New Build System. Since then, Buck2 has gained significant improvements in user experience and language support, making it an increasingly attractive option in the build systems space.
At Tweag, we adhere to high standards for reproducible builds, which Buck2 doesn’t fully uphold in its vanilla configuration. In this post, we will introduce our ruleset that provides integration with Nix. I’ll demonstrate how it can be used, and you will gain insights into how to leverage Nix to achieve more reliable and reproducible builds with Buck2.
Reproducibility, anyone?
In short, Buck2 is a fast, polyglot build tool very similar to Bazel. Notably, it also provides fine-grained distributed caching and even speaks (in its open source variant) the same remote caching and execution protocols used by Bazel. This means you’re able to utilize the same Bazel services available for caching and remote execution.
However, in contrast to Bazel, Buck2 uses a remote first approach and does not restrict build actions using a sandbox on the local machine. As a result build actions can be non-hermetic, meaning their outcome might depend on what files or programs happen to be present on the local machine. This lack of hermeticity can lead to non-reproducible builds, which is a critical concern for the effective caching of build artifacts.
Non-hermeticity issues can be elusive, often surfacing unexpectedly for new developers which effects on-boarding new team members, or open source contributors. If left undetected, they can even cause problems down the line in production, which is why we think reproducible builds are important!
Achieving Reproducibility with Nix
If we want reproducible builds, we must not rely on anything installed on the local machine. We need to precisely control every compiler and build tool which is used in our project. Although defining each and every one of these inside the Buck2 build itself is possible, it also would be a lot of work. The solution to this problem can be Nix.
Nix is a package manager and build system for Linux and Unix-like operating systems. With nixpkgs
, there is a very large and comprehensive collection of software packaged using Nix, which is extensible and can be adapted to one’s needs. Most importantly, Nix already strictly enforces hermeticity for its package builds and the nixpkgs
collection goes to great lengths to achieve reproducible builds.
So, using Nix to provide compilers and build tools for Buck2 is a way to benefit from that preexisting work and introduce hermetic toolchains into a Buck2 build.
Let’s first quickly look into the Nix setup and proceed with how we can integrate it into Buck2 later.
Nix with flakes
After installing Nix, the nix
command is available, and we can start declaring dependencies on packages from nixpkgs
in a nix
file. The Nix tool uses the Nix language, a domain-specific, purely functional and lazily evaluated programming language to define packages and declare dependencies. The language has some wrinkles, but don’t worry; we’ll only use basic expressions without delving into the more advanced concepts.
For example, here is a simple flake.nix
which provides the Rust compiler as a package output:
{
inputs = {
nixpkgs.url = "github:nixos/nixpkgs?ref=nixos-unstable";
};
outputs = { self, nixpkgs }:
{
packages = {
aarch64-darwin.rustc = nixpkgs.legacyPackages.aarch64-darwin.rustc;
x86_64-linux.rustc = nixpkgs.legacyPackages.x86_64-linux.rustc;
}
};
}
Note: While flakes have been widely used for a long time, the feature still needs to be enabled explicitly by setting extra-experimental-features = nix-command flakes
in the configuration. See the wiki for more information.
In essence, a Nix flake is a Nix expression following a specific schema. It defines its inputs (usually other flakes) and outputs (e.g. packages) which depend on the inputs. In this example the rustc
package from nixpkgs
is re-used for the output of this flake, but more complex expressions could be used just as well.
Inspecting this flake shows the following output:
$ nix flake show --all-systems
path:/source/project?lastModified=1745857313&narHash=sha256-e1sxfj1DZbRjhHWF7xfiI3wc1BpyqWQ3nLvXBKDya%2Bg%3D
└───packages
├───aarch64-darwin
│ └───rustc: package 'rustc-wrapper-1.86.0'
└───x86_64-linux
└───rustc: package 'rustc-wrapper-1.86.0'
In order to build the rustc
package output, we can call Nix in the directory of the flake.nix
file like this: nix build '.#rustc'
. This will either fetch pre-built artifacts of this package from a binary cache if available, or directly build the package if not. The result is the same in both cases: the rustc
package output will be available in the local nix store, and from there it can be used just like other software on the system.
$ nix build --print-out-paths '.#rustc'
/nix/store/ssid482a107q5vw18l9millwnpp4rgxb-rustc-wrapper-1.86.0-man
/nix/store/szc39h0qqfs4fvvln0c59pz99q90zzdn-rustc-wrapper-1.86.0
The output displayed above illustrates that a Nix build of a single package can produce multiple outputs. In this case the rustc
package was split into a default output and an additional, separate output for the man pages.
The default output contains the main binaries such as the Rust compiler:
$ /nix/store/szc39h0qqfs4fvvln0c59pz99q90zzdn-rustc-wrapper-1.86.0/bin/rustc --version
rustc 1.86.0 (05f9846f8 2025-03-31) (built from a source tarball)
It is also important to note that the output of a Nix package depends on the specific nixpkgs
revision stored in the flake.lock
file, rather than any changes in the local environment. This ensures that each developer checking out the project at any point in time will receive the exact same (reproducible) output no matter what.
Using Buck2
As part of our work for Mercury, a company providing financial services, we developed rules for Buck2 which can be used to integrate packages provided by a nix flake as part of a project’s build. Recently, we have been able to publish these rules, called buck2.nix
, as open source under the Apache 2 license.
To use these rules, you need to make them available in your project first. Add the following configuration to your .buckconfig
:
[cells]
nix = none
[external_cells]
nix = git
[external_cell_nix]
git_origin = https://github.com/tweag/buck2.nix.git
commit_hash = accae8c8924b3b51788d0fbd6ac90049cdf4f45a # change to use a different version
This configures a cell called nix
to be fetched from the specified repository on GitHub. Once set up, you can refer to that cell in your BUCK
files and load rules from it.
Note: for clarity, I am going to indicate the file name in the top most comment of a code block when it is not obvious from the context already
To utilize a Nix package from Buck2, we need to introduce a new target that runs nix build
inside of a build action producing a symbolic link to the nix store path as the build output. Here is how to do that using buck2.nix:
# BUCK
load("@nix//flake.bzl", "flake")
flake.package(
name = "rustc",
binary = "rustc",
path = "nix", # path to a nix flake
package = "rustc", # which package to build, default is the value of the `name` attribute
output = "out", # which output to build, this is the default
)
Note: this assumes the flake.nix
and accompanying flake.lock
file is found alongside the BUCK
file in the nix
subdirectory
With this build file in place, a new target called rustc
is made available which builds the output called out
of the rustc
package of the given flake. This target can be used as a dependency of other rules in order to generate an output artifact:
# BUCK
genrule(
name = "rust-info",
out = "rust-info.txt",
cmd = "$(exe :rustc) --version > ${OUT}"
)
Note: Buck2 supports expanding references in string parameters using macros, such as the $(exe )
part in the cmd
parameter above which expands to the path of the executable output of the :rustc
target
Using Buck2 (from nixpkgs
of course!) to build the rust-info
target yields:
$ nix run nixpkgs#buck2 -- build --show-simple-output :rust-info
Build ID: f3fec86b-b79f-4d8e-80c7-acea297d4a64
Loading targets. Remaining 0/10 24 dirs read, 97 targets declared
Analyzing targets. Remaining 0/20 5 actions, 5 artifacts declared
Executing actions. Remaining 0/5 9.6s exec time total
Command: build. Finished 2 local
Time elapsed: 10.5s
BUILD SUCCEEDED
buck-out/v2/gen/root/904931f735703749/__rust-info__/out/rust-info.txt
$ cat buck-out/v2/gen/root/904931f735703749/__rust-info__/out/rust-info.txt
rustc 1.86.0 (05f9846f8 2025-03-31) (built from a source tarball)
For this one-off command we just ran buck2
from the nixpkgs
flake on the current system. This is nice for illustration, but it is also not reproducible, and you’ll probably end up with a different Buck2 version when you try this on your machine.
In order to provide the same Buck2 version consistently, let’s add another Nix flake to our project:
# flake.nix
{
inputs = {
nixpkgs.url = "github:nixos/nixpkgs?ref=nixos-unstable";
};
outputs = { self, nixpkgs }:
{
devShells.aarch64-darwin.default =
nixpkgs.legacyPackages.aarch64-darwin.mkShellNoCC {
name = "buck2-shell";
packages = [ nixpkgs.legacyPackages.aarch64-darwin.buck2 ];
};
devShells.x86_64-linux.default =
nixpkgs.legacyPackages.x86_64-linux.mkShellNoCC {
name = "buck2-shell";
packages = [ nixpkgs.legacyPackages.x86_64-linux.buck2 ];
};
};
nixConfig.bash-prompt = "(nix) \\$ "; # visual clue if inside the shell
}
This flake defines a default development environment, or dev shell for short. It uses the mkShellNoCC
function from nixpkgs
which creates an environment where the programs from the given packages
are available in PATH
.
After entering the shell by running nix develop
in the directory of the flake.nix
file, the buck2
command has the exact same version for everyone working on the project as long as the committed flake.lock
file is not changed. For convenience, consider using direnv which automates entering the dev shell as soon as changing into the project directory.
Hello Rust
With all of that in place, let’s have a look at how to build something more interesting, like a Rust project.
Similar to the genrule
above, it would be possible to define custom rules utilizing the :rustc
target to compile real-world Rust projects. However, Buck2 already ships with rules for various languages in its prelude, including rules to build Rust libraries and binaries.
In a default project setup with Rust these rules would simply use whatever Rust compiler is installed in the system, which may cause build failures due to version mismatches.
To avoid this non-hermeticity, we’re going to instruct the Buck2 rules to use our pinned Rust version from nixpkgs
.
Let’s start by preparing such a default setup for the infamous “hello world” example in Rust:
# src/hello.rs
fn main() {
println!("Hello, world!");
}
# src/BUCK
rust_binary(
name = "hello",
srcs = ["hello.rs"],
)
Toolchains
What’s left to do to make these actually work is to provide a Rust toolchain. In this context, a toolchain is a configuration that specifies a set of tools for building a project, such as the compiler, the linker, and various command-line tools. In this way, toolchains are decoupled from the actual rule definitions and can be easily changed to suit one’s needs.
In Buck2, toolchains are expected to be available in the toolchains
cell under a specific name. Conventionally, the toolchains
cell is located in the toolchains
directory of a project. For example, all the Rust rules depend on the target toolchains//:rust
which is defined in toolchains/BUCK
and must provide Rust specific toolchain information.
Luckily, we do not need to define a toolchain rule ourselves but can re-use the nix_rust_toolchain
rule from buck2.nix
:
# toolchains/BUCK
load("@nix//toolchains:rust.bzl", "nix_rust_toolchain")
flake.package(
name = "clippy",
binary = "clippy-driver",
path = "nix",
)
flake.package(
name = "rustc",
binaries = ["rustdoc"],
binary = "rustc",
path = "nix",
)
nix_rust_toolchain(
name = "rust",
clippy = ":clippy",
default_edition = "2021",
rustc = ":rustc",
rustdoc = ":rustc[rustdoc]",
visibility = ["PUBLIC"],
)
The rustc
target is defined almost identically as before, but the nix_rust_toolchain
rule also expects the rustdoc
attribute to be present. In this case, the rustdoc
binary is available from the rustc
Nix package as well and can be referenced using the sub-target syntax :rustc[rustdoc]
which refers to the corresponding item of the binaries
attribute given to the flake.package
rule.
Additionally, we need to pass in the clippy-driver
binary, which is available from the clippy
package in the nixpkgs
collection. Thus, the flake.nix
file needs to be changed by adding the clippy
package outputs:
# toolchains/nix/flake.nix
{
inputs = {
nixpkgs.url = "github:nixos/nixpkgs?ref=nixos-unstable";
};
outputs =
{
self,
nixpkgs,
}:
{
packages = {
aarch64-darwin.rustc = nixpkgs.legacyPackages.aarch64-darwin.rustc;
aarch64-darwin.clippy = nixpkgs.legacyPackages.aarch64-darwin.clippy;
x86_64-linux.rustc = nixpkgs.legacyPackages.x86_64-linux.rustc;
x86_64-linux.clippy = nixpkgs.legacyPackages.x86_64-linux.clippy;
}
};
}
At this point we are able to successfully build and run the target src:hello
:
(nix) $ buck2 run src:hello
Build ID: 530a4620-bfb2-454d-bae1-e937ae9e764f
Analyzing targets. Remaining 0/53 75 actions, 101 artifacts declared
Executing actions. Remaining 0/11 1.1s exec time total
Command: run. Finished 3 local
Time elapsed: 0.7s
BUILD SUCCEEDED
Hello, world!
Building a real-world Rust project would be a bit more involved. Here is an interesting article how one can do that using Bazel.
Note that buck2.nix
currently also provides toolchain rules for C/C++ and Python. Have a look at the example project provided by buck2.nix
, which you can directly use as a template to start your own project:
$ nix flake new --template github:tweag/buck2.nix my-project
A big thank you to Mercury for their support and for encouraging us to share these rules as open source! If you’re looking for a different toolchain or have other suggestions, feel free to open a new issue. Pull requests are very welcome, too!
If you’re interested in exploring a more tightly integrated solution, you might want to take a look at the buck2-nix project, which also provides Nix integration. Since it defines an alternative prelude that completely replaces Buck2’s built-in rules, we could not use it in our project but drew good inspiration from it.
Conclusion
With the setup shown, we saw that all that is needed really is Nix (pun intended1):
- we provide the
buck2
binary with Nix as part of a development environment - we leverage Nix inside Buck2 to provide build tools such as compilers, their required utilities and third-party libraries in a reproducible way
Consequently, onboarding new team members no longer means following seemingly endless and quickly outdated installation instructions. Installing nix is easy; entering the dev shell is fast, and you’re up and running in no time!
And using Buck2 gives us fast, incremental builds by only building the minimal set of dependencies needed for a specific target.
Next time, I will delve into how we seamlessly integrated the Haskell toolchain libraries from Nix and how we made it fast as well.
- The name Nix is derived from the Dutch word niks, meaning nothing; build actions don’t see anything that hasn’t been explicitly declared as an input↩
Behind the scenes
Claudio is a software engineer with a passion for functional programming, striving for correctness and elegance.
If you enjoyed this article, you might be interested in joining the Tweag team.