-
Notifications
You must be signed in to change notification settings - Fork 324
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Signed-off-by: Austin Seipp <[email protected]>
- Loading branch information
1 parent
af81e39
commit dc1c726
Showing
1 changed file
with
207 additions
and
0 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,207 @@ | ||
# Buck2 builds | ||
|
||
> [!NOTE] | ||
> This document is primarily of interest to developers. See also [Contributing] | ||
> for more information on how to contribute in general. | ||
There is experimental support for building `jj` with [Buck2], a hermetic and | ||
reproducible build system, an alternative to Cargo. | ||
|
||
> [!IMPORTANT] | ||
> Buck2 support is a work in progress, and is not yet complete. It is not | ||
> recommended for primary development use. | ||
## Step 0: Installing Dotslash | ||
|
||
We use [Dotslash] to manage Buck2 build tooling in a way that's consistent | ||
across all developers and amenable to version control. This helps ensure every | ||
developer gets consistent results (which is a big part of the selling point of | ||
hermetic build tools.) | ||
|
||
You can install Dotslash binaries by following the instructions at: | ||
|
||
- <https://dotslash-cli.com/docs/installation/> | ||
|
||
Or, if you have Rust installed, you can install Dotslash by running: | ||
|
||
```sh | ||
cargo install dotslash | ||
``` | ||
|
||
Or, if you have Nix, you can install Dotslash with Nix by running: | ||
|
||
```sh | ||
nix profile install 'nixpkgs#dotslash' | ||
``` | ||
|
||
## Building `jj` with Buck2 | ||
|
||
Assuming `dotslash` exists somewhere in your `$PATH`, you can now build `jj` | ||
with the included `buck2` dotslash file that exists under `./tools/bin`: | ||
|
||
```sh | ||
./tools/bin/buck2 run cli -- version | ||
``` | ||
|
||
Dotslash will transparently run the correct version of `buck2`, and `buck2` will | ||
build everything needed to run `jj`. | ||
|
||
## Buck2 crash course | ||
|
||
The following is an extremely minimal crash course in Buck2 concepts and how to | ||
use it. | ||
|
||
### Targets | ||
|
||
Buck2 is used to build **targets**, that exists in **packages**, which are part | ||
of a **cell**. The most explicit syntax for referring to a target is the | ||
following: | ||
|
||
```text | ||
cell//path/to/package:target-name | ||
``` | ||
|
||
You normally use a so-called "fully qualified" name like the above as an | ||
argument to `buck2 build`. | ||
|
||
A cell is a short name that maps to a directory in the code repository. A | ||
package is a subdirectory underneath the cell that contains the build rules for | ||
the targets. A target is a buildable unit of code, like a binary or a library, | ||
named in the `BUILD` file inside that package. | ||
|
||
A fully-qualified reference to a target works anywhere in the source code tree, | ||
so you can build or test any component no matter what directory you're in. | ||
|
||
So, given a cell named `foobar//` located underneath `code/foobar`, and a | ||
package `bar/baz` in that cell, leads to a file | ||
|
||
```text | ||
code/foobar/bar/baz/BUILD | ||
``` | ||
|
||
Which contains the targets that can be built. | ||
|
||
There are several shorthands for a target: | ||
|
||
- NIH. | ||
|
||
### Graphs: Target & Action | ||
|
||
NIH. | ||
|
||
### Package files | ||
|
||
NIH. | ||
|
||
### Mode files | ||
|
||
In order to support concepts like debug and release builds, we use the concept | ||
of "mode files" in Buck2. These are files that contain a list of command line | ||
options to apply to a build to achieve the desired effect. | ||
|
||
For example, to build in debug mode, you can simply include the contents of the | ||
file `mode//debug` (using cell syntax) onto the command line. This can | ||
conveniently be done with "at-file" syntax when invoking `buck2`: | ||
|
||
```sh | ||
buck2 build cli @mode//debug | ||
buck2 build cli @mode//release | ||
``` | ||
|
||
Where `@path/to/file` is the at-file syntax for including the contents of a file | ||
on the command line. This syntax supports `cell//` references to Buck cells, as | ||
well. | ||
|
||
In short, `buck2 build @mode//file` will apply the contents of `file` to your | ||
invocation. We keep a convenient set of these files maintained under the | ||
`mode//` cell, located under [`./buck/mode`](../buck/mode). | ||
|
||
#### At-file syntax | ||
|
||
The `buck2` CLI supports a convenient modern feature called "at-file" syntax, | ||
where the invocation `buck2 @path/to/file` is effectively equivalent to the | ||
bash-ism `buck2 $(cat path/to/file)`, where each line of the file is a single | ||
command line entry, in a consistent and portable way that doesn't have any limit | ||
to the size of the underlying file. | ||
|
||
For example, assuming the file `foo/bar` contained the contents | ||
|
||
```text | ||
--foo=1 | ||
--bar=false | ||
``` | ||
|
||
Then `buck2 --test @foo/bar` and `buck2 --test --foo=1 --bar=false` are | ||
equivalent. | ||
|
||
### Buck Extension Language (BXL) | ||
|
||
NIH. | ||
|
||
## Examples | ||
|
||
Some examples are included below. | ||
|
||
### Run BoringSSL speed tests | ||
|
||
```sh | ||
buck2 run third-party//bssl:bssl @mode//release -- speed | ||
``` | ||
|
||
### Build all third-party Rust dependencies | ||
|
||
```sh | ||
buck2 build third-party//rust | ||
``` | ||
|
||
### Build all `http_archive` dependencies | ||
|
||
```sh | ||
buck2 build $(buck2 uquery "kind('http_archive', deps('//...'))" | grep third-party//) | ||
``` | ||
|
||
## Development notes | ||
|
||
Notes for `jj` developers using Buck2. | ||
|
||
### Build mode reference | ||
|
||
- `@mode//debug` | ||
- `@mode//release` | ||
|
||
### Cargo dependency management | ||
|
||
Although Buck2 downloads and runs `rustc` on its own to build crate | ||
dependencies, our `Cargo.toml` build files act as the source of truth for | ||
dependency information in both Cargo and Buck2. | ||
|
||
Updating the dependency graph for Cargo-based projects typically comes in one of | ||
two forms: | ||
|
||
- Updating a dependency version in the top-level workspace `Cargo.toml` file | ||
- Adding a newly required dependency to `[dependencies]` in the `Cargo.toml` | ||
file for a crate | ||
|
||
After doing either of these actions, you can synchronize the Buck2 dependencies | ||
with the Cargo dependencies with the following script: | ||
|
||
```bash | ||
./buck/third-party/rust/synchronize.py | ||
``` | ||
|
||
This must be run from the root of the repository. | ||
|
||
This will re-synchronize all `third-party//rust` crates with the versions in the | ||
workspace Cargo file, and then also update the `BUILD` files in the source code | ||
with any newly added build dependencies that were added or removed (not just | ||
updated). | ||
|
||
### `rust-analyzer` support | ||
|
||
Coming soon. | ||
|
||
<!-- References --> | ||
|
||
[Contributing]: https://martinvonz.github.io/jj/latest/contributing/ | ||
[Buck2]: https://buck2.build/ | ||
[Dotslash]: https://dotslash-cli.com/ |