Introduction

ninja-files is a collection of rust functions to make writing build.ninja files easy to accomplish in rust. This allows rust crates and modules to be used to provide "meta-build" functionality, then leaning on ninja to provide build functionality.

Ninja itself is fast, and able to only rebuild when needed, much like make, but supports more functionality. However, ninja explicitly does not support wildcards, and other forms of file discovery. This makes it difficult to use directly.

Other build tools support outputting to ninja, such as cmake, however you need to learn the cmake or other build tool syntax.

This project lets you use programming language to 'build your own' generator. This way you can think in normal programming concepts, e.g walk this directory. In fact, as there is excellent rust crates for this, you can just pull them in and use them.

Contributing

ninja-files is free and open source. The code is hosted on sourcehut, and patches are welcome to the mailing list.

ninja-files is not just aiming to be the components, but also a mono-repo in which lots of useful functions are stored. This encourages the re-useability, so the effort spent writing rules for a particular program can be re-used

Installation

The easiest way to add the ninja files capability to your project is to create a rust project that consumes it.

Creating a configure executable

cargo new tools/configure

This would create a 'configure' binary application in the tools folder in your project. This can actually be anywhere, but assuming you have other folders and languages, we have put it 'out of the way', to not pollute the directory tool much.

Adding the dependencies

In your Cargo.toml add the below

[dependencies]
ninja-files = { version = "0.1.0" , features = ["format"]}

This adds the meta-package, with serialization included. This is required if you wish to write it to a file. However you can also write your own serializer if you require, nothing about the code will stop you, though you shouldn't need too.

Writing your main function

The barebones setup only requires the below In tools/configure/src/main.rs change to the below

fn main() {
    let ninja = FileBuilder::new().build().unwrap();
    let file = std::fs::File::create("build.ninja").unwrap();
    let _ = write_ninja_file(&ninja, file).unwrap();
}

Note there is no elegant error handling, crashing is most likely okay in a configure script, as we exit-out with an error

The above creates a empty ninja file, but it does create it.

This can be ran with cargo run -p configure which will compile the configure program

./configure in repository root

While the above works, it's not that intuitive for new project contributors. A general standard does exist in build systems. That is the ./configure script

Luckily, it's easy to setup a simple script that just calls the rust one.

Make the below file in your repositories root, calling it configure

#!/usr/bin/env -S cargo run --package configure

Make it executable chmod +x ./configure

Now users can run ./configure and have the build.ninja created.

Running ninja

As the above produces a build.ninja file, you can just run ninja in the project root, and it will rebuild anything out of date. Now the above sample doesn't list anything to build, so it won't do much yet

Reconfigure

One of the problems with a ./configure script. Is that we might not always remember to re-run it, this is pretty annoying. Your next build might not pick up dependencies, or worse, succeed but not have everything you wanted to include.

Ninja provides a solution to this in the form of generators.

A simple generator

As your project may have different needs, this is provided as sample code, that can be tweaked as needed

Add the ninja files cargo crate as a dependency

ninja-files-cargo = { version = "0.1.0" }

#![allow(unused)]

fn main() {
const BUILD_NINJA: &str = "build.ninja";
const CONFIGURE_TOML: &str = "util/configure/Cargo.toml";
const CONFIGURE_EXE: &str = "target/debug/configure";
const CONFIGURE_RULE: &str = "configure";

fn configure() -> FileBuilder {
    let configure_rule = {
        let command = CommandBuilder::new(CONFIGURE_EXE);
        RuleBuilder::new(command).generator(true)
    };

    let cargo = ninja_files_cargo::build(CONFIGURE_TOML, CONFIGURE_EXE);

    let configure_file = { BuildBuilder::new(CONFIGURE_RULE).explicit(CONFIGURE_EXE) };

    FileBuilder::new()
        .rule(CONFIGURE_RULE, configure_rule)
        .file(BUILD_NINJA, configure_file)
        .merge(&cargo)
}
}

This is a simple case of building a rule and a build target in ninja. The rule is just running our configure program (and this rule is defined in the configure program). The build rule is stating that build.ninja is dependent on the compiled configure program (the rust program, not the entrypoint in repo root) The configure program is built by the cargo crate. This is smart enough to know when the source .rs files have changed, and rebuild the configure program when needed

This means that the build ninja, is dependent on the rust program and is regenerated if the rust program changes. Very handy as you add in new rules

Merge with the other rules

The function is good and all, but how do we actually use it?. Luckily the FileBuilder type supports merge. We can merge multiple files together before finally rendering them.

Change the main.rs from installation to look like below

fn main() {
	let configure = configure();
    let ninja = FileBuilder::new().merge(&configure).build().unwrap();
    let file = std::fs::File::create("build.ninja").unwrap();
    let _ = write_ninja_file(&ninja, file).unwrap();
}

We will need to run ./configure or cargo run -p configure to get the build.ninja to have our new capabilty the first time, but after that any run of ninja will rebuild build.ninja if it needs to

Cargo

The cargo module shouldn't be too suprising, we use it to build rust projects. Technically in a rust only project, this might not have much value, cargo itself exists, why bother not just re-use that. The reasoning lies in when trying to have a project that does much more than just compile rust.

Rust as 'scripting'

When building projects, we are really running programs to produce files. Most developers could easily do this in their favorite language, like rust. The cargo module makes this easy, you could produce a binary crate, that reads args and writes files out. This could then be used to produce something more complicated, like a static html website, but you can manipulate your program in the repo itself, rather than requiring an external tool.

The cargo module shines here, as it's able to also know when the sources of the 'template' program changed, and thus will rebuild the templating program. Meaning up to date outputs.

Installation

Simply add to your dependencies

[dependencies]
ninja-files-cargo = "0.1.0"

Usage

This exposes a function ninja-files-cargo::build, which takes the Cargo.toml, and the target exe that will be built. This works even in workspaces (you just need to track where the file will be output in the workspace).

It also works for libraries, so can be a cheap way to 'build all' including rust libs, without remembering all the flags and commands

Passwordstore

The password store module is designed to get passwords from pass https://www.passwordstore.org

This allows you to 'get' secrets that are required, avoiding the checking into a source repository problem. This is handy when using ninja more as a 'Continous Delivery' tool, as it can get the password from pass and then provide that as inputs to further stages.

This module actually exists to facilitate building kubernetes and talosos configurations

File changes

A benefit of managing passwords this way, is we only 'rebuild' the password when the gpg file changes. Passwordstore encrypts the passwords in the users home directory, so there is a direct file dependency between that and the output. Perfect for ninja to manage.

You can rotate your passwords, re-run ninja, and have your deploy scripts use the new password. Re-running the deploy script won't have to try to grab the password again.