Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
257 changes: 257 additions & 0 deletions docs/Developer-Guide_Extension-ccache-remote.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,257 @@
# Extension: ccache-remote

[ccache](https://ccache.dev/) significantly speeds up repeated kernel and
U-Boot builds by caching compiled objects. By default, each build host has
its own local cache. This extension adds a **shared remote cache** so that
multiple build machines (or a CI server and a developer workstation) can all
benefit from objects already compiled by someone else.

Typical scenarios:

- A home lab with several SBCs used as build hosts — first build populates
the cache, subsequent builds on other machines hit it.
- A CI server and developer laptops sharing one cache server so that a
developer rarely needs to compile from scratch after CI has run.
- A single machine where the local cache was lost (disk wiped) but the
remote cache is still warm.

The extension supports two backends:

- **Redis** — lower latency, recommended for fast local networks. Cache lives
entirely in RAM on the server, so capacity is limited by available memory.
- **HTTP/WebDAV** (nginx) — somewhat slower, but stores cache on disk. Can
be orders of magnitude larger than a RAM-backed Redis instance, making it
a better choice when cache size matters more than speed.

### Auto-discovery vs. explicit configuration

The simplest deployment uses **Avahi (DNS-SD)**. Once the cache server is
set up and publishing its service, and Avahi is installed on all build hosts,
enabling the extension is all that is needed — no URL configuration required.

If Avahi is not available or not desirable, the server URL must be provided
explicitly via `CCACHE_REMOTE_STORAGE`, or for remote servers via
`CCACHE_REMOTE_DOMAIN` (DNS SRV). See [Parameters](#parameters) below.

The extension automatically enables `USE_CCACHE=yes` and handles all ccache
configuration including Docker pass-through.

## Quick start

If a cache server is already running on the local network and advertising
itself via DNS-SD (Avahi), no configuration is needed:

```bash
./compile.sh BOARD=<board> BRANCH=<branch> ENABLE_EXTENSIONS="ccache-remote"
```

With an explicit Redis server:

```bash
./compile.sh BOARD=<board> BRANCH=<branch> \
ENABLE_EXTENSIONS="ccache-remote" \
CCACHE_REMOTE_STORAGE="redis://192.168.1.65:6379"
```

With an explicit HTTP/WebDAV server:

```bash
./compile.sh BOARD=<board> BRANCH=<branch> \
ENABLE_EXTENSIONS="ccache-remote" \
CCACHE_REMOTE_STORAGE="http://192.168.1.65:8088/ccache/"
```

To enable permanently from a userconfig file:

```bash
enable_extension "ccache-remote"
# optionally:
# CCACHE_REMOTE_STORAGE="redis://192.168.1.65:6379"
```

## Requirements

- **ccache**: version 4.4 or newer (for remote storage support).
- **Redis backend**: Redis server accessible from the build host.
- **HTTP backend**: nginx with WebDAV module (`nginx-extras`).
- **Auto-discovery**: `avahi-browse` (package `avahi-utils`) on the build host.

## Parameters

| Variable | Default | Description |
|---|---|---|
| **`CCACHE_REMOTE_STORAGE`** | _(empty)_ | Remote storage URL. If not set, auto-discovery is attempted. |
| **`CCACHE_REMOTE_DOMAIN`** | _(empty)_ | Domain for DNS SRV discovery (e.g. `example.com`). |
| **`CCACHE_REMOTE_ONLY`** | `no` | Set to `yes` to disable local cache and use only remote storage (saves local disk space). |
| **`CCACHE_REDIS_CONNECT_TIMEOUT`** | `500` | Redis connection timeout in milliseconds. |
| **`CCACHE_READONLY`** | _(empty)_ | Set to any value to use cache read-only (don't write new entries). |
| **`CCACHE_RECACHE`** | _(empty)_ | Recompile everything and update cache (bypass existing entries). |
| **`CCACHE_RESHARE`** | _(empty)_ | Push existing local cache entries to remote storage. |
| **`CCACHE_MAXSIZE`** | _(empty)_ | Maximum local cache size (e.g. `10G`). |
| **`CCACHE_NAMESPACE`** | _(empty)_ | Namespace for cache isolation between projects or branches. |
| **`CCACHE_DISABLE`** | _(empty)_ | Set to any value to disable ccache completely. |

For the full list of supported variables see the `CCACHE_PASSTHROUGH_VARS`
array in the extension source or the
[ccache configuration reference](https://ccache.dev/manual/latest.html#config_remote_storage).

### CCACHE_REMOTE_STORAGE URL format

```
Redis: redis://[[USER:]PASSWORD@]HOST[:PORT][|attribute=value...]
HTTP: http://HOST[:PORT]/PATH/[|attribute=value...]
```

Common attributes:

| Attribute | Default | Description |
|---|---|---|
| `connect-timeout` | `100` ms | Connection timeout. Increase on slow networks. |
| `operation-timeout` | `10000` ms | Per-operation timeout. |

Examples:

```
redis://192.168.1.65:6379|connect-timeout=500
redis://default:secretpass@192.168.1.65:6379|connect-timeout=500
http://192.168.1.65:8088/ccache/
```

!!! warning
Redis passwords must not contain `/`, `+`, `=`, or spaces — these break
URL parsing. Generate a safe password with:
```bash
openssl rand -hex 24
```

## Server discovery

When `CCACHE_REMOTE_STORAGE` is not set, the extension tries to discover a
cache server automatically in this order:

1. **DNS-SD** (mDNS / Avahi): browse for `_ccache._tcp` on the local network.
Requires `avahi-browse` on the build host and the cache server publishing
a DNS-SD service. Redis is preferred over HTTP when both are found.

2. **DNS SRV**: query `_ccache._tcp.DOMAIN` when `CCACHE_REMOTE_DOMAIN` is
set. Useful for remote or cloud-hosted build servers.

3. **Legacy mDNS**: resolve the hostname `ccache.local` (falls back to Redis
on port 6379). Kept for backward compatibility.

If none of these succeeds, the extension silently falls back to local cache only.

## Cache sharing requirements

For the cache to be shared across multiple build hosts, **the Armbian source
tree must be at the same path on all machines** (e.g. `/home/build/armbian`).
ccache includes the working directory in the cache key, so different paths
produce different keys and the cache will not be shared.

Docker builds are not affected — they always use a consistent internal path.

## Server setup

### Redis (recommended)

1. Install:

```bash
apt install redis-server avahi-daemon avahi-utils
```

2. Configure Redis — copy `misc/redis/redis-ccache.conf` from the extension
directory and include it in `/etc/redis/redis.conf`:

```bash
echo "include /etc/redis/redis-ccache.conf" >> /etc/redis/redis.conf
systemctl restart redis-server
```

3. Set a password (recommended for any non-isolated network):

```bash
openssl rand -hex 24 # use output as password
# set requirepass <password> in redis-ccache.conf
```

4. Publish the DNS-SD service so clients discover it automatically:

```bash
# Static (always advertise):
cp misc/avahi/ccache-redis.service /etc/avahi/services/

# Or tied to redis-server lifecycle:
cp misc/systemd/ccache-avahi-redis.service /etc/systemd/system/
systemctl enable --now ccache-avahi-redis
```

### HTTP/WebDAV (nginx)

1. Install:

```bash
apt install nginx-extras avahi-daemon avahi-utils
```

2. Enable the WebDAV site:

```bash
cp misc/nginx/ccache-webdav.conf /etc/nginx/sites-available/ccache-webdav
ln -s /etc/nginx/sites-available/ccache-webdav /etc/nginx/sites-enabled/
mkdir -p /var/cache/ccache-webdav/ccache
chown -R www-data:www-data /var/cache/ccache-webdav
systemctl reload nginx
```

3. Verify:

```bash
curl -X PUT -d "test" http://localhost:8088/ccache/test.txt
curl http://localhost:8088/ccache/test.txt
```

!!! warning
The provided nginx configuration has no authentication. Use only on a
trusted private network.

4. Publish the DNS-SD service:

```bash
cp misc/avahi/ccache-webdav.service /etc/avahi/services/
```

### DNS SRV records (remote servers)

For build hosts that cannot reach the cache server via mDNS, set
`CCACHE_REMOTE_DOMAIN` and create DNS records:

Redis backend:
```text
_ccache._tcp.example.com. SRV 0 0 6379 ccache.example.com.
_ccache._tcp.example.com. TXT "type=redis"
```

HTTP/WebDAV backend:
```text
_ccache._tcp.example.com. SRV 0 0 8088 ccache.example.com.
_ccache._tcp.example.com. TXT "type=http" "path=/ccache/"
```

### mDNS client requirements

For `.local` hostname resolution to work on the build host:

```bash
# Option A: systemd-resolved
apt install libnss-resolve

# Option B: standalone
apt install libnss-mdns
```

## References

- [ccache remote storage documentation](https://ccache.dev/manual/latest.html#config_remote_storage)
- [ccache Redis how-to](https://ccache.dev/howto/redis-storage.html)
- [ccache HTTP how-to](https://ccache.dev/howto/http-storage.html)
118 changes: 118 additions & 0 deletions docs/Developer-Guide_Extension-kernel-rust.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,118 @@
# Extension: kernel-rust

The Linux kernel has been gaining Rust support since version 6.1, with more
subsystems and drivers being written in Rust alongside traditional C code.
In the near future, practically useful drivers and subsystems for Armbian users
are expected to require or benefit from Rust kernel support. Enabling
`CONFIG_RUST` in the kernel build unlocks this capability.

Without this extension, Armbian kernels are built without Rust support —
the toolchain is not installed and `CONFIG_RUST` does not appear in
menuconfig. This extension installs a pinned Rust toolchain via `rustup`,
enables `CONFIG_RUST` automatically, and configures all required build
parameters so that Rust kernel modules can be compiled alongside C code.
The toolchain versions and other aspects of the build can be adjusted via
[parameters](#parameters) passed to the build system.

## Why rustup instead of distro packages

Distribution-provided Rust packages currently cannot fully satisfy all the
requirements for enabling Rust in a recent kernel. Mixing some components
from the distro with others from external sources is fragile and not
worthwhile. For this reason the extension downloads a complete, version-pinned
toolchain via `rustup` independently of the host distro.

When the base distro used by the Armbian build system provides packages
sufficient to build the Rust environment for the latest kernels entirely from
distro sources, Armbian will likely switch to using them.

## Quick start

```bash
./compile.sh BOARD=<board> BRANCH=<branch> ENABLE_EXTENSIONS="kernel-rust"
```

To open menuconfig with Rust options visible:

```bash
./compile.sh kernel-config BOARD=<board> BRANCH=<branch> ENABLE_EXTENSIONS="kernel-rust"
```

To enable permanently from a userconfig file:

```bash
enable_extension "kernel-rust"
```

## Requirements

- **Kernel version**: 6.12 or newer. The minimum required rustc version depends on the kernel release; see the [Rust-for-Linux version policy](https://rust-for-linux.com/rust-version-policy).
- **Host package**: `libclang-dev` — installed automatically by the extension.
- **Build host architecture**: x86_64, aarch64, or riscv64.
- Internet access on first use (downloads rustup-init and toolchain components).

## Parameters

| Variable | Default | Description |
|---|---|---|
| **`RUST_VERSION`** | `1.85.0` | rustc version installed via rustup. |
| **`BINDGEN_VERSION`** | `0.71.1` | `bindgen-cli` version installed via cargo. |
| **`RUST_KERNEL_SAMPLES`** | `no` | Set to `yes` to build sample Rust kernel modules (`rust_minimal`, `rust_print`, `rust_driver_faux`) as loadable modules. Useful for smoke-testing the toolchain. |
| **`RUST_EXTRA_COMPONENTS`** | _(empty array)_ | Additional rustup components to install (e.g. `clippy`, `llvm-tools`). |
| **`RUST_EXTRA_CARGO_CRATES`** | _(empty array)_ | Additional cargo crates to install. Supports `name` or `name@version` syntax. |

## Toolchain cache

The toolchain is installed once into `${SRC}/cache/tools/rustup/` and reused
across builds. The cache is content-addressed by a hash of:

- `RUST_VERSION`
- `BINDGEN_VERSION`
- build host architecture
- `RUST_EXTRA_COMPONENTS`
- `RUST_EXTRA_CARGO_CRATES`

Changing any of these values invalidates the cache and triggers a full reinstall
on the next build. The marker file is `.marker-<hash>` inside the cache directory.

## Extensibility

Other extensions can request additional toolchain components or crates before
the toolchain is installed:

```bash
# In your extension file:
RUST_EXTRA_COMPONENTS+=("clippy" "llvm-tools")
RUST_EXTRA_CARGO_CRATES+=("mdbook" "cargo-deb@2.11.0")
```

## Kernel artifact versioning

The extension adds a short hash of `RUST_VERSION|BINDGEN_VERSION` to the kernel
artifact version string (key `_R`, e.g. `rust1a2b`). This ensures that changing
toolchain versions triggers a kernel rebuild even if the kernel source is
unchanged.

## Implementation notes

### env -i isolation

The kernel build runs under `env -i`, which clears the entire environment.
The extension passes Rust tool paths directly as make parameters
(`RUSTC=`, `RUSTFMT=`, `BINDGEN=`) and sets `RUST_LIB_SRC` via the make
environment array. Direct paths into the toolchain sysroot are used instead
of rustup proxy binaries, so `RUSTUP_HOME` does not need to be in the
cleared environment.

### ccache and Rust

ccache does not support rustc (upstream won't-fix since 2019). The kernel
build system has no `RUSTC_WRAPPER` mechanism, so Rust compilation is not
cached by ccache. Only C/assembly compilation benefits from ccache when this
extension is active.

## References

- [Rust in the Linux kernel — quick start](https://docs.kernel.org/rust/quick-start.html)
- [Rust for Linux — version policy](https://rust-for-linux.com/rust-version-policy)
- [rustup installation](https://rust-lang.github.io/rustup/installation/index.html)
Loading
Loading