docs: add analytic frule/rrule formulas for scalar operations #16
Description
Summary
The cases/ directory contains numerical oracle data (finite-difference and PyTorch references) for scalar operations, but lacks documentation of the analytic derivative formulas (frule/rrule) that implementations should follow.
Motivation
chainrules-scalarops in tenferro-rs is being extended to provide frule/rrule for all standard scalar operations. Having the canonical formulas documented here ensures:
- A single source of truth for derivative correctness across implementations
- Clear specification for complex-valued Wirtinger derivatives (∂f/∂z̄ convention)
- Easy review of edge cases (e.g., branch cuts, singularities at zero)
Scope
Add a document (e.g., docs/scalar-ad-formulas.md) covering at minimum:
Unary real/complex
| op | primal | frule: df = ? |
rrule: ∂L/∂x = ? |
|---|---|---|---|
| sqrt | √x | dx / (2√x) | ḡ / (2√x) |
| exp | eˣ | eˣ · dx | ḡ · eˣ |
| expm1 | eˣ−1 | eˣ · dx | ḡ · eˣ |
| log | ln x | dx / x | ḡ / x |
| log1p | ln(1+x) | dx / (1+x) | ḡ / (1+x) |
| sin | sin x | cos(x) · dx | ḡ · cos(x) |
| cos | cos x | −sin(x) · dx | −ḡ · sin(x) |
| tan | tan x | dx / cos²(x) | ḡ / cos²(x) |
| tanh | tanh x | (1−tanh²x) · dx | ḡ · (1−tanh²x) |
| asin | arcsin x | dx / √(1−x²) | ḡ / √(1−x²) |
| acos | arccos x | −dx / √(1−x²) | −ḡ / √(1−x²) |
| atan | arctan x | dx / (1+x²) | ḡ / (1+x²) |
| sinh | sinh x | cosh(x) · dx | ḡ · cosh(x) |
| cosh | cosh x | sinh(x) · dx | ḡ · sinh(x) |
| asinh | arcsinh x | dx / √(x²+1) | ḡ / √(x²+1) |
| acosh | arccosh x | dx / √(x²−1) | ḡ / √(x²−1) |
| atanh | arctanh x | dx / (1−x²) | ḡ / (1−x²) |
| conj | x̄ | d̄x | ḡ̄ |
Binary
| op | frule | rrule (∂L/∂x, ∂L/∂y) |
|---|---|---|
| add | dx + dy | (ḡ, ḡ) |
| sub | dx − dy | (ḡ, −ḡ) |
| mul | dx·ȳ + dy·x̄ | (ḡ·ȳ, ḡ·x̄) |
| div | dx/ȳ − dy·x̄/y² | (ḡ/ȳ, −ḡ·x̄/y²) |
| powf | n·x^(n−1)·dx | ḡ·n·x^(n−1) |
| powi | n·x^(n−1)·dx | ḡ·n·x^(n−1) |
Notes to include
- Complex convention: Wirtinger calculus with conj() on partial derivatives where needed
- Edge cases / singularities (sqrt at 0, log at 0, etc.)
- Relationship to existing oracle test data in
cases/
Blocked by
- tensor4all/tenferro-rs chainrules-scalarops Phase 2a (adding the missing frule/rrule implementations)
Once the implementations are finalized, the formulas here should match exactly.