feat: add ClassifierStrategy to gate ANE vs CPU-tiled classifier for large-vocab models

[christopherkarani]

Summary

• Add ClassifierStrategy enum that selects between ANE and CPU-tiled classifier paths based on model vocab/dModel size
• Gate ANE classifier compilation in ensureHybridCompiledLlama — skips compile for large-vocab models (saves 1 ANE compile)
• Add CPU-tiled greedy decode path using FP16TiledClassifier.tiledMatvecArgmax with CPU RMSNorm
• Pre-convert lmHead weights to FP16 at build time (no per-token conversion overhead)
• Complete UInt16 → TokenID migration across codebase

Test plan

• ClassifierStrategyTests — 7 tests (selection logic + boundary + FP16TiledClassifier correctness)
• Full test suite — 208/210 passed (2 pre-existing MigrationParityTests failures)
• Hardware verification: TinyLlama (32K vocab) no longer crashes with statusType=0x9

🤖 Generated with Claude Code

[christopherkarani]

Code review

Found 4 issues:

1. RoPE hook silently drops error context (CLAUDE.md says "ALWAYS handle errors comprehensively... Never silently swallow errors")

The applyRoPEHook closure catches SurfaceIO errors, discards them into a readOK = false boolean, then throws a generic invalidArguments("RoPE hook surface read failed") with no underlying error details. Every other error handler in this file preserves context via "\(error)" interpolation. This is the only call site that swallows it, making hardware debugging significantly harder.

https://github.com/christopherkarani/Espresso/blob/28142453e779b7813d29eb2cfb25b395f2f7bdfe/Sources/RealModelInference/RealModelInferenceEngine.swift#L2338-L2345

2. Llama generation never stops on EOS token (comment at line 2527 acknowledges this but does not implement it)

generateIncrementalHybridLlama only stops when effectiveMaxTokens or maxSeq is reached. The comment says "Llama EOS varies by model -- use vocab-1 as a safe sentinel or check config" but no check is implemented. The GPT-2 path has explicit if nextToken == Self.gpt2EOSToken { break } guards. Generation will produce garbage output past the model's natural stop point.

https://github.com/christopherkarani/Espresso/blob/28142453e779b7813d29eb2cfb25b395f2f7bdfe/Sources/RealModelInference/RealModelInferenceEngine.swift#L2525-L2530

3. GPT-2 greedy path does not check classifierStrategy, will attempt ANE classifier for large-vocab GPT-2

The Llama path correctly gates ANE classifier compilation on classifierStrategy == .ane (line 1659) and adds it to the useANEGreedyHead check (line 2299). The GPT-2 path in ensureHybridCompiled (line 1544) unconditionally compiles the ANE classifier, and generateIncrementalHybrid (line 1739) does not check classifierStrategy. GPT-2-124M has vocab=50257, dModel=768 (38.6M elements > 16M limit), so classifierStrategy would be .cpuTiled, yet the GPT-2 path ignores this and attempts to use the ANE classifier anyway.

https://github.com/christopherkarani/Espresso/blob/28142453e779b7813d29eb2cfb25b395f2f7bdfe/Sources/RealModelInference/RealModelInferenceEngine.swift#L1737-L1743

4. GGUFModelLoader does not read ropeTheta from GGUF metadata, silently defaults to 10,000

GGUFModelLoader.prepare() constructs MultiModelConfig without setting ropeTheta, so it defaults to 10_000.0. The same PR registers Llama 3.2 models with ropeTheta: 500_000.0 in ModelRegistry. A GGUF-loaded Llama 3.2 model will silently use the wrong RoPE base frequency, producing incorrect positional encodings.

https://github.com/christopherkarani/Espresso/blob/28142453e779b7813d29eb2cfb25b395f2f7bdfe/Sources/EspressoGGUF/GGUFModelLoader.swift#L52-L66

---

Generated with Claude Code

_{- If this code review was useful, please react with 👍. Otherwise, react with 👎.}