ANE llama runner fixes for iOS26

[metascroy]

This PR:

• Resolves [CoreML Backend] macOS 26.1 ANE regression: fp16 LLaMA inference produces inf/nan (worked on macOS 15.7) #15833. The main issue is that iOS26 has a bug in its ANE implementation of SDPA, so we break SDPA into matmuls and softmax to avoid this.
• iOS26 also has large performance regressions over iOS18, so we tweak the model slightly (by exporting in parts) to achieve iOS18-level performance.
• We also add a C++ runner for the model split in parts.

With these changes, we estimate performance of Llama1B on iPhone 15 Pro / iOS 26 at:

• 30 tok/sec decode
• 1900 tok/sec prefill

Differential Revision: D88083155

[pytorch-bot]

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/16057

• 📄 Preview Python docs built from this PR

Note: Links to docs will display an error until the docs builds have been completed.

❌ 3 New Failures, 7 Cancelled Jobs

As of commit c78b111 with merge base 144a37d ():

NEW FAILURES - The following jobs have failed:

• Apple / build-demo-ios / macos-job (gh)
  RuntimeError: Command bash /Users/runner/work/_temp/exec_script failed with exit code 128
• pull / android / run-emulator (gh)
  The process '/usr/bin/sh' failed with exit code 255
• trunk / test-mcu-cortex-m-backend / linux-job (gh)
  RuntimeError: Command docker exec -t 099749671a11b5adf3fc2f3622cf7420ad0053d21a83452e66d3c78212f33fcd /exec failed with exit code 1

CANCELLED JOBS - The following jobs were cancelled. Please retry:

• periodic / test-models-linux (buck2, mv3, portable, linux.2xlarge, 90) / linux-job (gh)
• periodic / test-models-linux (buck2, mv3, xnnpack-quantization-delegation, linux.2xlarge, 90) / linux-job (gh)
• periodic / test-models-linux (cmake, mv3, portable, linux.2xlarge, 90) / linux-job (gh)
• periodic / test-models-linux (cmake, mv3, xnnpack-quantization-delegation, linux.2xlarge, 90) / linux-job (gh)
• periodic / test-models-linux (cmake, vit, portable, linux.2xlarge, 90) / linux-job (gh)
• periodic / test-models-linux (cmake, vit, xnnpack-quantization-delegation, linux.2xlarge, 90) / linux-job (gh)
• trunk / test-models-macos-cpu (llama3_2_vision_encoder, portable) / macos-job (gh)
  ##[error]The operation was canceled.

This comment was automatically generated by Dr. CI and updates every 15 minutes.

[meta-codesync]

@metascroy has exported this pull request. If you are a Meta employee, you can view the originating Diff in D88083155.

[github-actions]

This PR needs a release notes: label

If your change should be included in the release notes (i.e. would users of this library care about this change?), please use a label starting with release notes:. This helps us keep track and include your important work in the next release notes.

To add a label, you can comment to pytorchbot, for example
@pytorchbot label "release notes: none"

For more information, see
https://github.com/pytorch/pytorch/wiki/PyTorch-AutoLabel-Bot#why-categorize-for-release-notes-and-how-does-it-work.