[megatron, vllm] feat: NVFP4 (W4A16) QAT training support via ModelOpt

[jQizhang]

What does this PR do?

This PR adds support for NVFP4 (W4A16) Quantization-Aware Training (QAT) in verl's Megatron training pipeline, with quantized weight transfer to the vLLM rollout engine for inference. The implementation leverages NVIDIA ModelOpt for quantization.

Checklist Before Starting

• Search for similar PRs. Paste at least one query link here: ...
• Format the PR title as [{modules}] {type}: {description} (This will be checked by the CI)
  • {modules} include fsdp, megatron, veomni, sglang, vllm, rollout, trainer, ci, training_utils, recipe, hardware, deployment, ray, worker, single_controller, misc, perf, model, algo, env, tool, ckpt, doc, data, cfg, reward
  • If this PR involves multiple modules, separate them with , like [megatron, fsdp, doc]
  • {type} is in feat, fix, refactor, chore, test
  • If this PR breaks any API (CLI arguments, config, function signature, etc.), add [BREAKING] to the beginning of the title.
  • Example: [BREAKING][fsdp, megatron] feat: dynamic batching

Test

This feature has been validated through end-to-end QAT training experiments:

• Model: Qwen3-8B (dense) and Qwen3-30B-A3B (MoE)
• Setup: Training with Megatron, inference with vLLM
• Quantization: NVFP4 W4A16 via ModelOpt
• Verification: Confirmed that QAT-calibrated quantized weights are correctly transferred to vLLM and produce valid inference outputs (matching ModelOpt standalone export baselines)

API and Usage Example

Enable NVFP4 QAT by adding the following to the actor Megatron config:

actor:
  megatron:
    quantization: nvfp4      # Quantization method
    enable_qat: true          # Enable QAT during training

rollout:
  quantization: nvfp4         # Tell vLLM to expect NVFP4 quantized weights

The training loop automatically:

1. Applies NVFP4 QAT quantizers to the Megatron actor model at initialization
2. Trains with simulated quantization (forward pass uses fake-quantized weights)
3. Before rollout, converts bf16 weights to NVFP4 format (packed uint8 + scales) and streams them to vLLM
4. vLLM loads the quantized weights using patched ModelOptNvFp4LinearMethod / ModelOptNvFp4FusedMoE for inference

No changes to the training script are required beyond the configuration above.

Design & Code Changes

Demonstrate the high-level design if this PR is complex, and list the specific changes.

Checklist Before Submitting

Important

Please check all the following items before requesting a review, otherwise the reviewer might deprioritize this PR for review.

• Read the Contribute Guide.
• Apply pre-commit checks: pre-commit install && pre-commit run --all-files --show-diff-on-failure --color=always
• Add / Update the documentation.
• Add unit or end-to-end test(s) to the CI workflow to cover all the code. If not feasible, explain why: ...
• Once your PR is ready for CI, send a message in the ci-request channel in the verl Slack workspace. (If not accessible, please try the Feishu group (飞书群).)
• If your PR is related to the recipe submodule, please also update the reference to the submodule commit via git submodule update --remote or cd recipe && git pull origin main.

[CLAassistant]

Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.

---

root seems not to be a GitHub user. You need a GitHub account to be able to sign the CLA. If you have already a GitHub account, please add the email address used for this commit to your account.
_{You have signed the CLA already but the status is still pending? Let us recheck it.}

[gemini-code-assist]

Code Review

This pull request introduces support for NVFP4 Quantization-Aware Training (QAT) by integrating with NVIDIA's ModelOpt. The changes are comprehensive, touching upon configuration, the Megatron training worker, and adding utility modules for QAT processing and vLLM patching. The implementation appears well-thought-out, especially in handling the complexities of distributed training environments (TP, PP, EP). My main feedback is on a piece of duplicated code that should be refactored to improve maintainability. Overall, this is a solid feature addition.

[gemini-code-assist]

This helper function _create_param_from_subclass_attributes is a duplicate of the one defined at the top level of this file (line 107). To avoid code duplication and improve maintainability, please remove this inner function and use the top-level one instead. The other function process_weights_after_loading_moe in this file already uses the top-level helper.

[ISEEKYAN]

better to use the same configs as in FSDP to avoid confusing the users