[recipe] feat: add QWen 30b moe dapo script that can run on a single 80GB node

[vermouth1992]

What does this PR do?

• As title
• Achieves around 0.28 AIME'24 after 100 steps which takes around 1 day on a H800 single node
• Note that we start from base model

Checklist Before Starting

• Search for similar PRs. Paste at least one query link here: ...
• Format the PR title as [{modules}] {type}: {description} (This will be checked by the CI)
  • {modules} include fsdp, megatron, sglang, vllm, rollout, trainer, ci, training_utils, recipe, hardware, deployment, ray, worker, single_controller, misc, perf, model, algo, env, tool, ckpt, doc, data
  • If this PR involves multiple modules, separate them with , like [megatron, fsdp, doc]
  • {type} is in feat, fix, refactor, chore, test
  • If this PR breaks any API (CLI arguments, config, function signature, etc.), add [BREAKING] to the beginning of the title.
  • Example: [BREAKING][fsdp, megatron] feat: dynamic batching

Test

For changes that can not be tested by CI (e.g., algorithm implementation, new model support), validate by experiment(s) and show results like training curve plots, evaluation results, etc.

API and Usage Example

Demonstrate how the API changes if any, and provide usage example(s) if possible.

# Add code snippet or script demonstrating how to use this

Design & Code Changes

Demonstrate the high-level design if this PR is complex, and list the specific changes.

Checklist Before Submitting

Important

Please check all the following items before requesting a review, otherwise the reviewer might deprioritize this PR for review.

• Read the Contribute Guide.
• Apply pre-commit checks: pre-commit install && pre-commit run --all-files --show-diff-on-failure --color=always
• Add / Update the documentation.
• Add unit or end-to-end test(s) to the CI workflow to cover all the code. If not feasible, explain why: ...
• Once your PR is ready for CI, send a message in the ci-request channel in the verl Slack workspace.

[gemini-code-assist]

Code Review

This pull request introduces a new recipe script for running DAPO on a Qwen-3 30B model and updates the reward scoring logic to support a new MATH dataset. There's a critical configuration error in the script that will prevent it from running, as well as a high-severity issue with hardcoded paths that impacts reusability.

[gemini-code-assist]

The fsdp_size is hardcoded to 32, but the script is configured to run on a single node with 8 GPUs (NNODES=1, NGPUS_PER_NODE=8). The FSDP size must match the total number of GPUs allocated for the task, which is 8 in this case. With the current configuration, the training job will likely hang while waiting for 32 GPU resources that are not available.

Suggested change

	fsdp_size=32
	fsdp_size=$((NNODES * NGPUS_PER_NODE))

[gemini-code-assist]

The script contains hardcoded, user-specific absolute paths for the model and datasets. This limits the script's portability and reusability. These paths should be parameterized by using environment variables and failing with an informative message if they are not set.

Suggested change

	MODEL_PATH=/mnt/hdfs/zhangchi.usc1992_lf_lq/models/Qwen3-30B-A3B-Base
	CKPTS_DIR=${CKPTS_DIR:-"${RAY_DATA_HOME}/ckpts/${project_name}/${exp_name}"}
	TRAIN_FILE=/mnt/hdfs/zhangchi.usc1992_ssd_hldy/dataset/dapo-math-17k.parquet
	TEST_FILE=/mnt/hdfs/zhangchi.usc1992_ssd_hldy/dataset/aime-2024.parquet
	MODEL_PATH=${MODEL_PATH:?"MODEL_PATH must be set to the path of the base model."}
	CKPTS_DIR=${CKPTS_DIR:-${RAY_DATA_HOME}/ckpts/${project_name}/${exp_name}}
	TRAIN_FILE=${TRAIN_FILE:?"TRAIN_FILE must be set to the path of the training data."}
	TEST_FILE=${TEST_FILE:?"TEST_FILE must be set to the path of the test data."}

[vermouth1992]

/gemini review

[gemini-code-assist]

Code Review

This pull request adds a new script for DAPO training on a single node and updates the reward scoring logic. The script changes seem mostly correct for adapting the training to a smaller-scale environment. However, I've identified a critical issue in the distributed training configuration within the new script that could lead to runtime failures or incorrect behavior. The python change to support a new dataset appears correct.

[gemini-code-assist]

This parallelism configuration appears to be incorrect and could lead to runtime errors or silent performance issues. With a total of 8 GPUs for the node (NNODES=1, NGPUS_PER_NODE=8), and a sequence parallelism size (sp_size) of 4 (defined on line 49), the data parallelism degree (FSDP size) should be 2, such that dp_size * sp_size = world_size (i.e., 2 * 4 = 8).

Currently, fsdp_size is set to 8. If fsdp_size represents the data parallelism degree, the total required GPUs would be 8 * 4 = 32, which far exceeds the 8 available GPUs. This is a critical misconfiguration.

Assuming sp_size=4 is desired, fsdp_size should be changed to 2.

Suggested change

	fsdp_size=8
	fsdp_size=2

[vermouth1992]

This parallelism configuration appears to be incorrect and could lead to runtime errors or silent performance issues. With a total of 8 GPUs for the node (NNODES=1, NGPUS_PER_NODE=8), and a sequence parallelism size (sp_size) of 4 (defined on line 49), the data parallelism degree (FSDP size) should be 2, such that dp_size * sp_size = world_size (i.e., 2 * 4 = 8).

Currently, fsdp_size is set to 8. If fsdp_size represents the data parallelism degree, the total required GPUs would be 8 * 4 = 32, which far exceeds the 8 available GPUs. This is a critical misconfiguration.

Assuming sp_size=4 is desired, fsdp_size should be changed to 2.

This is incorrect