[Qwen3] StateDictAdapter support for MoE model

[shuhuayu]

Reused StateDictAdapter support for DeepSeek V3 model to implement Qwen3 StateDictAdapter. Updated a checkpoint loading api to support distributed huggingface checkpoint loading when unpicklable objects exist.

[wwwjn]

Thanks for working on this again! Can you attach a screenshot of your local run after loading HF weights?

[shuhuayu]

Thanks for working on this again! Can you attach a screenshot of your local run after loading HF weights?

Sure, here is a local run after loading weights from huggingface Qwen/Qwen3-30B-A3B-Instruct-2507 model:

[vwxyzjn]

@shuhuayu nice PR! The loss and grad norms looks a bit high though -- any idea why?

Another good way to validate the implementation is to run inference to check if you can get the same output tokens as the HF implementaiton.

[shuhuayu]

@shuhuayu nice PR! The loss and grad norms looks a bit high though -- any idea why?

Another good way to validate the implementation is to run inference to check if you can get the same output tokens as the HF implementaiton.

@vwxyzjn Thanks for the suggestion! I guess the large losses and gradient norms may be due to the suboptimal training configs we set for debug. We have verified the KL Divergence between a huggingface model and a converted torchtitan model on Qwen3 30B-A3B.

[tianyu-l]

LGTM
In the future, we should consider adding unit tests for MoEStateDictAdapter.