[NPU Roadmap] NPU support for Liger-Kernel

[zheliuyu]

Thanks very much to Liger-Kernel for accepting our first native support pr.
This roadmap details the future plans for NPU native support. Welcome to join in the discussion.

NPU Native Support

• [RFC] Native Ascend NPU Support for Liger Kernel #954
• Add Ascend NPU device support. #955
• [RFC] Code Organization for New Backend Support #965

Unit Test Coverage Improvement: Functionality & Precision

Ref: #1022

Performance Enhancement For High-Frequency APIs

Unit tests can be used to APIs functionality and precision. However, as third-party devices may not fully align in their usage patterns of Triton, a performance optimization process is required.

We selected three training software with varying dependencies on Liger-Kernel APIs. We will subsequently measure the actual performance of these APIs and may initiate pull requests to provide native support.

Software	Utils	APIs	Status
VeOmni	liger_kernel.transformers import	LigerFusedLinearCrossEntropyLoss	DOING, @zheliuyu
	liger_kernel.transformers.rms_norm import	LigerRMSNorm	DOING, @ji-huazhong
	liger_kernel.transformers.rope import	liger_rotary_pos_emb	DOING, @TianHao324
	liger_kernel.transformers.swiglu import	LigerSwiGLUMLP	DOING, @jiaqiw09
	liger_kernel.transformers.layer_norm import	LigerLayerNorm	DOING, @pillumina
	liger_kernel.transformers.qwen2vl_mrope import	liger_multimodal_rotary_pos_emb	DOING, @TianHao324
	liger_kernel.ops.swiglu import	LigerSiLUMulFunction	DOING, @UserChen666
LLaMA-Factory	liger_kernel.transformers import	apply_liger_kernel_to_gemma
	liger_kernel.transformers import	apply_liger_kernel_to_gemma2
	liger_kernel.transformers import	apply_liger_kernel_to_gemma3
	liger_kernel.transformers import	apply_liger_kernel_to_gemma3_text
	liger_kernel.transformers import	apply_liger_kernel_to_glm4
	liger_kernel.transformers import	apply_liger_kernel_to_glm4v
	liger_kernel.transformers import	apply_liger_kernel_to_granite
	liger_kernel.transformers import	apply_liger_kernel_to_llama
	liger_kernel.transformers import	apply_liger_kernel_to_llava
	liger_kernel.transformers import	apply_liger_kernel_to_mistral
	liger_kernel.transformers import	apply_liger_kernel_to_mixtral
	liger_kernel.transformers import	apply_liger_kernel_to_mllama
	liger_kernel.transformers import	apply_liger_kernel_to_olmo2
	liger_kernel.transformers import	apply_liger_kernel_to_paligemma
	liger_kernel.transformers import	apply_liger_kernel_to_phi3
	liger_kernel.transformers import	apply_liger_kernel_to_qwen2
	liger_kernel.transformers import	apply_liger_kernel_to_qwen2_vl
	liger_kernel.transformers import	apply_liger_kernel_to_qwen2_5_vl
	liger_kernel.transformers import	apply_liger_kernel_to_qwen3
	liger_kernel.transformers import	apply_liger_kernel_to_qwen3_moe
verl	liger_kernel.transformers.monkey_patch import	_apply_liger_kernel_to_instance

[Tcc0403]

Is there any repo with npu ci where contributors can test around?

[Ginray]

Is there any repo with npu ci where contributors can test around?

We’ve built NPU CI for LlamaFactory, swift and verl, and will verify liger kernel’s compatibility with these libraries in follow-up work. Additionally, we hope to set up NPU CI for the liger kernel library, and can provide NPU machines to support this effort. We’d like to connect with you to move this forward.

[momochen]

@Ginray hey there, we are excited to see community supports to NPU, aligned with our long term goal to support heterogenous accelerators. Let's setup a formal meeting sometime this week on making NPU CI available. @Tcc0403 let's find a mutually convenient time.

[zheliuyu]

Some new players have joined the game. ✨

• VeOmini
  • LigerRMSNorm @ji-huazhong
  • LigerSwiGLUMLP @jiaqiw09

[Tcc0403]

let's find a mutually convenient time.

@momochen Any time is fine. I can work around it.

[TianHao324]

I am working on matters related to liger_rotary_pos_emb.

[zheliuyu]

I am working on matters related to liger_rotary_pos_emb.

cooool! Let's get started.

[pillumina]

FYI:
I'll work on this one: LigerLayerNorm

[UserChen666]

FYI:
I'll work on this one: LigerSiLUMulFunction

[TianHao324]

FYI:
liger_rotary_pos_emb ：Done
I'll work on this one: liger_multimodal_rotary_pos_emb

[Tcc0403]

Is there a working group for NPU support? It would be great if we can see what folks are working on!

[jiaqiw09]

Is there a working group for NPU support? It would be great if we can see what folks are working on!

@Tcc0403
Would it be possible to create a dedicated channel in official Discord, so we can share technical details and coordinate more efficiently to help build the ecosystem?

[Tcc0403]

Would it be possible to create a dedicated channel in official Discord, so we can share technical details and coordinate more efficiently to help build the ecosystem?

We have a discord channel in GPU MODE server and slack channel for Liger Kernel. Both are official channels where we used to discuss things, but they aren't that active recently. If you mean a dedicated discrod server, we'll need to discuss internally how we can run it.

[jiaqiw09]

We have a discord channel in GPU MODE server and slack channel for Liger Kernel. Both are official channels where we used to discuss things, but they aren't that active recently. If you mean a dedicated discrod server, we'll need to discuss internally how we can run it.

Got it, thanks!

I was referring to adding a new channel under the existing "Computing Platforms" category (similar to #cuda or #rocm) in discord, rather than creating a new server or a new top-level category. Sorry for the ambiguous wording earlier.

For example, something like a #ascend or #npu channel would already work very well for us.

[Tcc0403]

Oh, gpu mode doesn't belong to liger kernel team, so that's not in our control. But gpu mode is really cool with oss stuffs, I believe it's more than welcome to have npu channel there. cc @msaroufim @S1ro1

Also, as you request, @momochen is going to create liger kernel discord server for better collaboration with oss community!