[RFC] An error occurred when running pytest tests/test_nsa.py in the native-sparse-attention folder

[PXHnb666]

Proposal

Hello, I want to apply your NSA attention mechanism on my local large model. It appears when I run tests/test_nsa.py

"ERROR tests/ test_nsa.py-ValueError: 'bitnet' is already used by a Transformers config, pick another name"

Is there any good way to solve it?

Rationale

No response

[haok1402]

I tried to search and replace all BitNet with WitNet, (i.e. anything that doesn't conflict with HuggingFace config), and it can resolve this issue. But, another orthogonal issue occurs with the test script, as mentioned in #31

[lijiazheng99]

I ran into the same collision with transformers, where bitnet is already registered in CONFIG_MAPPING. A minimal fix is to allow duplicate registration in fla by setting exist_ok=True for the BitNet entries:

# fla/models/bitnet/__init__.py
...
AutoConfig.register(BitNetConfig.model_type, BitNetConfig, exist_ok=True)
AutoModel.register(BitNetConfig, BitNetModel, exist_ok=True)
AutoModelForCausalLM.register(BitNetConfig, BitNetForCausalLM, exist_ok=True)
...

[haok1402]

@lijiazheng99 that is elegant!