Skip to content

Latest commit

 

History

History
45 lines (35 loc) · 2.39 KB

vicuna_weights_version.md

File metadata and controls

45 lines (35 loc) · 2.39 KB
Raw

Vicuna-7B/13B

Weights version v1.1 v0
Link 7B, 13B 7B, 13B
Separator </s> ###
FastChat PyPI package compatibility >= v0.2.1 <= v0.1.10
FastChat source code compatibility after tag v0.2.1 tag v0.1.10

Major updates of weights v1.1

  • Refactor the tokenization and separator. In Vicuna v1.1, the separator has been changed from ### to the EOS token </s>. This change makes it easier to determine the generation stop criteria and enables better compatibility with other libraries.
  • Fix the supervised fine-tuning loss computation for better model quality.

Example prompt (Weight v1.1)

A chat between a user and an assistant.

USER: Hello!
ASSISTANT: Hello!</s>
USER: How are you?
ASSISTANT: I am good.</s>

See a full prompt template here.

Example prompt (Weight v0)

A chat between a human and an assistant.

### Human: Hello!
### Assistant: Hello!
### Human: How are you?
### Assistant: I am good.

See the full prompt template here.

Tokenizer issues

There are some frequently asked tokenizer issues (lm-sys#408). Some of them are not only related to FastChat or Vicuna weights but are also related to how you convert the base llama model.

We suggest that you use transformers>=4.28.0 and redo the weight conversion for the base llama model. After applying the delta, you should have a file named special_tokens_map.json in your converted weight folder for either v0 or v1.1. The contents of this file should be the same as this file: https://huggingface.co/lmsys/vicuna-13b-delta-v0/blob/main/special_tokens_map.json. If the file is not present, please copy the special_tokens_map.json and tokenizer_config.json files from https://huggingface.co/lmsys/vicuna-13b-delta-v0/tree/main to your converted weight folder. This works for both v0 and v1.1.