Skip to content

Commit af66d6f

Browse files
committed
Rust - Bump to 0.10.1 for release
1 parent f9c76b6 commit af66d6f

File tree

2 files changed

+6
-1
lines changed

2 files changed

+6
-1
lines changed

tokenizers/CHANGELOG.md

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -4,6 +4,11 @@ All notable changes to this project will be documented in this file.
44
The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
55
and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
66

7+
## [0.10.1]
8+
9+
### Fixed
10+
- [#226]: Fix the word indexes when there are special tokens
11+
712
## [0.10.0]
813

914
### Changed

tokenizers/Cargo.toml

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -2,7 +2,7 @@
22
authors = ["Anthony MOI <m.anthony.moi@gmail.com>"]
33
edition = "2018"
44
name = "tokenizers"
5-
version = "0.10.0"
5+
version = "0.10.1"
66
homepage = "https://github.com/huggingface/tokenizers"
77
repository = "https://github.com/huggingface/tokenizers"
88
documentation = "https://docs.rs/tokenizers/"

0 commit comments

Comments
 (0)