Enable accented chars & word joiner char handling

sanjayaksaxena released this 12 Apr 15:51

· 10 commits to master since this release

452c990

Version 1.7.0 April 12, 2024

✨ Features

Accented characters now also include diacritical marks. 🙌
Token containing a word joiner is treated as a single token. ✅
Shape of token containing accented chars, diacritical marks & word joiner char(s) is now determined after removing them. 👏

Assets 2