You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: docs/tutorial/tutorial-basics/basic-types.md
+2-1Lines changed: 2 additions & 1 deletion
Original file line number
Diff line number
Diff line change
@@ -87,7 +87,8 @@ This print-out includes the token index (3) and the lexical value of the token (
87
87
When you create a [`Sentence`](#flair.data.Sentence) as above, the text is automatically tokenized (segmented into words) using the [segtok](https://pypi.org/project/segtok/) library.
88
88
89
89
```{note}
90
-
You can also use a different tokenizer if you like. To learn more about this, check out our tokenization tutorial.
90
+
You can also use a different tokenizer by passing a different [`Tokenizer`](#flair.tokenization.Tokenizer ) to the Sentence
0 commit comments