Skip to content

rm non_word_boundaries#98

Open
Yvette-Wang wants to merge 29 commits intovi3k6i5:masterfrom
smoothnlp:haizhe
Open

rm non_word_boundaries#98
Yvette-Wang wants to merge 29 commits intovi3k6i5:masterfrom
smoothnlp:haizhe

Conversation

@Yvette-Wang
Copy link
Copy Markdown

No description provided.

vi3k6i5 and others added 29 commits November 10, 2017 20:47
added reference to flashtext paper
  `charactes` | `characters`
  `explaination` | `explanation`
  `matche` | `match`
Fix issue with incomplete keyword at the end of the sentence
Performances improvement for strings manipulations
@iwpnd
Copy link
Copy Markdown

iwpnd commented Nov 25, 2019

I don't think this helps flashtexts intended use. Removing word boundaries entirely will only result in "words found in words" kind of results. As seen here:

from flashtext import KeywordProcessor
kp = KeywordProcessor()
kp.add_keywords_from_list(['cat', 'catch'])
kp.non_word_boundaries = "_"

text = 'Try to catch this.'
kp.extract_keywords(text)
>> ['cat']

Since flashtext stops at the first hit, it won't even find "catch".

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

6 participants