multi-word-tokens

[Shasetty]

Hi Sir/Madam.
Shankar from Bangalore.

From 4.0.0 version of corenlp : issues in multi-word-tokens were solved.

Can you please inform me?
Stanford lexparser (englishPCFG.ser.gz) belong to which date or beyond which date, are capable to handle multi-word-token (issues).

Thanks in advance

[AngledLuffa]

I don't believe this actually applies to the English CoreNLP. The multiword support is for French and Spanish. FWIW, there has been some discussion of including MWT support in Stanza (our Python package) for English. The datasets mostly support it, but it might be a weird API change for users who don't expect it. The idea would be that words such as "won't", which are already separated into two words, would now be marked as part of the same token.

…

On Sat, Jun 10, 2023 at 2:01 AM Shasetty ***@***.***> wrote: Hi Sir/Madam. Shankar from Bangalore. From 4.0.0 version of corenlp : issues in multi-word-tokens were solved. Can you please inform me? Stanford lexparser (englishPCFG.ser.gz) belong to which date or beyond which date, are capable to handle multi-word-token (issues). Thanks in advance — Reply to this email directly, view it on GitHub <#1364>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AA2AYWISM2BLT5IF3ZATWIDXKQZVNANCNFSM6AAAAAAZBRWMKA> . You are receiving this because you are subscribed to this thread.Message ID: ***@***.***>

[Shasetty]

Thank You for the information