Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Upgrade transformers to the latest version #521

Closed
isaac091 opened this issue Sep 16, 2024 · 3 comments · Fixed by #583
Closed

Upgrade transformers to the latest version #521

isaac091 opened this issue Sep 16, 2024 · 3 comments · Fixed by #583
Labels
maintenance

Comments

@isaac091
Copy link
Collaborator

This is needed to be able to use SDPA for NLLB (#478), which should hopefully be released in the next minor version of transformers.

There may be others, but at least one breaking change that we will need to take care of is the refactoring of the NllbTokenizer, specifically in regards to the language code special tokens. This change came in 4.38, and I believe the main changes came in this PR.
@isaac091 isaac091 moved this from 🆕 New to 🔖 Ready in SIL-NLP Research Sep 16, 2024
@isaac091
Copy link
Collaborator Author

The changes should be coming in transformers 4.46

@TaperChipmunk32
Copy link
Collaborator

TaperChipmunk32 commented Oct 23, 2024 •
edited
Loading

It is also worth noting that in this pr for version 4.42.0; they removed the features that were deprecated in version 4.38.

@isaac091
Copy link
Collaborator Author

4.46 is released and it looks like the changes are in there

@TaperChipmunk32 TaperChipmunk32 mentioned this issue Nov 5, 2024
Closed
@ddaspit ddaspit mentioned this issue Nov 5, 2024
Merged
@TaperChipmunk32 TaperChipmunk32 linked a pull request Nov 7, 2024 that will close this issue
Remove OpenNMT and Tensorflow, Update to Python 3.10 #583
Merged
@github-project-automation github-project-automation bot moved this from 🔖 Ready to ✅ Done in SIL-NLP Research Nov 11, 2024
@TaperChipmunk32 TaperChipmunk32 mentioned this issue Nov 11, 2024
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
maintenance
Projects
SIL-NLP Research
Status: ✅ Done
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Remove OpenNMT and Tensorflow, Update to Python 3.10 sillsdev/silnlp
2 participants
@TaperChipmunk32 @isaac091