Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Plan for extra-large model support (OPT 175B, Bloom 176B) #46

Closed
ccmaymay opened this issue Aug 4, 2022 · 2 comments
Closed

Plan for extra-large model support (OPT 175B, Bloom 176B) #46

ccmaymay opened this issue Aug 4, 2022 · 2 comments
Assignees
@ccmaymay

Comments

@ccmaymay
Copy link
Collaborator

ccmaymay commented Aug 4, 2022

No description provided.

@ccmaymay
Copy link
Collaborator Author

ccmaymay commented Aug 4, 2022

@rekriz11 @ruyimarone what would it take to support inference on the large (8 x 80GB A100) OPT model? Do you know if that's possible with the public metaseq software? And do all GPUs still need to be on the same host, or can they be distributed?

@ccmaymay ccmaymay changed the title Look up supporting multiple machines (large OPT, BLOOM) Plan for extra-large model support (OPT 175B, Bloom 176B) Aug 4, 2022
@ccmaymay ccmaymay self-assigned this Aug 9, 2022
@ccmaymay
Copy link
Collaborator Author

ccmaymay commented Aug 9, 2022

See #47, #48

@ccmaymay ccmaymay closed this as completed Aug 9, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@ccmaymay ccmaymay

Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@ccmaymay