Skip to content

Commit

Permalink
Parameter count in summary table (#773)
Browse files Browse the repository at this point in the history
* draft parameter count

* added disclaimer

* added missing param counts
  • Loading branch information
hiwotadese authored Jan 6, 2025
1 parent 1c1c619 commit 4733379
Showing 1 changed file with 11 additions and 9 deletions.
20 changes: 11 additions & 9 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -45,15 +45,17 @@ Some these benchmarks are rather slow or take a long time to run on the referenc
# MLPerf Training v4.1 (Submission Deadline Oct 11, 2024)
*Framework here is given for the reference implementation. Submitters are free to use their own frameworks to run the benchmark.

| model | reference implementation | framework | dataset
| ---- | ---- | ---- | ---- |
| RetinaNet | [vision/object detection](https://github.com/mlcommons/training/tree/master/single_stage_detector) | pytorch | OpenImages
| Stable Diffusionv2 | [image generation](https://github.com/mlcommons/training/tree/master/stable_diffusion) | pytorch | LAION-400M-filtered
| BERT-large | [language/nlp](https://github.com/mlcommons/training/tree/master/language_model/tensorflow/bert) | tensorflow | Wikipedia 2020/01/01
| GPT3 | [language/llm](https://github.com/mlcommons/training/tree/master/large_language_model) | paxml,megatron-lm | C4
| LLama2 70B-LoRA | [language/LLM fine-tuning](https://github.com/mlcommons/training/tree/master/llama2_70b_lora) | pytorch | SCROLLS GovReport
| DLRMv2 | [recommendation](https://github.com/mlcommons/training/tree/master/recommendation_v2/torchrec_dlrm) | torchrec | Criteo 3.5TB multi-hot
| RGAT | [GNN](https://github.com/mlcommons/training/tree/master/graph_neural_network) | pytorch | IGBFull
| model | reference implementation | framework | dataset | model parameter count
| ---- | ---- | ---- | ---- | ----
| RetinaNet | [vision/object detection](https://github.com/mlcommons/training/tree/master/single_stage_detector) | pytorch | OpenImages | 37M
| Stable Diffusionv2 | [image generation](https://github.com/mlcommons/training/tree/master/stable_diffusion) | pytorch | LAION-400M-filtered | 865M
| BERT-large | [language/nlp](https://github.com/mlcommons/training/tree/master/language_model/tensorflow/bert) | tensorflow | Wikipedia 2020/01/01 | 340M
| GPT3 | [language/llm](https://github.com/mlcommons/training/tree/master/large_language_model) | paxml,megatron-lm | C4 | 175B
| LLama2 70B-LoRA | [language/LLM fine-tuning](https://github.com/mlcommons/training/tree/master/llama2_70b_lora) | pytorch | SCROLLS GovReport | 70B
| DLRMv2 | [recommendation](https://github.com/mlcommons/training/tree/master/recommendation_v2/torchrec_dlrm) | torchrec | Criteo 3.5TB multi-hot | 167M
| RGAT | [GNN](https://github.com/mlcommons/training/tree/master/graph_neural_network) | pytorch | IGBFull | 25M

*Note model parameter count is not the same as active parameter that are being trained in the benchmark.

# MLPerf Training v4.0 (Submission Deadline May 10, 2024)
*Framework here is given for the reference implementation. Submitters are free to use their own frameworks to run the benchmark.
Expand Down

0 comments on commit 4733379

Please sign in to comment.