Skip to content
Merged
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
27 changes: 19 additions & 8 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,18 +1,29 @@
<p align="center">
<img src="assets/nvidia-cosmos-header.png" alt="NVIDIA Cosmos Header">
<span style="font-size: 18px;">New GitHub page for NVIDIA Cosmos: <b><a href="https://github.com/nvidia-cosmos">https://github.com/nvidia-cosmos</a></b></span>
</p>

Thank you all for the valuable feedback! We have restructured the codebase to make it easier to use and contribute to.
# Nvidia Cosmos

Cosmos World Foundation Models come in three model types which can all be customized in post-training: [cosmos-predict](https://github.com/nvidia-cosmos/cosmos-predict1), [cosmos-transfer](https://github.com/nvidia-cosmos/cosmos-transfer1), and [cosmos-reason](https://github.com/nvidia-cosmos/cosmos-reason1):

| | Predict | Transfer | Reason |
| ----- | :---: | :---: | :---: |
| **Type** | World Generation | Multi-Controlnet | Reasoning VLM |
| **Function** | Predict novel future frames given initial frames | Transfer existing control frames into photoreal frames within a video clip | Reason against frames within a video clip |
| **Use Cases** | Data Generation & Policy Evaluation | Data Augmentation | Data Curation |
| **Inputs** | Text, Image, Video | Multiple Video Modalities such as RGB, Depth, Segmentation, and more. | Video & Text |
| **Outputs** | Video | Video | Text |

#

# Use Cases in Physical AI Development

Our world foundation models are purpose-built to accelerate improving performance in downstream model tasks in various stages, as illustrated here in the flywheel.

<p align="center">
<span style="font-size: 18px;">New GitHub page for NVIDIA Cosmos: <b><a href="https://github.com/nvidia-cosmos">https://github.com/nvidia-cosmos</a></b></span>
<img src="https://raw.githubusercontent.com/nvidia-cosmos/cosmos-transfer1/main/assets/DataFlywheel.jpg" alt="NVIDIA Cosmos Data Flywheel">
</p>

NVIDIA Cosmos now includes three subprojects:

1. [Cosmos-Predict1](https://github.com/nvidia-cosmos/cosmos-predict1) is a collection of general-purpose world foundation models for Physical AI that can be fine-tuned into customized world models for downstream applications.
2. [Cosmos-Transfer1](https://github.com/nvidia-cosmos/cosmos-transfer1) is a world-to-world transfer model designed to bridge the perceptual divide between simulated and real-world environments.
3. [Cosmos-Reason1](https://github.com/nvidia-cosmos/cosmos-reason1) models understand the physical common sense and generate appropriate embodied decisions in natural language through long chain-of-thought reasoning processes.

-----------------------------------------------------------

Expand Down
Loading