File tree Expand file tree Collapse file tree 1 file changed +5
-5
lines changed Expand file tree Collapse file tree 1 file changed +5
-5
lines changed Original file line number Diff line number Diff line change 2
2
ZeroBand is a production ready codebase for decentralized training of LLM
3
3
4
4
5
- ## developlment
5
+ ## Developlment
6
6
7
7
install uv
8
8
@@ -40,22 +40,22 @@ run your code using
40
40
uv run ...
41
41
```
42
42
43
- ## quick check
43
+ ## Quick check
44
44
45
45
To check that everything is working you can do
46
46
47
47
``` bash
48
- ZERO_BAND_LOG_LEVEL=DEBUG torchrun --nproc_per_node=2 src/zeroband/train.py @configs/debug/normal.toml
48
+ ZERO_BAND_LOG_LEVEL=DEBUG torchrun --nproc_per_node=2 src/zeroband/train.py @configs/debug/normal.toml
49
49
```
50
50
51
- ## run diloco
51
+ ## Run diloco
52
52
53
53
To run diloco locally you can use the helper script ` scripts/simulatsimulate_multi_nodee_mutl.sh `
54
54
55
55
:note: you need 4 gpus to run the following command
56
56
57
57
``` bash
58
- ZERO_BAND_LOG_LEVEL=DEBUG ./scripts/simulate_multi_node .sh 2 2 src/zeroband/train.py @configs/debug/diloco.toml
58
+ ZERO_BAND_LOG_LEVEL=DEBUG ./scripts/simulate_multi_node_diloco .sh 2 2 src/zeroband/train.py @configs/debug/diloco.toml
59
59
```
60
60
61
61
if you have only two gpus
You can’t perform that action at this time.
0 commit comments