在训练nanoGPT时出现了以下报错,请问我该怎么做 #17525
Unanswered
1os3
asked this question in
code help: NLP / ASR / TTS
Replies: 0 comments
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Uh oh!
There was an error while loading. Please reload this page.
-
(base) PS E:\GPT-2\nanoGPT> python train.py config/train_shakespeare_char.py --compile=False --block_size=64
Overriding config with config/train_shakespeare_char.py:
train a miniature character-level shakespeare model
good for debugging and playing on macbooks and such
out_dir = 'out-shakespeare-char'
eval_interval = 250 # keep frequent because we'll overfit
eval_iters = 200
log_interval = 10 # don't print too too often
we expect to overfit on this small dataset, so only save when val improves
always_save_checkpoint = False
wandb_log = False # override via command line if you like
wandb_project = 'shakespeare-char'
wandb_run_name = 'mini-gpt'
dataset = 'shakespeare_char'
gradient_accumulation_steps = 1
batch_size = 64
block_size = 256 # context of up to 256 previous characters
baby GPT model :)
n_layer = 6
n_head = 6
n_embd = 384
dropout = 0.2
learning_rate = 1e-3 # with baby networks can afford to go a bit higher
max_iters = 5000
lr_decay_iters = 5000 # make equal to max_iters usually
min_lr = 1e-4 # learning_rate / 10 usually
beta2 = 0.99 # make a bit bigger because number of tokens per iter is small
warmup_iters = 100 # not super necessary potentially
on macbook also add
device = 'cpu' # run on cpu only
compile = False # do not torch compile the model
Overriding: compile = False
Overriding: block_size = 64
tokens per iteration will be: 4,096
Traceback (most recent call last):
File "E:\GPT-2\nanoGPT\train.py", line 110, in
ctx = nullcontext() if device_type == 'cpu' else torch.amp.autocast(device_type=device_type, dtype=ptdtype)
File "D:\PSAutoRecover\lib\site-packages\torch\amp\autocast_mode.py", line 234, in init
raise RuntimeError('Current CUDA Device does not support bfloat16. Please switch dtype to float16.')
RuntimeError: Current CUDA Device does not support bfloat16. Please switch dtype to float16.
最后一行好像是错误
Uploading nanoGPT…
Beta Was this translation helpful? Give feedback.
All reactions