No dropout in this version #672

Unanswered

Phil-U-U asked this question in Q&A

Phil-U-U
Jul 9, 2024

Hi all,

I noticed that for this version of implementing the transformer, we actually don't have any dropout layer, which was used in multiple layers as we can see in https://github.com/karpathy/nanoGPT/blob/master/model.py

I wonder is there any specific reason for that?

BTW, really like this community.

Cheers,
Phil

Replies: 0 comments

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment