File tree Expand file tree Collapse file tree 2 files changed +4
-4
lines changed Expand file tree Collapse file tree 2 files changed +4
-4
lines changed Original file line number Diff line number Diff line change @@ -31,7 +31,7 @@ special `<|EndofText|>` token or runs out of context.
31
31
### Context Size
32
32
33
33
To predict the next token, Transformer-based large language models will _ attend_ (look at all
34
- previous tokens, the prompt) with a processus called _ Attention_ . This computationally expensive
34
+ previous tokens, the prompt) with a process called _ Attention_ . This computationally expensive
35
35
process imposes constraints on the maximum amount of text a model can operate with. This maximum
36
36
length is referred to as its ** context size** . Each large language model has a specific
37
37
context size, but it generally consists of 4000 tokens (~ 4000 words). Some have 2000, others
@@ -109,7 +109,7 @@ viable the approach described above.
109
109
110
110
Along with a well thought-out UI, improvements in instruction-following was one of the core advances
111
111
of ChatGPT, possibly explaining its rapid success. ChatGPT relied on a novel instruction-following
112
- fine-tuning paradigm called [ reinforcement learning from human feedback] ( # ) .
112
+ fine-tuning paradigm called [ reinforcement learning from human feedback] ( https://en.wikipedia.org/wiki/Reinforcement_learning_from_human_feedback ) .
113
113
114
114
### Fine-tuning
115
115
Original file line number Diff line number Diff line change @@ -130,7 +130,7 @@ Dust apps can also define datasets which are arrays of JSON objects. Datasets' m
130
130
- Store example inputs on which the app is run during its design. These datasets are pulled from
131
131
` input ` blocks.
132
132
- Store few-shot examples used when prompting models. These datasets are made available to ` llm `
133
- blocks through the use of ` data ` blocks (see [ core blocks] ( # ) ).
133
+ blocks through the use of ` data ` blocks (see [ core blocks] ( /core-blocks ) ).
134
134
135
135
All datasets are automatically versioned and each app version points to their specific dataset
136
136
version.
@@ -142,7 +142,7 @@ Each input's execution trace is completely independent and these cannot be cross
142
142
runs of an app are stored along with the app version and all the associated block outputs. They can
143
143
be retrieved from the ` Runs ` panel.
144
144
145
- Other [ core blocks] ( # ) allow the further parallelization of the execution such as the ` map ` and
145
+ Other [ core blocks] ( /core-blocks ) allow the further parallelization of the execution such as the ` map ` and
146
146
` reduce ` blocks. Dust's execution engine will also take care of automatically parallelizing
147
147
execution eagearly when they are used.
148
148
You can’t perform that action at this time.
0 commit comments