General Comments

[MasonMarchetti]

Below are the comments I have compiled from reading through the paper. Note that I have very limited knowledge of the subject so many of the conceptual concerns may be due to a lack of technical knowledge. Nevertheless here are a my comments as a layperson:

p.4 - difference between cloze and prefix prompting, concepts of hard vs soft prompts, and idea of what constitutes task agnostic techniques not entirely clear to me

P.5 - MMLU benchmarking criteria is introduced with very little explanation, perhaps that would good for the reader to have bit of context (it is included in references) also note that references and acknowledgements sections are not included in table of contents

2.1.1 - meaning of masked frame / window

2.1.1 - text only mentions arXiv, yet graphic (and preceding paragraph) include that records are pulled from SS and ACL as well

I found figure 2.3 very helpful for understanding the parameters and characteristics of exemplars, perhaps make it more of a focal point

Figure 2.5 - 'letter' should be 'letters'

p.10 - difference between few shot prompting and few shot learning is a bit confusing

2.2.2 - paragraph wording

2.2.3.2 - 'present' should be 'presents'

2.4 - technique -> techniques

2.4 - concept of 'frozen LLM' a bit elusive to me

2.4 - in the RLprompt paragraph, the phrasing 'grammatical gibberish' might have a better alternative

2.4 - DP20 paragraph wording

2.5 - should be "variance response format"

3.2 - one case of the word "technique" needs to be made plural

4.0 - section 4 first paragraph, second sentence needs to be fixed

4.1.3 - what is meant by "toy environments"

5.1.1 - definition of prompt injection, are original developer instruction in the prompt of being overridden with the use of the prompt?

5.1.3 - technique -> techniques

6.1 - technique -> techniques

6.2.1 - signal -> signals

6.2.3 - explain concepts of precision and recall.

Overall I found it very informative and thorough but a bit challenging as a non-expert. There were many instances of words that were incorrectly not pluralized, specifically the word technique. My other general advice would be to give context for some more specific phrases and technical concepts e.g. MMLU, backprop, decoder-only models, or even more basic things like what an exemplar label is. some of these concepts get introduced somewhat abruptly and could discombobulate non technical readers. Food for thought if this paper is intended to be read by the lay-person

[trigaten]

Ty!