LaMDA: Large Model Fine-Tuning via Spectrally Decomposed Low-Dimensional Adaptation （2024.06.18）
Seyedarmin Azizi, Souvik Kundu, M. Pedram
The Impact of Initialization on LoRA Finetuning Dynamics （2024.06.12）
Soufiane Hayou, Nikhil Ghosh, Bin Yu
An Empirical Study on Parameter-Efficient Fine-Tuning for MultiModal Large Language Models （2024.06.07）
Xiongtao Zhou, Jie He, Yuhua Ke, Guangyao Zhu, V'ictor Guti'errez-Basulto, etc
Leveraging Visual Tokens for Extended Text Contexts in Multi-Modal Learning （2024.06.04）
Alex Jinpeng Wang, Linjie Li, Yiqi Lin, Min Li, Lijuan Wang, etc
Learning to grok: Emergence of in-context learning and skill composition in modular arithmetic tasks （2024.06.04）
Tianyu He, Darshil Doshi, Aritra Das, Andrey Gromov
Long Context is Not Long at All: A Prospector of Long-Dependency Data for Large Language Models （2024.05.28）
Longze Chen, Ziqiang Liu, Wanwei He, Yunshui Li, Run Luo, etc . - 【arXiv.org】
Efficient Prompt Tuning by Multi-Space Projection and Prompt Fusion （2024.05.19）
Pengxiang Lan, Enneng Yang, Yuting Liu, Guibing Guo, Linying Jiang, etc . - 【arXiv.org】
MAML-en-LLM: Model Agnostic Meta-Training of LLMs for Improved In-Context Learning （2024.05.19）
Sanchit Sinha, Yuguang Yue, Victor Soto, Mayank Kulkarni, Jianhua Lu, etc . - 【arXiv.org】
Improving Diversity of Commonsense Generation by Large Language Models via In-Context Learning （2024.04.25）
Tianhui Zhang, Bei Peng, D. Bollegala . - 【arXiv.org】
Stronger Random Baselines for In-Context Learning （2024.04.19）
Gregory Yauney, David M. Mimno . - 【arXiv.org】
Qinyuan Wu, Mohammad Aflah Khan, Soumi Das, Vedant Nanda, Bishwamittra Ghosh, etc . - 【arXiv.org】
Point-In-Context: Understanding Point Cloud via In-Context Learning （2024.04.18）
Mengyuan Liu, Zhongbin Fang, Xia Li, Joachim Buhmann, Xiangtai Li, etc . - 【arXiv.org】
AFLoRA: Adaptive Freezing of Low Rank Adaptation in Parameter Efficient Fine-Tuning of Large Models （2024.03.20）
Zeyu Liu, Souvik Kundu, Anni Li, Junrui Wan, Lianghao Jiang, etc
Towards Multimodal In-Context Learning for Vision & Language Models （2024.03.19）
Sivan Doveh, Shaked Perek, M. J. Mirza, Amit Alfassy, Assaf Arbelle, etc . - 【arXiv.org】
ExploRLLM: Guiding Exploration in Reinforcement Learning with Large Language Models （2024.03.14）
Runyu Ma, Jelle Luijkx, Zlatan Ajanovic, Jens Kober
Attention Prompt Tuning: Parameter-efficient Adaptation of Pre-trained Models for Spatiotemporal Modeling （2024.03.11）
W. G. C. Bandara, Vishal M. Patel
Bias-Augmented Consistency Training Reduces Biased Reasoning in Chain-of-Thought （2024.03.08）
James Chua, Edward Rees, Hunar Batra, Samuel R. Bowman, Julian Michael, etc
Securing Reliability: A Brief Overview on Enhancing In-Context Learning for Foundation Models （2024.02.27）
Yunpeng Huang, Yaonan Gu, Jingwei Xu, Zhihong Zhu, Zhaorun Chen, etc
GISTEmbed: Guided In-sample Selection of Training Negatives for Text Embedding Fine-tuning （2024.02.26）
Aivin V. Solatorio . - 【arXiv.org】
DiffuCOMET: Contextual Commonsense Knowledge Diffusion （2024.02.26）
Silin Gao, Mete Ismayilzada, Mengjie Zhao, Hiromi Wakaki, Yuki Mitsufuji, etc
Long-Context Language Modeling with Parallel Context Encoding （2024.02.26）
Howard Yen, Tianyu Gao, Danqi Chen
Training Nonlinear Transformers for Efficient In-Context Learning: A Theoretical Learning and Generalization Analysis （2024.02.23）
Hongkang Li, Meng Wang, Songtao Lu, Xiaodong Cui, Pin-Yu Chen
Addressing Order Sensitivity of In-Context Demonstration Examples in Causal Language Models （2024.02.23）
Yanzheng Xiang, Hanqi Yan, Lin Gui, Yulan He
In-Context Learning of a Linear Transformer Block: Benefits of the MLP Component and One-Step GD Initialization （2024.02.22）
Ruiqi Zhang, Jingfeng Wu, Peter L. Bartlett
Unlocking Instructive In-Context Learning with Tabular Prompting for Relational Triple Extraction （2024.02.21）
Guozheng Li, Wenjun Ke, Peng Wang, Zijie Xu, Ke Ji, etc
Smaug: Fixing Failure Modes of Preference Optimisation with DPO-Positive （2024.02.20）
Arka Pal, Deep Karkhanis, Samuel Dooley, Manley Roberts, Siddartha Naidu, etc
Feedback Loops With Language Models Drive In-Context Reward Hacking （2024.02.09）
Alexander Pan, Erik Jones, Meena Jagadeesan, Jacob Steinhardt . - 【arXiv.org】
On the Convergence of Zeroth-Order Federated Tuning in Large Language Models （2024.02.08）
Zhenqing Ling, Daoyuan Chen, Liuyi Yao, Yaliang Li, Ying Shen . - 【arXiv.org】
EmojiCrypt: Prompt Encryption for Secure Communication with Large Language Models （2024.02.08）
Guo Lin, Wenyue Hua, Yongfeng Zhang . - 【arXiv.org】
Large Language Model Meets Graph Neural Network in Knowledge Distillation （2024.02.08）
Shengxiang Hu, Guobing Zou, Song Yang, Yanglan Gan, Bofeng Zhang, etc . - 【arXiv.org】
G-LLaVA: Solving Geometric Problem with Multi-Modal Large Language Model （2023.12.18）
Jiahui Gao, Renjie Pi, Jipeng Zhang, Jiacheng Ye, Wanjun Zhong, etc
A mathematical perspective on Transformers （2023.12.17）
Borjan Geshkovski, Cyril Letrouit, Yury Polyanskiy, Philippe Rigollet
Mitigating Label Bias in Machine Learning: Fairness through Confident Learning （2023.12.14）
Yixuan Zhang, Boyu Li, Zenan Ling, Feng Zhou . - 【arXiv.org】
Control Risk for Potential Misuse of Artificial Intelligence in Science （2023.12.11）
Jiyan He, Weitao Feng, Yaosen Min, Jingwei Yi, Kunsheng Tang, etc
WonderJourney: Going from Anywhere to Everywhere （2023.12.06）
Hong-Xing Yu, Haoyi Duan, Junhwa Hur, Kyle Sargent, Michael Rubinstein, etc
Minimizing Factual Inconsistency and Hallucination in Large Language Models （2023.11.23）
I. Muneeswaran, Shreya Saxena, Siva Prasad, M. V. S. Prakash, Advaith Shankar, etc . - 【arXiv.org】
Igniting Language Intelligence: The Hitchhiker's Guide From Chain-of-Thought Reasoning to Language Agents （2023.11.20）
Zhuosheng Zhang, Yao Yao, Aston Zhang, Xiangru Tang, Xinbei Ma, etc . - 【arXiv.org】
An Embodied Generalist Agent in 3D World （2023.11.18）
Jiangyong Huang, Silong Yong, Xiaojian Ma, Xiongkun Linghu, Puhao Li, etc . - 【arXiv.org】
MedAgents: Large Language Models as Collaborators for Zero-shot Medical Reasoning （2023.11.16）
Xiangru Tang, Anni Zou, Zhuosheng Zhang, Yilun Zhao, Xingyao Zhang, etc . - 【arXiv.org】
Towards Verifiable Text Generation with Symbolic References （2023.11.15）
Lucas Torroba Hennigen, Zejiang Shen, Aniruddha Nrusimha, Bernhard Gapp, David Sontag, etc . - 【arXiv.org】
Learning skillful medium-range global weather forecasting. （2023.11.14）
Remi Lam, Alvaro Sanchez-Gonzalez, Matthew Willson, Peter Wirnsberger, Meire Fortunato, etc . - 【Science】
u-LLaVA: Unifying Multi-Modal Tasks via Large Language Model （2023.11.09）
Jinjin Xu, Liwu Xu, Yuzhe Yang, Xiang Li, Yanchun Xie, etc . - 【arXiv.org】
Levels of AGI: Operationalizing Progress on the Path to AGI （2023.11.04）
Meredith Ringel Morris, Jascha Narain Sohl-Dickstein, Noah Fiedel, T. Warkentin, Allan Dafoe, etc . - 【arXiv.org】
Which Examples to Annotate for In-Context Learning? Towards Effective and Efficient Selection （2023.10.30）
Costas Mavromatis, Balasubramaniam Srinivasan, Zhengyuan Shen, Jiani Zhang, H. Rangwala, etc . - 【arXiv.org】
CodeFusion: A Pre-trained Diffusion Model for Code Generation （2023.10.26）
Mukul Singh, J. Cambronero, Sumit Gulwani, Vu Le, Carina Negreanu, etc
SuperHF: Supervised Iterative Learning from Human Feedback （2023.10.25）
Gabriel Mukobi, Peter Chatain, Su Fong, Robert Windesheim, Gitta Kutyniok, etc
In-Context Learning Creates Task Vectors （2023.10.24）
Roee Hendel, Mor Geva, Amir Globerson . - 【Conference on Empirical Methods in Natural Language Processing】
Woodpecker: Hallucination Correction for Multimodal Large Language Models （2023.10.24）
Shukang Yin, Chaoyou Fu, Sirui Zhao, Tong Xu, Hao Wang, etc
In-context Learning with Transformer Is Really Equivalent to a Contrastive Learning Pattern （2023.10.20）
Ruifeng Ren, Yong Liu . - 【arXiv.org】
MemGPT: Towards LLMs as Operating Systems （2023.10.12）
Charles Packer, Vivian Fang, Shishir G. Patil, Kevin Lin, Sarah Wooders, etc
LongLoRA: Efficient Fine-tuning of Long-Context Large Language Models （2023.09.21）
Yukang Chen, Shengju Qian, Haotian Tang, Xin Lai, Zhijian Liu, etc . - 【arXiv.org】
Adapting Large Language Models via Reading Comprehension （2023.09.18）
Daixuan Cheng, Shaohan Huang, Furu Wei . - 【arXiv.org】
Giraffe: Adventures in Expanding Context Lengths in LLMs （2023.08.21）
Arka Pal, Deep Karkhanis, Manley Roberts, S. Dooley, Arvind Sundararajan, etc . - 【arXiv.org】
Prompt Switch: Efficient CLIP Adaptation for Text-Video Retrieval （2023.08.15）
Chaorui Deng, Qi Chen, Pengda Qin, Dave Zhenyu Chen, Qi Wu . - 【arXiv.org】
Exploring the Intersection of Large Language Models and Agent-Based Modeling via Prompt Engineering （2023.08.14）
Edward Junprung . - 【arXiv.org】
PromptCARE: Prompt Copyright Protection by Watermark Injection and Verification （2023.08.05）
Hongwei Yao, Jian Lou, Kui Ren, Zhan Qin . - 【arXiv.org】
Learning to Retrieve In-Context Examples for Large Language Models （2023.07.14）
Liang Wang, Nan Yang, Furu Wei
Brain in a Vat: On Missing Pieces Towards Artificial General Intelligence in Large Language Models （2023.07.07）
Yuxi Ma, Chi Zhang, Song-Chun Zhu . - 【arXiv.org】
Understanding In-Context Learning via Supportive Pretraining Data （2023.06.26）
Xiaochuang Han, Daniel Simig, Todor Mihaylov, Yulia Tsvetkov, Asli Celikyilmaz, etc . - 【Annual Meeting of the Association for Computational Linguistics】
Schema-learning and rebinding as mechanisms of in-context learning and emergence （2023.06.16）
Siva K. Swaminathan, A. Dedieu, Rajkumar Vasudeva Raju, M. Shanahan, M. Lázaro-Gredilla, etc . - 【arXiv.org】
MetaVL: Transferring In-Context Learning Ability From Language Models to Vision-Language Models （2023.06.02）
Masoud Monajatipoor, Liunian Harold Li, Mozhdeh Rouhsedaghat, Lin F. Yang, Kai-Wei Chang . - 【arXiv.org】
Measuring and Mitigating Constraint Violations of In-Context Learning for Utterance-to-API Semantic Parsing （2023.05.24）
Shufan Wang, Sebastien Jean, Sailik Sengupta, James Gung, Nikolaos Pappas, etc
OverPrompt: Enhancing ChatGPT Capabilities through an Efficient In-Context Learning Approach （2023.05.24）
Jiazheng Li, Runcong Zhao, Yulan He, Lin Gui
Self-ICL: Zero-Shot In-Context Learning with Self-Generated Demonstrations （2023.05.24）
Wei-Lin Chen, Cheng-Kuang Wu, Hsin-Hsi Chen . - 【Conference on Empirical Methods in Natural Language Processing】
Adversarial Demonstration Attacks on Large Language Models （2023.05.24）
Jiongxiao Wang, Zichen Liu, Keun Hee Park, Muhao Chen, Chaowei Xiao
Frugal Prompting for Dialog Models （2023.05.24）
Bishal Santra, Sakya Basak, Abhinandan De, Manish Gupta, Pawan Goyal
Coverage-based Example Selection for In-Context Learning （2023.05.24）
Shivanshu Gupta, Sameer Singh, Matt Gardner
SummIt: Iterative Text Summarization via ChatGPT （2023.05.24）
Haopeng Zhang, Xiao Liu, Jiawei Zhang
Exploring Diverse In-Context Configurations for Image Captioning （2023.05.24）
Xu Yang, Yongliang Wu, Mingzhuo Yang, Haokun Chen
Mastering the ABCDs of Complex Questions: Answer-Based Claim Decomposition for Fine-grained Self-Evaluation （2023.05.24）
Nishant Balepur, Jie Huang, Samraj Moorjani, Hari Sundaram, Kevin Chen-Chuan Chang
In-Context Demonstration Selection with Cross Entropy Difference （2023.05.24）
Dan Iter, Reid Pryzant, Ruochen Xu, Shuohang Wang, Yang Liu, etc
ExpertPrompting: Instructing Large Language Models to be Distinguished Experts （2023.05.24）
Benfeng Xu, An Yang, Junyang Lin, Quang Wang, Chang Zhou, etc
Active Learning Principles for In-Context Learning with Large Language Models （2023.05.23）
Katerina Margatina, Timo Schick, Nikolaos Aletras, Jane Dwivedi-Yu
Skill-Based Few-Shot Selection for In-Context Learning （2023.05.23）
Shengnan An, Bo Zhou, Zeqi Lin, Qiang Fu, Bei Chen, etc
Label Words are Anchors: An Information Flow Perspective for Understanding In-Context Learning （2023.05.23）
Lean Wang, Lei Li, Damai Dai, Deli Chen, Hao Zhou, etc
Make a Choice! Knowledge Base Question Answering with In-Context Learning （2023.05.23）
Chuanyuan Tan, Yuehe Chen, Wenbiao Shao, Wenliang Chen
Concept-aware Training Improves In-context Learning Ability of Language Models （2023.05.23）
Michal vStef'anik, Marek Kadlvc'ik
RetICL: Sequential Retrieval of In-Context Examples with Reinforcement Learning （2023.05.23）
Alexander Scarlatos, Andrew Lan
Can We Edit Factual Knowledge by In-Context Learning? （2023.05.22）
Ce Zheng, Lei Li, Qingxiu Dong, Yuxuan Fan, Zhiyong Wu, etc
Iterative Forward Tuning Boosts In-context Learning in Language Models （2023.05.22）
Jiaxi Yang, Binyuan Hui, Min Yang, Binhua Li, Fei Huang, etc
Explaining How Transformers Use Context to Build Predictions （2023.05.21）
Javier Ferrando, Gerard I. Gállego, Ioannis Tsiamas, M. Costa-jussà
Few-Shot Dialogue Summarization via Skeleton-Assisted Prompt Transfer （2023.05.20）
Kaige Xie, Tong Yu, Haoliang Wang, Junda Wu, Handong Zhao, etc
Logic-LM: Empowering Large Language Models with Symbolic Solvers for Faithful Logical Reasoning （2023.05.20）
Liangming Pan, Alon Albalak, Xinyi Wang, William Yang Wang
RCOT: Detecting and Rectifying Factual Inconsistency in Reasoning by Reversing Chain-of-Thought （2023.05.19）
Tianci Xue, Ziqi Wang, Zhenhailong Wang, Chi Han, Pengfei Yu, etc . - 【arXiv.org】
Do Models Really Learn to Follow Instructions? An Empirical Study of Instruction Tuning （2023.05.19）
Po-Nien Kung, Nanyun Peng . - 【arXiv.org】
AutoTrial: Prompting Language Models for Clinical Trial Design （2023.05.19）
Zifeng Wang, Cao Xiao, Jimeng Sun . - 【arXiv.org】
Efficient Prompting via Dynamic In-Context Learning （2023.05.18）
Wangchunshu Zhou, Yuchen Jiang, Ryan Cotterell, Mrinmaya Sachan . - 【arXiv.org】
Generalized Planning in PDDL Domains with Pretrained Large Language Models （2023.05.18）
Tom Silver, Soham Dan, Kavitha Srinivas, J. Tenenbaum, L. Kaelbling, etc . - 【arXiv.org】
Discriminative Diffusion Models as Few-shot Vision and Language Learners （2023.05.18）
Xuehai He, Weixi Feng, Tsu-Jui Fu, Varun Jampani, Arjun Reddy Akula, etc . - 【arXiv.org】
Temporal Knowledge Graph Forecasting Without Knowledge Using In-Context Learning （2023.05.17）
Dong-Ho Lee, Kian Ahrabian, Woojeong Jin, Fred Morstatter, J. Pujara . - 【arXiv.org】
What In-Context Learning "Learns" In-Context: Disentangling Task Recognition and Task Learning （2023.05.16）
Jane Pan, Tianyu Gao, Howard Chen, Danqi Chen . - 【arXiv.org】
Enriching language models with graph-based context information to better understand textual data （2023.05.10）
Albert Roethel, M. Ganzha, Anna Wr'oblewska . - 【arXiv.org】
Why Johnny Can’t Prompt: How Non-AI Experts Try (and Fail) to Design LLM Prompts （2023.04.19）
J. Zamfirescu-Pereira, Richmond Y. Wong, Bjoern Hartmann, Qiang Yang . - 【International Conference on Human Factors in Computing Systems】
SPDF: Sparse Pre-training and Dense Fine-tuning for Large Language Models （2023.03.18）
Vithursan Thangarasa, Abhay Gupta, William Marshall, Tianda Li, Kevin Leong, etc . - 【Conference on Uncertainty in Artificial Intelligence】
Larger language models do in-context learning differently （2023.03.07）
Jerry W. Wei, Jason Wei, Yi Tay, Dustin Tran, Albert Webson, etc . - 【ArXiv】
Language Model Crossover: Variation through Few-Shot Prompting （2023.02.23）
Elliot Meyerson, M. Nelson, Herbie Bradley, Arash Moradi, Amy K. Hoover, etc . - 【ArXiv】
How Does In-Context Learning Help Prompt Tuning? （2023.02.22）
Simeng Sun, Yang Liu, Dan Iter, Chenguang Zhu, Mohit Iyyer . - 【ArXiv】
Compositional Exemplars for In-context Learning （2023.02.11）
Jiacheng Ye, Zhiyong Wu, Jiangtao Feng, Tao Yu, Lingpeng Kong . - 【International Conference on Machine Learning】
PLACES: Prompting Language Models for Social Conversation Synthesis （2023.02.07）
Maximillian Chen, A. Papangelis, Chenyang Tao, Seokhwan Kim, Andrew Rosenbaum, etc . - 【ArXiv】
Large Language Models Are Implicitly Topic Models: Explaining and Finding Good Demonstrations for In-Context Learning （2023.01.27）
Xinyi Wang, Wanrong Zhu, William Yang Wang . - 【ArXiv】
Transformers as Algorithms: Generalization and Stability in In-context Learning （2023.01.17）
Yingcong Li, M. E. Ildiz, Dimitris Papailiopoulos, Samet Oymak
OPT-IML: Scaling Language Model Instruction Meta Learning through the Lens of Generalization （2022.12.22）
S. Iyer, Xiaojuan Lin, Ramakanth Pasunuru, Todor Mihaylov, Daniel Simig, etc . - 【ArXiv】
Prompt-Augmented Linear Probing: Scaling Beyond The Limit of Few-shot In-Context Learners （2022.12.21）
Hyunsoo Cho, Hyuhng Joon Kim, Junyeob Kim, Sang-Woo Lee, Sang-goo Lee, etc . - 【ArXiv】
In-context Learning Distillation: Transferring Few-shot Learning Ability of Pre-trained Language Models （2022.12.20）
Yukun Huang, Yanda Chen, Zhou Yu, K. McKeown . - 【ArXiv】
Self-adaptive In-context Learning （2022.12.20）
Zhiyong Wu, Yaoxiang Wang, Jiacheng Ye, Lingpeng Kong . - 【ArXiv】
Is GPT-3 a Good Data Annotator? （2022.12.20）
Bosheng Ding, Chengwei Qin, Linlin Liu, Lidong Bing, Shafiq R. Joty, etc . - 【ArXiv】
Towards Understanding Chain-of-Thought Prompting: An Empirical Study of What Matters （2022.12.20）
Boshi Wang, Sewon Min, Xiang Deng, Jiaming Shen, You Wu, etc . - 【ArXiv】
Why Can GPT Learn In-Context? Language Models Secretly Perform Gradient Descent as Meta-Optimizers （2022.12.20）
Damai Dai, Yutao Sun, Li Dong, Y. Hao, Zhifang Sui, etc . - 【ArXiv】
Z-ICL: Zero-Shot In-Context Learning with Pseudo-Demonstrations （2022.12.19）
Xinxi Lyu, Sewon Min, Iz Beltagy, Luke Zettlemoyer, Hannaneh Hajishirzi . - 【Annual Meeting of the Association for Computational Linguistics】
Rethinking the Role of Scale for In-Context Learning: An Interpretability-based Case Study at 66 Billion Scale （2022.12.18）
Hritik Bansal, Karthik Gopalakrishnan, Saket Dingliwal, S. Bodapati, Katrin Kirchhoff, etc . - 【ArXiv】
Transformers learn in-context by gradient descent （2022.12.15）
J. Oswald, Eyvind Niklasson, E. Randazzo, J. Sacramento, A. Mordvintsev, etc . - 【ArXiv】
Structured Prompting: Scaling In-Context Learning to 1, 000 Examples （2022.12.13）
Y. Hao, Yutao Sun, Li Dong, Zhixiong Han, Yuxian Gu, etc . - 【ArXiv】
Diverse Demonstrations Improve In-context Compositional Generalization （2022.12.13）
Itay Levy, Ben Bogin, Jonathan Berant . - 【ArXiv】
What learning algorithm is in-context learning? Investigations with linear models （2022.11.28）
Ekin Akyürek, D. Schuurmans, Jacob Andreas, Tengyu Ma, Denny Zhou . - 【ArXiv】
What learning algorithm is in-context learning? Investigations with linear models （2022.11.28）
Ekin Akyürek, D. Schuurmans, Jacob Andreas, Tengyu Ma, Denny Zhou . - 【International Conference on Learning Representations】
Complementary Explanations for Effective In-Context Learning （2022.11.25）
Xi Ye, Srini Iyer, Asli Celikyilmaz, V. Stoyanov, Greg Durrett, etc . - 【ArXiv】
Complementary Explanations for Effective In-Context Learning （2022.11.25）
Xi Ye, Srini Iyer, Asli Celikyilmaz, Ves Stoyanov, Greg Durrett, etc . - 【Annual Meeting of the Association for Computational Linguistics】
Large Language Models with Controllable Working Memory （2022.11.09）
Daliang Li, A. Rawat, M. Zaheer, Xin Wang, M. Lukasik, etc . - 【ArXiv】
Tuning Language Models as Training Data Generators for Augmentation-Enhanced Few-Shot Learning （2022.11.06）
Yu Meng, Martin Michalski, Jiaxin Huang, Yu Zhang, T. Abdelzaher, etc . - 【ArXiv】
Large Language Models Are Human-Level Prompt Engineers （2022.11.03）
Yongchao Zhou, Andrei Ioan Muresanu, Ziwen Han, Keiran Paster, Silviu Pitis, etc . - 【ArXiv】
ProGen: Progressive Zero-shot Dataset Generation via In-context Feedback （2022.10.22）
Jiacheng Ye, Jiahui Gao, Jiangtao Feng, Zhiyong Wu, Tao Yu, etc . - 【Conference on Empirical Methods in Natural Language Processing】
Challenging BIG-Bench Tasks and Whether Chain-of-Thought Can Solve Them （2022.10.17）
Mirac Suzgun, Nathan Scales, Nathanael Scharli, Sebastian Gehrmann, Yi Tay, etc . - 【ArXiv】
In-context Learning and Induction Heads （2022.09.24）
Catherine Olsson, Nelson Elhage, Neel Nanda, Nicholas Joseph, Nova DasSarma, etc . - 【ArXiv】
Selecting Better Samples from Pre-trained LLMs: A Case Study on Question Generation （2022.09.22）
Xingdi Yuan, Tong Wang, Yen-Hsiang Wang, Emery Fine, Rania Abdelghani, etc . - 【Annual Meeting of the Association for Computational Linguistics】
On the Relation between Sensitivity and Accuracy in In-context Learning （2022.09.16）
Yanda Chen, Chen Zhao, Zhou Yu, K. McKeown, He He . - 【ArXiv】
What Can Transformers Learn In-Context? A Case Study of Simple Function Classes （2022.08.01）
Shivam Garg, Dimitris Tsipras, Percy Liang, G. Valiant . - 【ArXiv】
Rationale-Augmented Ensembles in Language Models （2022.07.02）
Xuezhi Wang, Jason Wei, D. Schuurmans, Quoc Le, E. Chi, etc . - 【ArXiv】
Self-Generated In-Context Learning: Leveraging Auto-regressive Language Models as a Demonstration Generator （2022.06.16）
Hyuhng Joon Kim, Hyunsoo Cho, Junyeob Kim, Taeuk Kim, Kang Min Yoo, etc . - 【ArXiv】
Instruction Induction: From Few Examples to Natural Language Task Descriptions （2022.05.22）
Or Honovich, Uri Shaham, Samuel R. Bowman, Omer Levy . - 【ArXiv】
Prototypical Calibration for Few-shot Learning of Language Models （2022.05.20）
Zhixiong Han, Y. Hao, Li Dong, Yutao Sun, Furu Wei . - 【ArXiv】
Few-Shot Parameter-Efficient Fine-Tuning is Better and Cheaper than In-Context Learning （2022.05.11）
Haokun Liu, Derek Tam, Mohammed Muqeeth, Jay Mohta, Tenghao Huang, etc . - 【ArXiv】
Improving In-Context Few-Shot Learning via Self-Supervised Training （2022.05.03）
Mingda Chen, Jingfei Du, Ramakanth Pasunuru, Todor Mihaylov, Srini Iyer, etc . - 【North American Chapter of the Association for Computational Linguistics】
On the Effect of Pretraining Corpora on In-context Learning by a Large-scale Language Model （2022.04.28）
Seongjin Shin, Sang-Woo Lee, Hwijeen Ahn, Sungdong Kim, Hyoungseok Kim, etc . - 【North American Chapter of the Association for Computational Linguistics】
Data Distributional Properties Drive Emergent In-Context Learning in Transformers （2022.04.22）
Stephanie C. Y. Chan, Adam Santoro, Andrew Kyle Lampinen, Jane X. Wang, Aaditya K Singh, etc . - 【ArXiv】
Can language models learn from explanations in context? （2022.04.05）
Andrew Kyle Lampinen, I. Dasgupta, Stephanie C. Y. Chan, Kory Matthewson, Michael Henry Tessler, etc . - 【Conference on Empirical Methods in Natural Language Processing】
Self-Consistency Improves Chain of Thought Reasoning in Language Models （2022.03.21）
Xuezhi Wang, Jason Wei, D. Schuurmans, Quoc Le, E. Chi, etc . - 【ArXiv】
Rethinking the Role of Demonstrations: What Makes In-Context Learning Work? （2022.02.25）
Sewon Min, Xinxi Lyu, Ari Holtzman, Mikel Artetxe, M. Lewis, etc . - 【Conference on Empirical Methods in Natural Language Processing】
Co-training Improves Prompt-based Learning for Large Language Models （2022.02.02）
Hunter Lang, Monica Agrawal, Yoon Kim, D. Sontag . - 【International Conference on Machine Learning】
Black-Box Tuning for Language-Model-as-a-Service （2022.01.10）
Tianxiang Sun, Yunfan Shao, Hong Qian, Xuanjing Huang, Xipeng Qiu . - 【International Conference on Machine Learning】
Learning To Retrieve Prompts for In-Context Learning （2021.12.16）
Ohad Rubin, Jonathan Herzig, Jonathan Berant . - 【North American Chapter of the Association for Computational Linguistics】
True Few-Shot Learning with Prompts—A Real-World Perspective （2021.11.26）
Timo Schick, Hinrich Schütze . - 【Transactions of the Association for Computational Linguistics】
An Explanation of In-context Learning as Implicit Bayesian Inference （2021.11.03）
Sang Michael Xie, Aditi Raghunathan, Percy Liang, Tengyu Ma . - 【International Conference on Learning Representations】
MetaICL: Learning to Learn In Context （2021.10.29）
Sewon Min, M. Lewis, Luke Zettlemoyer, Hannaneh Hajishirzi . - 【North American Chapter of the Association for Computational Linguistics】
Meta-learning via Language Model In-context Tuning （2021.10.15）
Yanda Chen, Ruiqi Zhong, Sheng Zha, G. Karypis, He He . - 【Annual Meeting of the Association for Computational Linguistics】
Multitask Prompted Training Enables Zero-Shot Task Generalization （2021.10.15）
Victor Sanh, Albert Webson, Colin Raffel, Stephen H. Bach, Lintang Sutawika, etc . - 【International Conference on Learning Representations】
The Inductive Bias of In-Context Learning: Rethinking Pretraining Example Design （2021.10.09）
Yoav Levine, Noam Wies, Daniel Jannai, D. Navon, Yedid Hoshen, etc . - 【International Conference on Learning Representations】
Reframing Instructional Prompts to GPTk’s Language （2021.09.16）
Swaroop Mishra, Daniel Khashabi, Chitta Baral, Yejin Choi, Hannaneh Hajishirzi . - 【Findings】
Noisy Channel Language Model Prompting for Few-Shot Text Classification （2021.08.09）
Sewon Min, Michael Lewis, Hannaneh Hajishirzi, Luke Zettlemoyer . - 【Annual Meeting of the Association for Computational Linguistics】
Multimodal Few-Shot Learning with Frozen Language Models （2021.06.25）
Maria Tsimpoukelli, Jacob Menick, Serkan Cabi, S. Eslami, Oriol Vinyals, etc . - 【Neural Information Processing Systems】
Fantastically Ordered Prompts and Where to Find Them: Overcoming Few-Shot Prompt Order Sensitivity （2021.04.18）
Yao Lu, Max Bartolo, Alastair Moore, S. Riedel, Pontus Stenetorp . - 【Annual Meeting of the Association for Computational Linguistics】
Calibrate Before Use: Improving Few-Shot Performance of Language Models （2021.02.19）
Tony Zhao, Eric Wallace, Shi Feng, D. Klein, Sameer Singh . - 【International Conference on Machine Learning】
What Makes Good In-Context Examples for GPT-3? （2021.01.17）
Jiachang Liu, Dinghan Shen, Yizhe Zhang, Bill Dolan, L. Carin, etc . - 【Workshop on Knowledge Extraction and Integration for Deep Learning Architectures; Deep Learning Inside Out】
Few-Shot Text Generation with Pattern-Exploiting Training （2020.12.22）
Timo Schick, Hinrich Schütze . - 【ArXiv】
InContext: A mobile application for the improvement of learning strategies at University （2020.06.01）
Claudia-A. Lerma-Noriega, María-L. Flores-Palacios, Genaro Rebolledo-Méndez
Language Models are Few-Shot Learners （2020.05.28）
Tom B. Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, J. Kaplan, etc . - 【Neural Information Processing Systems】
Young-Jun Lee, Chae-Gyun Lim, Ho-Jin Choi . - 【International Conference on Computational Linguistics】
Transformers as Algorithms: Generalization and Implicit Model Selection in In-context Learning
Yingcong Li, M. E. Ildiz, Dimitris Papailiopoulos, Samet Oymak . - 【ArXiv】
Shicheng Xu, Liang Pang, Huawei Shen, Xueqi Cheng, Tat-Seng Chua . - 【arXiv.org】