Project Structure

• PEFT - Contains the experiments related to the PEFT technique.
  • Prompt-Tuning - Contains the experiments related to the Prompt-Tuning technique.
  • Adapter-Tuning - Contains the experiments related to the Adapter-Tuning technique.
• transformer-circuits - Contains the experiments related to the Mechanical Interpretation technique.
• embeddings - How to extract embeddings from the LLMs

Techniques Used

This repository utilizes several techniques for fine-tuning and prompt-tuning of Language Models (LLMs). Here are some of the key techniques used:

1. PEFT (Parameter Efficient Fine Tuning): This technique involves creating effective prompts that guide the model to generate the desired output.

   • Prompt-Tuning:

     • This is a method of fine-tuning where the model is trained to respond to specific prompts with specific responses.
     • PEFT: Parameter-Efficient Fine-Tuning for Transformers - The original paper on PEFT.

   • Adapter-Tuning:

     • This technique involves adding and training small, task-specific modules in the model without modifying the pre-trained parameters.
     • AdapterHub - A library for using and sharing adapters for fine-tuning LLMs.
     • Adapter-Tuning: Overcoming Overfitting in Large Language Models - The original paper on Adapter-Tuning.

2. Mechanistic-Interpretability:

Datasets

Models

1. Llama-2
2. Gemma
3. Mistral

Libraries

1. Huggingface Transformers
2. PyTorch Lightning
3. AdapterHub
4. TransformerLens

Experiments

Results

• TBD

Resources

• Repositories: -litgpt by Lighting AI : LitGPT is a command-line tool designed to easily finetune, pretrain, evaluate, and deploy 20+ LLMs on your own data. It features highly-optimized training recipes for the world's most powerful open-source large-language-models (LLMs). It is built on top of the Hugging Face Transformers library and PyTorch Lightning.

• Libraries: -AdapterHub : AdapterHub is a library for using and sharing adapters for fine-tuning LLMs. It provides a simple API for adding adapters to LLMs and fine-tuning them on downstream tasks. It also offers a repository of pre-trained adapters for various tasks and languages.

• Blogs:

  • LESSWRONG - A community blog focused on rationality, AI alignment, and other topics.
  • Chris Olah's Blog - A blog by Chris Olah, a researcher at OpenAI, that covers a wide range of topics related to AI and machine learning.
  • Neel Nanda's Blog - A blog by Neel Nanda, a researcher at DeepMind, that covers topics related to AI alignment, interpretability, and other areas.
  • Transformer Circuits Thread

• Podcasts:

  • The Alignment Newsletter Podcast - A podcast that discusses AI alignment, rationality, and other related topics.
  • The Bayesian Conspiracy - A podcast that discusses Bayesian reasoning, rationality, and other topics.
  • Mechanistic Interpretability - NEEL NANDA (DeepMind)