XNLM Lab @ LCL 2023 🔍

This repository contains materials for the lab "Explaining Neural Language Models from Internal Representations to Model Predictions" by Alessio Miaschi and Gabriele Sarti at the AILC Lectures on Computational Linguistics 2023.

Contents

The lab session is divided in two parts:

• Notebooks 1.1 and 1.2 introduce probing approaches for evaluating the linguistic knowledge of neural language models and shows how diagnostic classifiers can be used to study models' internal representations.

• Notebook 2 discusses feature attribution methods and their usage to identify biases and prediction errors in NLMs for classification and generation tasks.

You can find a notebook for each part in the notebooks folder.

Tools and Frameworks

The lab sessions make use of the Jupyter environment. You can use the following links to get started:

• Jupyter Notebook/Lab Installation
• Jupyter Quickstart

Alternatively, it is possible to use the notebooks via the Google Colab web environment simply by clicking on the button at the beginning of each notebook. If you’re running on Windows, we recommend following along using a Colab notebook. If you’re using a Linux distribution or macOS, you can use either approach described here. For an intro to the Colab environment, refer to:

• Colab Quickstart

The file requirements.txt in this repository contains the list of all the packages required to run the lab sessions. You can create a Python virtual environment (Python>=3.8.1) and install them using the following command:

python -m venv venv
source venv/bin/activate
pip install -r requirements.txt

Make sure the virtual environment is activated before running Jupyter. If you are using Colab, simply run the cell at the beginning of each notebook to install the required packages. Refer to Using a Python Virtual Environment for more details on how to create and activate a virtual environment.

About us

Alessio Miaschi is a Post Doctoral Researcher at the ItaliaNLP Lab from the Institute for Computational Linguistics “A. Zampolli” (CNR-ILC, Pisa). He received his Ph.D. in Computer Science from the University of Pisa in 2022 with a thesis focused on the definition of techniques for interpreting and understanding the linguistic knowledge implicitly encoded in recent state-of-the-art Neural Language Models. His current research interest mainly focuses on the development and analysis of neural network models for language processing, as well as on the definition of NLP tools for educational applications.

Gabriele Sarti is a PhD student at the Computational Linguistics Group (GroNLP) of the University of Groningen. He is part of the Dutch consortium InDeep, working on interpretability for language generation and neural machine translation. Previously, he worked as a research intern at Amazon Translate NYC, a research scientist at Aindo, and a research assistant at the ItaliaNLP Lab (CNR-ILC, Pisa). His research aims to improve our understanding of generative neural language models’ inner workings, with the ultimate goal of enhancing the controllability and robustness of these systems in human-AI interactions.

Join the Conversation on RiTA 🇮🇹

RiTA (Risorse per la Lingua Italiana) is a new community aimed at bringing together enthusiasts and professionals of Italian computational linguistics and to create new collaborations for the construction of resources for the Italian language. If you are working on NLP-related topics and want to keep up with latest initiatives from the Italian research community, you can join the conversation on the RiTA Discord server. See you there! 🙂