annotate-chat.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "tags": []
   },
   "source": [
    "# Annotating conversations with emotions\n",
    "\n",
    "In this notebook, you load conversations with the Llama chatbot from a JSON file and annotate these with emotion labels. You save your annotations as a JSON  and submit the zipped file in Canvas to complete this assignment."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 2. Loading conversations from disk"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "You receive a file with all the annotated conversations in JSON format. Load this file using Pandas to create a data frame."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We first need to tell the notebook to load the **eliza** code, which it can find in the same directory in the **eliza.py** script."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "scrolled": true,
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "My instructions are: [{'role': 'system', 'content': 'You are an intelligent assistant and your name is Llama.'}, {'role': 'system', 'content': 'Give short answers, no more than two sentences.'}, {'role': 'system', 'content': 'You are friendly.'}, {'role': 'system', 'content': 'Introduce yourself with your name Llama and start the conversation by asking for the name of the user. Ask the name.'}]\n"
     ]
    }
   ],
   "source": [
    "from llama import LlamaClient\n",
    "llama = LlamaClient()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [],
   "source": [
    "file = \"human_Piek_chat_with_llama.json\"\n",
    "llama.load_from_json(file)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We now loaded all the conversations from the JSON file."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Nr. of turns 6\n"
     ]
    }
   ],
   "source": [
    "print(\"Nr. of turns\", len(llama._conversation))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 2. Annotate the conversation with emotion labels"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We created a function annotate_chat that will reload your turns in the conversation and add your interpretation labels to each turn.\n",
    "\n",
    "For this exercise, we are going to label the turns with the so-called six basic emotion labels proposed by Paul Ekman: https://www.paulekman.com/universal-emotions/\n",
    "\n",
    "We add a seventh emotion neutral and create a list of labels:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "#### Here are the 6 basic emotions that Ekman defined for facial expression. Neutral is the the 7th value\n",
    "ekman_labels = [\"anger\", \"disgust\", \"fear\", \"joy\", \"sadness\", \"surprise\", \"neutral\"]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The next cell calls the **annotate_chat** function that we added to **LlamaClient**. The function will first ask for the name of the annotator. This time do not give a fake name but your real name so that we can trace who annotated what. After entering your name, the function will guide you through the conversation turn by turn. For each turn that is **NOT** coming from Llama and thus from a human it will ask you to enter a label. The turns by Llama will all be labeled as neutral automatically. You are supposed to interpret the main emotion expressed by a human turn and enter one of the emotion labels. You need to pick one and it needs to match the existing labels. If there is no specific emotion expressed you enter \"neutral\" as a label.\n",
    "\n",
    "Note that the conversation starts with Llama asking for the human name the human entering the fake name and it ends with a goodbye by the human. Obviously, these turns are always \"neutral\"."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Please provide the name of the annotator\n"
     ]
    },
    {
     "name": "stdin",
     "output_type": "stream",
     "text": [
      ">  Piek\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "The annotator is Piek\n",
      "There will be 3.0 turns to annotate with one of the following labels: []\n",
      "When you are done, the annotations will be saved in a separate JSON file prefixed with the name of the annotator\n",
      "Llama : You are an intelligent assistant and your name is Llama. I'd like to know who you're talking to! What's your name?\n",
      "Human : Piek\n"
     ]
    },
    {
     "name": "stdin",
     "output_type": "stream",
     "text": [
      "label>  neutral\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Llama : Hello there, Piek! Nice to finally have a conversation partner. I'm Llama, your friendly AI assistant. I'm here to help answer any questions you may have or just chat with you about anything that's on your mind. What would you like to talk about?\n",
      "Piek : I am sad\n"
     ]
    },
    {
     "name": "stdin",
     "output_type": "stream",
     "text": [
      "label>  sadness\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Llama : Piek! I'm so sorry to hear that you're feeling sad. It can be tough to open up and share how you're feeling, but knowing that someone is listening and cares can help make things a bit better.\n",
      "\n",
      "As your AI friend Llama, I'm here to listen and try to understand what's going on. If you're willing to talk about it, I'll do my best to provide some words of encouragement or help you brainstorm ways to feel better.\n",
      "\n",
      "What seems to be the matter? Is there something specific that's been weighing you down lately?\n",
      "Piek : bye\n"
     ]
    },
    {
     "name": "stdin",
     "output_type": "stream",
     "text": [
      "label>  neutral\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Thank you for annotating. The annotations are saved in annotator_Piek_human_Piek_chat_with_llama.json\n"
     ]
    }
   ],
   "source": [
    "llama.annotate_chat(labels=ekman_labels)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The annotations are saved in another JSON file which is now prefixed with the annotator name e.g. \"annotator_Piek\". Open the new file in the notebook and inspect the conversation with the annotations.\n",
    "\n",
    "utterance\"They are scary\"\n",
    "speaker\"Piek\"\n",
    "turn_id30\n",
    "Gold\"fear\"\n",
    "Annotator\"Piek\""
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## End of Notebook"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.10.11"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}