01. Prompt engineering introduction.ipynb

{"nbformat":4,"nbformat_minor":0,"metadata":{"colab":{"provenance":[],"toc_visible":true},"kernelspec":{"name":"python3","display_name":"Python 3"},"language_info":{"name":"python"}},"cells":[{"cell_type":"markdown","metadata":{"id":"view-in-github"},"source":["<a href=\"https://colab.research.google.com/github/CDAC-lab/isie2023/blob/main/tutorial-notebook-1.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"]},{"cell_type":"markdown","source":["# Prompt Engineering\n","With prompt engineering we are learning how to get best out of large language models with structuring our inputs. It helps to customize large language models and integrate them to our use cases.\n","\n","This notebook will guide you through the process of prompt engineering with OpenAI's ChatGPT model.\n","\n","\n","*   Set up ChatGPT in Google Colab\n","*   Parameters\n","*   Basic Prompting\n","*   Advanced Prompting\n","\n"],"metadata":{"id":"hzma7M56r3C_"}},{"cell_type":"markdown","source":["## Setup\n","This section will guide you on how to set up the necessary OpenAI libraries in your development environment. These libraries provide the essential interfaces for interacting with OpenAI's models like ChatGPT.\n"],"metadata":{"id":"dJfhMPefFipY"}},{"cell_type":"markdown","source":["## Installing required libraries"],"metadata":{"id":"E-yxhl6iGWdi"}},{"cell_type":"code","execution_count":null,"metadata":{"colab":{"base_uri":"https://localhost:8080/"},"id":"Gi_qGE98rzUY","outputId":"0d6b234a-b4c6-4c7f-91c0-d1462111d46d","executionInfo":{"status":"ok","timestamp":1696836253712,"user_tz":-660,"elapsed":14212,"user":{"displayName":"Gihan Gamage","userId":"03983047858725985766"}}},"outputs":[{"output_type":"stream","name":"stdout","text":["Requirement already satisfied: pip in /usr/local/lib/python3.10/dist-packages (23.1.2)\n","Collecting pip\n","  Downloading pip-23.2.1-py3-none-any.whl (2.1 MB)\n","\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m2.1/2.1 MB\u001b[0m \u001b[31m20.6 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n","\u001b[?25hInstalling collected packages: pip\n","  Attempting uninstall: pip\n","    Found existing installation: pip 23.1.2\n","    Uninstalling pip-23.1.2:\n","      Successfully uninstalled pip-23.1.2\n","Successfully installed pip-23.2.1\n","\u001b[2K   \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m77.0/77.0 kB\u001b[0m \u001b[31m2.3 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n","\u001b[?25h\u001b[33mWARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv\u001b[0m\u001b[33m\n","\u001b[0m"]}],"source":["!pip install --upgrade pip\n","!pip install -q openai"]},{"cell_type":"code","source":["import openai"],"metadata":{"id":"liZ3RgbosZQM"},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":["## Connecting to OpenAI using a key\n","Learn how to securely connect to OpenAI's API using the provided key. This ensures authorized access to OpenAI's services."],"metadata":{"id":"U0I_Ad6xHMOY"}},{"cell_type":"code","source":["#You can API key by registering in OpenAPI here https://platform.openai.com/account/api-keys..\n","openai.api_key = 'give_your_key'"],"metadata":{"id":"Qc5iuxoZHLjL"},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":["## Using ChatGPT via the endpoint\n","Understand how to interact with the ChatGPT model using OpenAI's API endpoint. This includes how to send requests and handle responses from the model."],"metadata":{"id":"7X7gYKziE7Ie"}},{"cell_type":"code","source":["import requests\n","\n","#here we define a function to use open-ai API end point. So that we can play with the parameters.\n","# This model will use gpt-3.5-turbo(chatgpt) model in default\n","def use_endpoint(msg, temperature=1, top_p=1, model = \"gpt-3.5-turbo\"):\n","\n","  URL = \"https://api.openai.com/v1/chat/completions\"\n","\n","  payload = {\n","  \"model\": model,\n","  \"messages\": [{\"role\": \"user\", \"content\": msg}],\n","  \"temperature\" : temperature,\n","  \"top_p\":top_p,\n","  }\n","\n","  headers = {\n","  \"Content-Type\": \"application/json\",\n","  \"Authorization\": f\"Bearer {openai.api_key}\"\n","  }\n","\n","  response = requests.post(URL, headers=headers, json=payload, stream=False)\n","  response_json = response.json()\n","  # print(response_json)\n","  return response_json['choices'][0]['message']['content'].strip()"],"metadata":{"id":"RGX2d_Xcr_wQ"},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":["# Parameters\n","This section will detail the different parameters that can be adjusted when making requests to the ChatGPT model and how they influence the generated responses.\n","\n","There are several parameters in ChatGPT.\n","Max Length - Length of the response.\n","\n","*   Stop sequences - trigger words which will stop generating the terms\n","*   Frequency penalty - penalize tokens based on their current frequency\n","*   Presence penalty - penalize tokens based on their current presense\n","*   Temperature - Control the randomness of the response\n","*   Top P - Control the randomness of the response\n","\n","\n","\n"],"metadata":{"id":"2OUF_Tz7GK73"}},{"cell_type":"markdown","source":["## Temperature and Top-p\n","\n","Both of these parameters decides the degree of deterministic nature of the language model.\n","\n","Idea - large language model always trying to predict the next most probable word, given a sequence of words. So top p and temperature both decides the pool of words to pick the next word.\n","\n","\n","<img src=\"http://drive.google.com/uc?export=view&id=14gIyrIHTsJH4SOyHAk65cgT3ta2RKpSR\" />\n"],"metadata":{"id":"mKdflGRVFEEb"}},{"cell_type":"code","source":["#low temperature and high top-p\n","prompt = \"create a tag line for a coffee shop\"\n","for i in range(10):\n","  response = use_endpoint(prompt, temperature=0, top_p=0)\n","  print(response)"],"metadata":{"colab":{"base_uri":"https://localhost:8080/"},"id":"GsSR3QyoOGqv","outputId":"4814bf97-3a92-4583-976b-d748ad23278d","executionInfo":{"status":"ok","timestamp":1696838770257,"user_tz":-660,"elapsed":14288,"user":{"displayName":"Gihan Gamage","userId":"03983047858725985766"}}},"execution_count":null,"outputs":[{"output_type":"stream","name":"stdout","text":["\"Awaken your senses, one sip at a time.\"\n","\"Awaken your senses, one sip at a time.\"\n","\"Awaken your senses, one sip at a time.\"\n","\"Awaken your senses, one sip at a time.\"\n","\"Awaken your senses, one sip at a time.\"\n","\"Awaken your senses, one sip at a time.\"\n","\"Awaken your senses, one sip at a time.\"\n","\"Awaken your senses, one sip at a time.\"\n","\"Awaken your senses, one sip at a time.\"\n","\"Awaken your senses, one sip at a time.\"\n"]}]},{"cell_type":"code","source":["#high temperature and high top-p\n","for i in range(10):\n","  response = use_endpoint(prompt, temperature=1, top_p=1)\n","  print(response)"],"metadata":{"colab":{"base_uri":"https://localhost:8080/"},"id":"lrzoPouSOptL","outputId":"22e98bf2-7d13-4c5f-f4f1-878c933faa95","executionInfo":{"status":"ok","timestamp":1696838786912,"user_tz":-660,"elapsed":14393,"user":{"displayName":"Gihan Gamage","userId":"03983047858725985766"}}},"execution_count":null,"outputs":[{"output_type":"stream","name":"stdout","text":["\"Awaken your senses, one cup at a time\"\n","\"Awaken your senses with our brews and fuel your day with the perfect cup of joe!\"\n","\"Wake up and smell the aroma of blissful brews\"\n","\"Awaken your senses with every sip.\"\n","\"Awaken your senses with our divine brews!\"\n","\"Fuel your day, one sip at a time!\"\n","\"Fuel your day with a sip of perfection.\"\n","\"Awaken your senses with the perfect brew at our cozy coffee haven!\"\n","\"Awaken Your Senses, One Sip at a Time!\"\n","\"Awake your senses, one sip at a time!\"\n"]}]},{"cell_type":"markdown","source":["# Basic Prompting\n","The concept of prompting will be introduced here. Learn how to structure input prompts to guide the model's output effectively."],"metadata":{"id":"7v_OVzS4HZ_e"}},{"cell_type":"markdown","source":["## Summarisation\n","See how to utilize ChatGPT for text summarization tasks, providing a brief yet comprehensive overview of a larger text."],"metadata":{"id":"xUeqJ-ygI7t1"}},{"cell_type":"code","source":["# Example from DAIR.AI\n","prompt = \"\"\"Antibiotics are a type of medication used to treat bacterial infections.\n","They work by either killing the bacteria or preventing them from reproducing, allowing the body's immune system to fight off the infection.\n","Antibiotics are usually taken orally in the form of pills, capsules, or liquid solutions, or sometimes administered intravenously.\n","They are not effective against viral infections, and using them inappropriately can lead to antibiotic resistance.\n","\n","Explain the above text in a simple single sentence:\"\"\"\n","\n","print(use_endpoint(prompt, 1, 1))\n"],"metadata":{"colab":{"base_uri":"https://localhost:8080/"},"id":"cBYCqdiEI61P","outputId":"58afffe3-7d49-4945-a238-d114ed076a48","executionInfo":{"status":"ok","timestamp":1696839233293,"user_tz":-660,"elapsed":1986,"user":{"displayName":"Gihan Gamage","userId":"03983047858725985766"}}},"execution_count":null,"outputs":[{"output_type":"stream","name":"stdout","text":["Antibiotics are medicines that help the body fight bacterial infections by either killing the bacteria or stopping their growth.\n"]}]},{"cell_type":"markdown","source":["## Question-Answering\n","This section will demonstrate how to frame questions to the model in a way that elicits accurate and useful answers."],"metadata":{"id":"FwGJUT0UJePf"}},{"cell_type":"code","source":["# Example from DAIR.AI\n","prompt = \"\"\"Answer the question based on the context below. Keep the answer short and concise. Respond \"Unsure about answer\" if not sure about the answer.\n","\n","Context: Teplizumab traces its roots to a New Jersey drug company called Ortho Pharmaceutical. There, scientists generated an early version of the antibody, dubbed OKT3.\n","Originally sourced from mice, the molecule was able to bind to the surface of T cells and limit their cell-killing potential.\n","In 1986, it was approved to help prevent organ rejection after kidney transplants, making it the first therapeutic antibody allowed for human use.\n","\n","Question: What was OKT3 originally sourced from?\n","\n","Answer:\"\"\"\n","\n","print(use_endpoint(prompt, 0.5, 0.5))"],"metadata":{"colab":{"base_uri":"https://localhost:8080/"},"id":"ek5W-RxoJpE2","outputId":"d456f6f4-f9a3-461c-853b-51d245f1e8d8","executionInfo":{"status":"ok","timestamp":1696839197462,"user_tz":-660,"elapsed":1425,"user":{"displayName":"Gihan Gamage","userId":"03983047858725985766"}}},"execution_count":null,"outputs":[{"output_type":"stream","name":"stdout","text":["Mice.\n"]}]},{"cell_type":"markdown","source":["## Text Classification\n","Learn how to use ChatGPT for classifying text into predefined categories. This showcases the model's ability to transform unstructured text into structured data."],"metadata":{"id":"9FiC66vFLCV2"}},{"cell_type":"code","source":["# Example from DAIR.AI\n","prompt = \"\"\"Classify the text into neutral, negative or positive.\n","\n","Text: I think the food was okay.\n","\n","Sentiment:\"\"\"\n","\n","print(use_endpoint(prompt, 0.5, 0.5))"],"metadata":{"colab":{"base_uri":"https://localhost:8080/"},"outputId":"a1fa9eb7-eb07-4487-c02c-47f45120bb5b","id":"wRXnyvOpLCV3","executionInfo":{"status":"ok","timestamp":1696839305239,"user_tz":-660,"elapsed":953,"user":{"displayName":"Gihan Gamage","userId":"03983047858725985766"}}},"execution_count":null,"outputs":[{"output_type":"stream","name":"stdout","text":["Neutral\n"]}]},{"cell_type":"markdown","source":["\n","# Advanced Prompting with Zero-shot, One-shot, Few-shot learning\n","This part will dive deeper into the art of prompting. It will explore how to utilize the concepts of zero-shot, one-shot, and few-shot learning to guide the model's output even further. Learn how to provide context through examples to guide the model's behavior."],"metadata":{"id":"LWC4PoxLAdSy"}},{"cell_type":"markdown","source":["## Zero Shot Learning\n","\n","In zero-shot learning, the model generates an output for a specific task without having seen any explicit examples of that task during its training. The model relies solely on its general understanding of language and the specific prompt it's given. This is a challenging scenario, as the model needs to generalize well beyond its training data. This technique is useful when there are no available training examples for a specific task."],"metadata":{"id":"GC1HKenFv1qw"}},{"cell_type":"code","source":["prompt = \"\"\"Generate 10 possible names for my new dog.\"\"\"\n","\n","print(use_endpoint(prompt, 0.5, 1))"],"metadata":{"id":"OVt1fuvXv1bZ","outputId":"cb75d3be-4392-488d-edec-5ffddbcbfdab","colab":{"base_uri":"https://localhost:8080/"},"executionInfo":{"status":"ok","timestamp":1696840058074,"user_tz":-660,"elapsed":1990,"user":{"displayName":"Gihan Gamage","userId":"03983047858725985766"}}},"execution_count":null,"outputs":[{"output_type":"stream","name":"stdout","text":["1. Max\n","2. Luna\n","3. Charlie\n","4. Bella\n","5. Milo\n","6. Daisy\n","7. Cooper\n","8. Sadie\n","9. Rocky\n","10. Lucy\n"]}]},{"cell_type":"markdown","source":["## One Shot Learning\n","\n","One-shot learning refers to the situation where the model is provided with a single example of a task at inference time to guide its output. This single example acts as a point of reference for the model, helping it understand what's expected in the output. One-shot learning is useful when you have a limited number of examples for a specific task."],"metadata":{"id":"ubXLYAtfAjlr"}},{"cell_type":"code","source":["prompt = \"\"\"Generate 10 possible names for my new dog.\n","A dog name that I like is Banana.\"\"\"\n","\n","print(use_endpoint(prompt, 0.5, 1))"],"metadata":{"colab":{"base_uri":"https://localhost:8080/"},"id":"upUqsfyQjM_w","executionInfo":{"status":"ok","timestamp":1696840108363,"user_tz":-660,"elapsed":2228,"user":{"displayName":"Gihan Gamage","userId":"03983047858725985766"}},"outputId":"409026f2-bfbf-4bc1-efaa-7cc7eb423831"},"execution_count":null,"outputs":[{"output_type":"stream","name":"stdout","text":["1. Coco\n","2. Peanut\n","3. Biscuit\n","4. Mango\n","5. Hazel\n","6. Waffles\n","7. Sprinkles\n","8. Mochi\n","9. Poppy\n","10. Noodle\n"]}]},{"cell_type":"markdown","source":["## Few Shot Learning\n","\n","Few-shot learning involves providing the model with a small number of examples of a task at inference time. These examples serve to guide the model's generation and help it produce the desired output. The idea is that the model can generalize from these few examples to understand and complete the task. This technique is valuable when you have more than one but still a limited number of examples for a task."],"metadata":{"id":"DOH9ufb4TvmJ"}},{"cell_type":"markdown","source":["Basically we are giving some input-output pairs and improve the results. This is the primary fine-tuning method of GPT."],"metadata":{"id":"pdWNUp4FZDth"}},{"cell_type":"markdown","source":["With a basic prompt using ChatGPT."],"metadata":{"id":"-a8jNtTmZNcz"}},{"cell_type":"code","source":["prompt = \"\"\"Generate 10 possible names for my new dog.\n","Dog names that I like include:\n","– Banana\n","– Kiwi\n","– Pineapple\n","– Coconut\"\"\"\n","\n","print(use_endpoint(prompt))"],"metadata":{"colab":{"base_uri":"https://localhost:8080/"},"id":"hw-yqAH_B__p","outputId":"db3e8894-8fcf-4ba0-ae24-526e60e770d9","executionInfo":{"status":"ok","timestamp":1696840166847,"user_tz":-660,"elapsed":2176,"user":{"displayName":"Gihan Gamage","userId":"03983047858725985766"}}},"execution_count":null,"outputs":[{"output_type":"stream","name":"stdout","text":["1. Mango\n","2. Papaya\n","3. Peach\n","4. Guava\n","5. Apricot\n","6. Plum\n","7. Lychee\n","8. Fig\n","9. Pomegranate\n","10. Persimmon\n"]}]},{"cell_type":"markdown","source":["With the more examples the prompt included, the closer the generated output conforms to what is expected. With zero-shot, there may be no fruit names suggested; with one-shot, there may be several; and with few-shot, the suggestions may consist entirely of fruit-themed names."],"metadata":{"id":"IlW7CC2Vjvs-"}}]}