tutorial-notebook-3.ipynb

{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "colab": {
      "provenance": [],
      "include_colab_link": true
    },
    "kernelspec": {
      "name": "python3",
      "display_name": "Python 3"
    },
    "language_info": {
      "name": "python"
    },
    "accelerator": "GPU",
    "gpuClass": "standard"
  },
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "view-in-github",
        "colab_type": "text"
      },
      "source": [
        "<a href=\"https://colab.research.google.com/github/CDAC-lab/isie2023/blob/main/tutorial-notebook-3.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "# Exposing an AI based Question Answering Assistant on top of media content\n",
        "\n",
        "# Overview\n",
        "\n",
        "This notebook is designed to demonstrate an end-to-end pipeline leveraging the capabilities of OpenAI's Whisper, ChromaDB, and Langchain to enable intelligent querying of YouTube videos. We begin by taking a YouTube video URL, from which the audio is extracted and transcribed using Whisper, OpenAI's automatic speech recognition (ASR) system. This transcription is then vectorized using ChromaDB, a high-performance vector database, effectively transforming the unstructured text data into a structured, queryable form. Finally, Langchain is utilized to provide a natural language interface for querying the stored vector data, allowing users to extract meaningful information from the video content.\n",
        "\n",
        "## Table of Contents\n",
        "\n",
        "1. [Introduction and Setting Up](#section1)\n",
        "    - Introduction to the Notebook\n",
        "    - Installing Necessary Libraries\n",
        "    - Importing Libraries and Dependencies\n",
        "2. [Data Acquisition](#section2)\n",
        "    - Getting Video Data from YouTube\n",
        "    - Extracting Audio from YouTube Video\n",
        "3. [Transcription using Whisper](#section3)\n",
        "    - Introduction to Whisper\n",
        "    - Transcribing Audio to Text\n",
        "4. [Vectorization using ChromaDB](#section4)\n",
        "    - Introduction to ChromaDB\n",
        "    - Preprocessing Text for Vectorization\n",
        "    - Vectorizing Text Data\n",
        "5. [Querying with Langchain](#section5)\n",
        "    - Introduction to Langchain\n",
        "    - Setting Up Langchain for Querying\n",
        "    - Formulating and Executing Queries\n",
        "6. [Analysis and Visualization](#section6)\n",
        "    - Analyzing Query Results\n",
        "    - Visualizing Query Results\n",
        "7. [Conclusion and Possible Extensions](#section7)\n",
        "    - Summary of Achievements\n",
        "    - Potential Future Work\n",
        "8. [References and Additional Resources](#section8)\n",
        "\n",
        "\n"
      ],
      "metadata": {
        "id": "92pgzYPbqiRK"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "# Introduction and Setting Up\n",
        "\n",
        "## Introduction to the Notebook\n",
        "Welcome to our notebook! This project aims to create an end-to-end pipeline to extract, process, vectorize, and query the content of YouTube videos. By using state-of-the-art tools like Whisper, ChromaDB, and Langchain, we aim to transform unstructured video content into a structured and easily searchable form.\n",
        "\n",
        "## Installing Necessary Libraries\n",
        "In this section, we'll guide you through the installation process for all the necessary libraries that we'll use throughout this notebook. This includes OpenAI's Whisper for speech recognition, ChromaDB for vectorization, and Langchain for natural language querying.\n",
        "\n",
        "## Importing Libraries and Dependencies\n",
        "Here, we will import all the required Python libraries and dependencies that we'll be using in our notebook. This includes standard libraries for data handling and manipulation, as well as libraries specific to our pipeline such as the API wrappers for Whisper, ChromaDB, and Langchain."
      ],
      "metadata": {
        "id": "3_ddlZvzzN5R"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "## Install libraries"
      ],
      "metadata": {
        "id": "H_juZ2X-OP1q"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "!pip -qqq install git+https://github.com/openai/whisper.git\n",
        "!pip -qqq install pytube\n",
        "!pip install langchain\n",
        "!pip install chromadb\n",
        "!pip install openai"
      ],
      "metadata": {
        "id": "7N1JGqD-KGgV",
        "outputId": "9ee5e5a1-b8fd-45e8-b1a6-2008e70344d6",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "execution_count": 1,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "  Installing build dependencies ... \u001b[?25l\u001b[?25hdone\n",
            "  Getting requirements to build wheel ... \u001b[?25l\u001b[?25hdone\n",
            "  Preparing metadata (pyproject.toml) ... \u001b[?25l\u001b[?25hdone\n",
            "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m1.7/1.7 MB\u001b[0m \u001b[31m16.5 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[?25h  Building wheel for openai-whisper (pyproject.toml) ... \u001b[?25l\u001b[?25hdone\n",
            "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m57.6/57.6 kB\u001b[0m \u001b[31m3.5 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[?25hLooking in indexes: https://pypi.org/simple, https://us-python.pkg.dev/colab-wheels/public/simple/\n",
            "Collecting langchain\n",
            "  Downloading langchain-0.0.204-py3-none-any.whl (1.1 MB)\n",
            "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m1.1/1.1 MB\u001b[0m \u001b[31m10.1 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[?25hRequirement already satisfied: PyYAML>=5.4.1 in /usr/local/lib/python3.10/dist-packages (from langchain) (6.0)\n",
            "Requirement already satisfied: SQLAlchemy<3,>=1.4 in /usr/local/lib/python3.10/dist-packages (from langchain) (2.0.10)\n",
            "Collecting aiohttp<4.0.0,>=3.8.3 (from langchain)\n",
            "  Downloading aiohttp-3.8.4-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (1.0 MB)\n",
            "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m1.0/1.0 MB\u001b[0m \u001b[31m77.8 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[?25hCollecting async-timeout<5.0.0,>=4.0.0 (from langchain)\n",
            "  Downloading async_timeout-4.0.2-py3-none-any.whl (5.8 kB)\n",
            "Collecting dataclasses-json<0.6.0,>=0.5.7 (from langchain)\n",
            "  Downloading dataclasses_json-0.5.8-py3-none-any.whl (26 kB)\n",
            "Collecting langchainplus-sdk>=0.0.9 (from langchain)\n",
            "  Downloading langchainplus_sdk-0.0.11-py3-none-any.whl (22 kB)\n",
            "Requirement already satisfied: numexpr<3.0.0,>=2.8.4 in /usr/local/lib/python3.10/dist-packages (from langchain) (2.8.4)\n",
            "Requirement already satisfied: numpy<2,>=1 in /usr/local/lib/python3.10/dist-packages (from langchain) (1.22.4)\n",
            "Collecting openapi-schema-pydantic<2.0,>=1.2 (from langchain)\n",
            "  Downloading openapi_schema_pydantic-1.2.4-py3-none-any.whl (90 kB)\n",
            "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m90.0/90.0 kB\u001b[0m \u001b[31m14.6 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[?25hRequirement already satisfied: pydantic<2,>=1 in /usr/local/lib/python3.10/dist-packages (from langchain) (1.10.7)\n",
            "Requirement already satisfied: requests<3,>=2 in /usr/local/lib/python3.10/dist-packages (from langchain) (2.27.1)\n",
            "Requirement already satisfied: tenacity<9.0.0,>=8.1.0 in /usr/local/lib/python3.10/dist-packages (from langchain) (8.2.2)\n",
            "Requirement already satisfied: attrs>=17.3.0 in /usr/local/lib/python3.10/dist-packages (from aiohttp<4.0.0,>=3.8.3->langchain) (23.1.0)\n",
            "Requirement already satisfied: charset-normalizer<4.0,>=2.0 in /usr/local/lib/python3.10/dist-packages (from aiohttp<4.0.0,>=3.8.3->langchain) (2.0.12)\n",
            "Collecting multidict<7.0,>=4.5 (from aiohttp<4.0.0,>=3.8.3->langchain)\n",
            "  Downloading multidict-6.0.4-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (114 kB)\n",
            "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m114.5/114.5 kB\u001b[0m \u001b[31m17.5 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[?25hCollecting yarl<2.0,>=1.0 (from aiohttp<4.0.0,>=3.8.3->langchain)\n",
            "  Downloading yarl-1.9.2-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (268 kB)\n",
            "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m268.8/268.8 kB\u001b[0m \u001b[31m28.9 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[?25hCollecting frozenlist>=1.1.1 (from aiohttp<4.0.0,>=3.8.3->langchain)\n",
            "  Downloading frozenlist-1.3.3-cp310-cp310-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (149 kB)\n",
            "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m149.6/149.6 kB\u001b[0m \u001b[31m22.0 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[?25hCollecting aiosignal>=1.1.2 (from aiohttp<4.0.0,>=3.8.3->langchain)\n",
            "  Downloading aiosignal-1.3.1-py3-none-any.whl (7.6 kB)\n",
            "Collecting marshmallow<4.0.0,>=3.3.0 (from dataclasses-json<0.6.0,>=0.5.7->langchain)\n",
            "  Downloading marshmallow-3.19.0-py3-none-any.whl (49 kB)\n",
            "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m49.1/49.1 kB\u001b[0m \u001b[31m7.1 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[?25hCollecting marshmallow-enum<2.0.0,>=1.5.1 (from dataclasses-json<0.6.0,>=0.5.7->langchain)\n",
            "  Downloading marshmallow_enum-1.5.1-py2.py3-none-any.whl (4.2 kB)\n",
            "Collecting typing-inspect>=0.4.0 (from dataclasses-json<0.6.0,>=0.5.7->langchain)\n",
            "  Downloading typing_inspect-0.9.0-py3-none-any.whl (8.8 kB)\n",
            "Requirement already satisfied: typing-extensions>=4.2.0 in /usr/local/lib/python3.10/dist-packages (from pydantic<2,>=1->langchain) (4.5.0)\n",
            "Requirement already satisfied: urllib3<1.27,>=1.21.1 in /usr/local/lib/python3.10/dist-packages (from requests<3,>=2->langchain) (1.26.15)\n",
            "Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python3.10/dist-packages (from requests<3,>=2->langchain) (2022.12.7)\n",
            "Requirement already satisfied: idna<4,>=2.5 in /usr/local/lib/python3.10/dist-packages (from requests<3,>=2->langchain) (3.4)\n",
            "Requirement already satisfied: greenlet!=0.4.17 in /usr/local/lib/python3.10/dist-packages (from SQLAlchemy<3,>=1.4->langchain) (2.0.2)\n",
            "Requirement already satisfied: packaging>=17.0 in /usr/local/lib/python3.10/dist-packages (from marshmallow<4.0.0,>=3.3.0->dataclasses-json<0.6.0,>=0.5.7->langchain) (23.1)\n",
            "Collecting mypy-extensions>=0.3.0 (from typing-inspect>=0.4.0->dataclasses-json<0.6.0,>=0.5.7->langchain)\n",
            "  Downloading mypy_extensions-1.0.0-py3-none-any.whl (4.7 kB)\n",
            "Installing collected packages: mypy-extensions, multidict, marshmallow, frozenlist, async-timeout, yarl, typing-inspect, openapi-schema-pydantic, marshmallow-enum, langchainplus-sdk, aiosignal, dataclasses-json, aiohttp, langchain\n",
            "Successfully installed aiohttp-3.8.4 aiosignal-1.3.1 async-timeout-4.0.2 dataclasses-json-0.5.8 frozenlist-1.3.3 langchain-0.0.204 langchainplus-sdk-0.0.11 marshmallow-3.19.0 marshmallow-enum-1.5.1 multidict-6.0.4 mypy-extensions-1.0.0 openapi-schema-pydantic-1.2.4 typing-inspect-0.9.0 yarl-1.9.2\n",
            "Looking in indexes: https://pypi.org/simple, https://us-python.pkg.dev/colab-wheels/public/simple/\n",
            "Collecting chromadb\n",
            "  Downloading chromadb-0.3.26-py3-none-any.whl (123 kB)\n",
            "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m123.6/123.6 kB\u001b[0m \u001b[31m4.0 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[?25hRequirement already satisfied: pandas>=1.3 in /usr/local/lib/python3.10/dist-packages (from chromadb) (1.5.3)\n",
            "Collecting requests>=2.28 (from chromadb)\n",
            "  Downloading requests-2.31.0-py3-none-any.whl (62 kB)\n",
            "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m62.6/62.6 kB\u001b[0m \u001b[31m9.1 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[?25hRequirement already satisfied: pydantic>=1.9 in /usr/local/lib/python3.10/dist-packages (from chromadb) (1.10.7)\n",
            "Collecting hnswlib>=0.7 (from chromadb)\n",
            "  Downloading hnswlib-0.7.0.tar.gz (33 kB)\n",
            "  Installing build dependencies ... \u001b[?25l\u001b[?25hdone\n",
            "  Getting requirements to build wheel ... \u001b[?25l\u001b[?25hdone\n",
            "  Preparing metadata (pyproject.toml) ... \u001b[?25l\u001b[?25hdone\n",
            "Collecting clickhouse-connect>=0.5.7 (from chromadb)\n",
            "  Downloading clickhouse_connect-0.6.3-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (965 kB)\n",
            "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m965.1/965.1 kB\u001b[0m \u001b[31m5.3 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[?25hRequirement already satisfied: duckdb>=0.7.1 in /usr/local/lib/python3.10/dist-packages (from chromadb) (0.7.1)\n",
            "Collecting fastapi>=0.85.1 (from chromadb)\n",
            "  Downloading fastapi-0.97.0-py3-none-any.whl (56 kB)\n",
            "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m57.0/57.0 kB\u001b[0m \u001b[31m8.1 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[?25hCollecting uvicorn[standard]>=0.18.3 (from chromadb)\n",
            "  Downloading uvicorn-0.22.0-py3-none-any.whl (58 kB)\n",
            "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m58.3/58.3 kB\u001b[0m \u001b[31m6.1 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[?25hRequirement already satisfied: numpy>=1.21.6 in /usr/local/lib/python3.10/dist-packages (from chromadb) (1.22.4)\n",
            "Collecting posthog>=2.4.0 (from chromadb)\n",
            "  Downloading posthog-3.0.1-py2.py3-none-any.whl (37 kB)\n",
            "Requirement already satisfied: typing-extensions>=4.5.0 in /usr/local/lib/python3.10/dist-packages (from chromadb) (4.5.0)\n",
            "Collecting pulsar-client>=3.1.0 (from chromadb)\n",
            "  Downloading pulsar_client-3.2.0-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (5.3 MB)\n",
            "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m5.3/5.3 MB\u001b[0m \u001b[31m72.0 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[?25hCollecting onnxruntime>=1.14.1 (from chromadb)\n",
            "  Downloading onnxruntime-1.15.1-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (5.9 MB)\n",
            "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m5.9/5.9 MB\u001b[0m \u001b[31m100.4 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[?25hCollecting tokenizers>=0.13.2 (from chromadb)\n",
            "  Downloading tokenizers-0.13.3-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (7.8 MB)\n",
            "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m7.8/7.8 MB\u001b[0m \u001b[31m89.3 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[?25hRequirement already satisfied: tqdm>=4.65.0 in /usr/local/lib/python3.10/dist-packages (from chromadb) (4.65.0)\n",
            "Collecting overrides>=7.3.1 (from chromadb)\n",
            "  Downloading overrides-7.3.1-py3-none-any.whl (17 kB)\n",
            "Requirement already satisfied: certifi in /usr/local/lib/python3.10/dist-packages (from clickhouse-connect>=0.5.7->chromadb) (2022.12.7)\n",
            "Requirement already satisfied: urllib3>=1.26 in /usr/local/lib/python3.10/dist-packages (from clickhouse-connect>=0.5.7->chromadb) (1.26.15)\n",
            "Requirement already satisfied: pytz in /usr/local/lib/python3.10/dist-packages (from clickhouse-connect>=0.5.7->chromadb) (2022.7.1)\n",
            "Collecting zstandard (from clickhouse-connect>=0.5.7->chromadb)\n",
            "  Downloading zstandard-0.21.0-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (2.7 MB)\n",
            "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m2.7/2.7 MB\u001b[0m \u001b[31m74.5 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[?25hCollecting lz4 (from clickhouse-connect>=0.5.7->chromadb)\n",
            "  Downloading lz4-4.3.2-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (1.3 MB)\n",
            "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m1.3/1.3 MB\u001b[0m \u001b[31m70.7 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[?25hCollecting starlette<0.28.0,>=0.27.0 (from fastapi>=0.85.1->chromadb)\n",
            "  Downloading starlette-0.27.0-py3-none-any.whl (66 kB)\n",
            "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m67.0/67.0 kB\u001b[0m \u001b[31m9.5 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[?25hCollecting coloredlogs (from onnxruntime>=1.14.1->chromadb)\n",
            "  Downloading coloredlogs-15.0.1-py2.py3-none-any.whl (46 kB)\n",
            "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m46.0/46.0 kB\u001b[0m \u001b[31m4.0 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[?25hRequirement already satisfied: flatbuffers in /usr/local/lib/python3.10/dist-packages (from onnxruntime>=1.14.1->chromadb) (23.3.3)\n",
            "Requirement already satisfied: packaging in /usr/local/lib/python3.10/dist-packages (from onnxruntime>=1.14.1->chromadb) (23.1)\n",
            "Requirement already satisfied: protobuf in /usr/local/lib/python3.10/dist-packages (from onnxruntime>=1.14.1->chromadb) (3.20.3)\n",
            "Requirement already satisfied: sympy in /usr/local/lib/python3.10/dist-packages (from onnxruntime>=1.14.1->chromadb) (1.11.1)\n",
            "Requirement already satisfied: python-dateutil>=2.8.1 in /usr/local/lib/python3.10/dist-packages (from pandas>=1.3->chromadb) (2.8.2)\n",
            "Requirement already satisfied: six>=1.5 in /usr/local/lib/python3.10/dist-packages (from posthog>=2.4.0->chromadb) (1.16.0)\n",
            "Collecting monotonic>=1.5 (from posthog>=2.4.0->chromadb)\n",
            "  Downloading monotonic-1.6-py2.py3-none-any.whl (8.2 kB)\n",
            "Collecting backoff>=1.10.0 (from posthog>=2.4.0->chromadb)\n",
            "  Downloading backoff-2.2.1-py3-none-any.whl (15 kB)\n",
            "Requirement already satisfied: charset-normalizer<4,>=2 in /usr/local/lib/python3.10/dist-packages (from requests>=2.28->chromadb) (2.0.12)\n",
            "Requirement already satisfied: idna<4,>=2.5 in /usr/local/lib/python3.10/dist-packages (from requests>=2.28->chromadb) (3.4)\n",
            "Requirement already satisfied: click>=7.0 in /usr/local/lib/python3.10/dist-packages (from uvicorn[standard]>=0.18.3->chromadb) (8.1.3)\n",
            "Collecting h11>=0.8 (from uvicorn[standard]>=0.18.3->chromadb)\n",
            "  Downloading h11-0.14.0-py3-none-any.whl (58 kB)\n",
            "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m58.3/58.3 kB\u001b[0m \u001b[31m7.6 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[?25hCollecting httptools>=0.5.0 (from uvicorn[standard]>=0.18.3->chromadb)\n",
            "  Downloading httptools-0.5.0-cp310-cp310-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (414 kB)\n",
            "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m414.1/414.1 kB\u001b[0m \u001b[31m32.1 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[?25hCollecting python-dotenv>=0.13 (from uvicorn[standard]>=0.18.3->chromadb)\n",
            "  Downloading python_dotenv-1.0.0-py3-none-any.whl (19 kB)\n",
            "Requirement already satisfied: pyyaml>=5.1 in /usr/local/lib/python3.10/dist-packages (from uvicorn[standard]>=0.18.3->chromadb) (6.0)\n",
            "Collecting uvloop!=0.15.0,!=0.15.1,>=0.14.0 (from uvicorn[standard]>=0.18.3->chromadb)\n",
            "  Downloading uvloop-0.17.0-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (4.1 MB)\n",
            "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m4.1/4.1 MB\u001b[0m \u001b[31m30.5 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[?25hCollecting watchfiles>=0.13 (from uvicorn[standard]>=0.18.3->chromadb)\n",
            "  Downloading watchfiles-0.19.0-cp37-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (1.3 MB)\n",
            "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m1.3/1.3 MB\u001b[0m \u001b[31m70.8 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[?25hCollecting websockets>=10.4 (from uvicorn[standard]>=0.18.3->chromadb)\n",
            "  Downloading websockets-11.0.3-cp310-cp310-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (129 kB)\n",
            "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m129.9/129.9 kB\u001b[0m \u001b[31m12.8 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[?25hRequirement already satisfied: anyio<5,>=3.4.0 in /usr/local/lib/python3.10/dist-packages (from starlette<0.28.0,>=0.27.0->fastapi>=0.85.1->chromadb) (3.6.2)\n",
            "Collecting humanfriendly>=9.1 (from coloredlogs->onnxruntime>=1.14.1->chromadb)\n",
            "  Downloading humanfriendly-10.0-py2.py3-none-any.whl (86 kB)\n",
            "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m86.8/86.8 kB\u001b[0m \u001b[31m11.3 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[?25hRequirement already satisfied: mpmath>=0.19 in /usr/local/lib/python3.10/dist-packages (from sympy->onnxruntime>=1.14.1->chromadb) (1.3.0)\n",
            "Requirement already satisfied: sniffio>=1.1 in /usr/local/lib/python3.10/dist-packages (from anyio<5,>=3.4.0->starlette<0.28.0,>=0.27.0->fastapi>=0.85.1->chromadb) (1.3.0)\n",
            "Building wheels for collected packages: hnswlib\n",
            "  Building wheel for hnswlib (pyproject.toml) ... \u001b[?25l\u001b[?25hdone\n",
            "  Created wheel for hnswlib: filename=hnswlib-0.7.0-cp310-cp310-linux_x86_64.whl size=2119779 sha256=d2749c57f2e58bc4647765bb4f4066c83549ff69ca2ba3193a816c54b0c0d735\n",
            "  Stored in directory: /root/.cache/pip/wheels/8a/ae/ec/235a682e0041fbaeee389843670581ec6c66872db856dfa9a4\n",
            "Successfully built hnswlib\n",
            "Installing collected packages: tokenizers, monotonic, zstandard, websockets, uvloop, requests, python-dotenv, pulsar-client, overrides, lz4, humanfriendly, httptools, hnswlib, h11, backoff, watchfiles, uvicorn, starlette, posthog, coloredlogs, clickhouse-connect, onnxruntime, fastapi, chromadb\n",
            "  Attempting uninstall: requests\n",
            "    Found existing installation: requests 2.27.1\n",
            "    Uninstalling requests-2.27.1:\n",
            "      Successfully uninstalled requests-2.27.1\n",
            "\u001b[31mERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts.\n",
            "google-colab 1.0.0 requires requests==2.27.1, but you have requests 2.31.0 which is incompatible.\u001b[0m\u001b[31m\n",
            "\u001b[0mSuccessfully installed backoff-2.2.1 chromadb-0.3.26 clickhouse-connect-0.6.3 coloredlogs-15.0.1 fastapi-0.97.0 h11-0.14.0 hnswlib-0.7.0 httptools-0.5.0 humanfriendly-10.0 lz4-4.3.2 monotonic-1.6 onnxruntime-1.15.1 overrides-7.3.1 posthog-3.0.1 pulsar-client-3.2.0 python-dotenv-1.0.0 requests-2.31.0 starlette-0.27.0 tokenizers-0.13.3 uvicorn-0.22.0 uvloop-0.17.0 watchfiles-0.19.0 websockets-11.0.3 zstandard-0.21.0\n",
            "Looking in indexes: https://pypi.org/simple, https://us-python.pkg.dev/colab-wheels/public/simple/\n",
            "Collecting openai\n",
            "  Downloading openai-0.27.8-py3-none-any.whl (73 kB)\n",
            "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m73.6/73.6 kB\u001b[0m \u001b[31m3.7 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[?25hRequirement already satisfied: requests>=2.20 in /usr/local/lib/python3.10/dist-packages (from openai) (2.31.0)\n",
            "Requirement already satisfied: tqdm in /usr/local/lib/python3.10/dist-packages (from openai) (4.65.0)\n",
            "Requirement already satisfied: aiohttp in /usr/local/lib/python3.10/dist-packages (from openai) (3.8.4)\n",
            "Requirement already satisfied: charset-normalizer<4,>=2 in /usr/local/lib/python3.10/dist-packages (from requests>=2.20->openai) (2.0.12)\n",
            "Requirement already satisfied: idna<4,>=2.5 in /usr/local/lib/python3.10/dist-packages (from requests>=2.20->openai) (3.4)\n",
            "Requirement already satisfied: urllib3<3,>=1.21.1 in /usr/local/lib/python3.10/dist-packages (from requests>=2.20->openai) (1.26.15)\n",
            "Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python3.10/dist-packages (from requests>=2.20->openai) (2022.12.7)\n",
            "Requirement already satisfied: attrs>=17.3.0 in /usr/local/lib/python3.10/dist-packages (from aiohttp->openai) (23.1.0)\n",
            "Requirement already satisfied: multidict<7.0,>=4.5 in /usr/local/lib/python3.10/dist-packages (from aiohttp->openai) (6.0.4)\n",
            "Requirement already satisfied: async-timeout<5.0,>=4.0.0a3 in /usr/local/lib/python3.10/dist-packages (from aiohttp->openai) (4.0.2)\n",
            "Requirement already satisfied: yarl<2.0,>=1.0 in /usr/local/lib/python3.10/dist-packages (from aiohttp->openai) (1.9.2)\n",
            "Requirement already satisfied: frozenlist>=1.1.1 in /usr/local/lib/python3.10/dist-packages (from aiohttp->openai) (1.3.3)\n",
            "Requirement already satisfied: aiosignal>=1.1.2 in /usr/local/lib/python3.10/dist-packages (from aiohttp->openai) (1.3.1)\n",
            "Installing collected packages: openai\n",
            "Successfully installed openai-0.27.8\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "## Import libraries"
      ],
      "metadata": {
        "id": "I-BlQeoaKGvG"
      }
    },
    {
      "cell_type": "code",
      "execution_count": 2,
      "metadata": {
        "id": "RyBfYgKpNzuE"
      },
      "outputs": [],
      "source": [
        "#libraries for google drive authentication\n",
        "from pydrive.auth import GoogleAuth\n",
        "from pydrive.drive import GoogleDrive\n",
        "from google.colab import auth\n",
        "from oauth2client.client import GoogleCredentials\n",
        "\n",
        "import whisper\n",
        "import torch\n",
        "import os\n",
        "from pytube import YouTube\n",
        "from langchain.text_splitter import CharacterTextSplitter\n",
        "from langchain.document_loaders import DataFrameLoader\n",
        "from langchain.vectorstores import Chroma\n",
        "from langchain.chains import RetrievalQAWithSourcesChain\n",
        "from langchain.embeddings.openai import OpenAIEmbeddings\n",
        "from langchain.llms import OpenAI\n",
        "import pandas as pd"
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "# Import the libraries\n",
        "\n",
        "# Set the device\n",
        "device = \"cuda\" if torch.cuda.is_available() else \"cpu\"\n",
        "\n",
        "# Load the model\n",
        "whisper_model = whisper.load_model(\"large\", device=device)"
      ],
      "metadata": {
        "id": "yQmc9_K4OXPu",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "d0b69f7c-a256-496d-d030-d6b4e8626a7c"
      },
      "execution_count": 3,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "100%|██████████████████████████████████████| 2.87G/2.87G [00:27<00:00, 111MiB/s]\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "# Data Acquisition\n",
        "\n",
        "## Getting Video Data from YouTube\n",
        "In this section, we'll discuss how to input a YouTube video URL and use it to extract the video data. This involves using a YouTube data extraction library to access and download the video.\n",
        "\n",
        "## Extracting Audio from YouTube Video\n",
        "After obtaining the video, the next step is to extract the audio which will be transcribed into text. We'll discuss the method used to perform this extraction and the format in which the audio data is saved.\n",
        "\n",
        "# Transcription using Whisper\n",
        "\n",
        "## Introduction to Whisper\n",
        "Whisper is OpenAI's automatic speech recognition (ASR) system. In this section, we'll provide a brief introduction to Whisper and explain how it is used to transcribe the audio from our YouTube video.\n",
        "\n",
        "## Transcribing Audio to Text\n",
        "Here, we'll walk you through the process of transcribing the extracted audio into text using Whisper. This involves sending the audio data to the Whisper API and receiving a text transcript in return."
      ],
      "metadata": {
        "id": "qfXYH59i3oMY"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "## Extract the audio from youtube video"
      ],
      "metadata": {
        "id": "EuqnkSwXLHNq"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "def extract_and_save_audio(video_URL, destination, final_filename):\n",
        "  video = YouTube(video_URL)#get video\n",
        "  audio = video.streams.filter(only_audio=True).first()#seperate audio\n",
        "  output = audio.download(output_path = destination)#download and save for transcription\n",
        "  _, ext = os.path.splitext(output)\n",
        "  new_file = final_filename + '.mp3'\n",
        "  os.rename(output, new_file)"
      ],
      "metadata": {
        "id": "PNN0oi6wiW0o"
      },
      "execution_count": 4,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "# Video to audio\n",
        "video_URL = 'https://www.youtube.com/watch?v=EdlWxfDLhwQ'\n",
        "destination = \".\"\n",
        "final_filename = \"ICIT 2023 - Opening Keynote\"\n",
        "extract_and_save_audio(video_URL, destination, final_filename)"
      ],
      "metadata": {
        "id": "QSwh2yMrLZV4"
      },
      "execution_count": 5,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "## Transcribe"
      ],
      "metadata": {
        "id": "716kqAfWLS8u"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "# run the whisper model\n",
        "# audio_file = \"ICIT 2023 - Opening Keynote.mp3\"\n",
        "# result = whisper_model.transcribe(audio_file)"
      ],
      "metadata": {
        "id": "avu00PEtdJjd"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "This is a 1 hour video file, it takes around 10 minutes to complete the transcription. For the time being, a pre-transcribed version is being loaded from the disk."
      ],
      "metadata": {
        "id": "7FkY8A1AdJqz"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "#authenticate with you google drive credentials\n",
        "auth.authenticate_user()\n",
        "gauth = GoogleAuth()\n",
        "gauth.credentials = GoogleCredentials.get_application_default()\n",
        "drive = GoogleDrive(gauth)\n",
        "\n",
        "# This is the file ID of the data set, this will download the datafile from the shared location\n",
        "transcription_id = '1bm20FzcRWfOkGBEIJHId2cT-8yuJ0HCY'\n",
        "transcription_data = drive.CreateFile({'id':transcription_id})\n",
        "transcription_data.GetContentFile('transcription.csv')"
      ],
      "metadata": {
        "id": "2yjFdMojcth7"
      },
      "execution_count": 6,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "## Chunk Clips"
      ],
      "metadata": {
        "id": "c7UsxaG9XpXa"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "transcription = pd.read_csv('transcription.csv')\n",
        "\n",
        "def chunk_clips(transcription, clip_size):\n",
        "  texts = []\n",
        "  sources = []\n",
        "  for i in range(0,len(transcription),clip_size):\n",
        "    clip_df = transcription.iloc[i:i+clip_size,:]\n",
        "    text = \" \".join(clip_df['text'].to_list())\n",
        "    source = str(round(clip_df.iloc[0]['start']/60,2))+ \" - \"+str(round(clip_df.iloc[-1]['end']/60,2)) + \" min\"\n",
        "    print(text)\n",
        "    print(source)\n",
        "    texts.append(text)\n",
        "    sources.append(source)\n",
        "\n",
        "  return [texts,sources]\n",
        "\n"
      ],
      "metadata": {
        "id": "03XK_EkvNFRY"
      },
      "execution_count": 7,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "chunks = chunk_clips(transcription, 50)\n",
        "documents = chunks[0]\n",
        "sources = chunks[1]"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "MH7FhnoeYIlI",
        "outputId": "56a0ca43-8e45-4316-8c03-9b50bfaaddd2"
      },
      "execution_count": 9,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            " Good morning, everybody.  We have a tight schedule, so we need to get going.  So welcome to the 24th edition of ICIT,  International Conference on Industrial Technology.  This is one of our flagship conferences.  My name is Milos Manic, and I am the general chair  of the conference, also president-elect.  My bills are paid by Virginia Common University,  where I'm a professor and director of VCU Cybersecurity  Center.  I also have affiliation with my Idaho National Lab.  As you may have noticed, I change universities,  but never labs.  IEEE IES society is one of the most internationally diverse  society.  Our society has members from all continents,  many, many countries.  It's one of the most diverse in IEEE, and one of the oldest.  Just a few years back, we were celebrating 70 years  of industrial society.  At this conference, we managed to collect about 130 submissions,  accepted over 100 papers, and ended up  with a very unique program.  I would dare to say that ICIT is one of the first,  in many ways.  You will notice the main theme of cybersecurity.  Here, we are running the Capture the Flag competition,  and it's run online internationally.  All around the world, we have teams,  and the winners will be announced tomorrow, I believe.  Paolo, Victor, who else is running, please wave.  Yes.  We have a number of interesting activities  that include, but are not limited to,  industry forum, panels, diversity and inclusion,  students and young professionals.  I'd like to recognize, I'd like to try  to recognize as many people as possible.  It's going to be tough, but our honorary chairs  are distinguished past presidents of the society  and distinguished eminent academics, Kohei, Leopoldo,  Mo Yan, Vrenloo, Makoto, Xing, and Dave Irwin.  General chairs, besides me, Kamal Al-Khaddad,  well-known figure in our society and around the world.  Every conference has its tough times  when you have to make tough decisions,  and they're tied to finance, timing, and everything.  And Kamal is the one that comes and brings thinking out  of the box and get us out of the trouble.  So thank you very much, Kamal.\n",
            "0.0 - 3.56 min\n",
            " I'd like to recognize Louise, the lead of Technical Program  Committee.  Louise is from beautiful city of Lisbon, Lisboa,  in Portugal, and brings the centuries-old wisdom  from Portugal.  When we get into a tough spot, Louise just takes some time,  thinks a little bit, and proposes a solution  that we usually end up going with.  So thank you very much, Louise.  Other program chairs that contributed to this conference  are Yang Shi, VP for Conferences, Huijun Gao,  Sheldon Williamson, Makoto Iwasaki, and Dasvin De Silva.  Thank you, Dasvin.  Dasvin is also running Zoom business behind the scene.  Treasurer Oleg Malinovsky, as many others  have gotten delayed.  He's at some airport and should be joining us later.  Publicity chairs Morgan Kiani, Leila Parsenherb-Hass.  Tutorial chairs Maria Valla, Antonio Luque, Tilo Sauter.  Antonio is here.  I saw him somewhere.  Industry forum and panel, Stamatis.  Stamatis, I saw you somewhere here.  Stamatis over there.  Victor Huang sitting here.  Dietmar Bruckner.  OK, I saw him last night.  He's joining us shortly.  Industry exhibition, Michael Condry,  student-ranked young professionals, Marek Yashinsky.  Marek is there.  Mihoko Onitsuma, Haniba Hedi, Sandun Mavikumbure,  Victor Kobelian.  Diversity of inclusion, this is one of the first also.  This is our first conference where  we are organizing diversity and inclusion workshop.  Please put in your calendars and join the workshop  on Thursday.  So diversity and inclusion will be happening on Thursday.  There's a packed program, very interesting speakers.  Please join.  Diversity inclusion is led by Lucila Lobello.  Lucia has arrived but probably enjoying breakfast.  Mariam Seydfard, Morgan Kiani, Regina Ross,  Aphrodite goes by Vanni Philippas.  And we have international advisory board  of Kiraki Hashimoto, Ramu Krishnan, OK, Kainak, OK,  somewhere here.  I saw him, Kinlong Han, Gerhard Hanke for Hong Kong,  Sudip, Hamas Zunder, Herb Hess, and Terry Martin.\n",
            "3.56 - 6.3 min\n",
            " So we ended up with about 110 papers, 14 special sessions.  Even for a conference, smaller conference of this size,  you end up with about 500 people working as reviewers  or taking the role of authors and so on.  For this conference, we had easy 100 plus volunteers.  And this is one thing that our society is proud of.  It is run by volunteers.  Everything is done by volunteers.  Volunteers create programs, volunteers  create new initiatives, volunteers propose new  projects, and so on.  So what I'm really proud of is that we  are running two panels with colleagues from national labs.  The first one will be right after the keynote  and the second one will be on Thursday.  So I am really happy to see Iroquois National  Lab and Oak Ridge National Lab taking  the lead in activities at this conference.  And I hope we continue and just continue  increasing that involvement.  So thank you very much to both.  Last but not least, one of the first that I see  is a strategic workshop.  We call it Vision 2030.  Many things have changed over the years and we adapted.  And now is a good time to get together and discuss  path forward and plan our beautiful future together.  So I'd like to recognize finance committee where  this idea workshop started.  Mihoko, Marshan, Kamal, Dietmar, and the officers.  Can I ask officers to stand up?  Here is Tilo Sauter, Vienna Academy of Sciences,  Stamatis, SAP, Germany slash Greece,  Antonio Sevilla, VP membership, Stoyan,  VP planning and development.  We have a couple more that will be joining us later.  Now I'd like to ask our president, Mariusz Malinowski,  to just say a couple of words.  And after that, we'll move on with the keynote.  Thank you very much, Mariusz.  Thank you very much all.  Thank you very much.  This is really a couple of words because many of things  were said by Milosz.  So the only one person which was not thanked,  we have to thank Milosz for organizing this conference.  Please, applause for Milosz.  Yes, thank you very much for invitation  for this very beautiful place, Orlando.  We can enjoy the nice weather.\n",
            "6.3 - 9.48 min\n",
            " I think the whole year it's nice weather here.  So I hope that taking account this very good circumstances,  this is a good time to have a fruitful conference.  So I hope that for conference participants,  this is a good occasion to have interesting discussions,  maybe to start the collaboration between the universities,  between the researchers.  But parallelly, this is organized  this strategic workshop, which is, I think,  very important for our society.  Of course, I have some of the data  which I can show you later on starting  this strategic workshop, showing that position of our society  is strong.  I think the financial is strong.  It takes on the journals.  It takes off the membership because we almost  reached 10,000 in the end of 2022,  taking account the technical activity,  as well as also another field of including the students,  women engineering, and so on.  But the question is, without any honest discussion  about the future, how we are thinking about our society  being in 2030 or maybe in 2040, we  are not able to keep our good position continuously.  Therefore, I think that we need this discussion, especially  to improve our operations in the short term  and also to have some of the strategic goals for the 2030.  So thank you very much for coming.  I know that it was not so easy to come, especially  during the close to the Easter time.  But I really believe that this workshop, this meeting,  it's crucial for our society for the next year.  So thank you very much.  Thank you very much.  Now the time is to go ahead with our first keynote.  I would like to invite Paul Titan to introduce  our eminent speaker.  OK.  Paul, thank you very much.  Thank you, Milos.  And thank you, everybody, for being here today.  My name is Paul Titus.  I'm from the Idaho National Laboratory.  It was very easy for us to come here.  There is a significant weather change.  Right now, there is quite a bit of snow still in Idaho.  It's melting.  But here, it was beautiful to see no snow.  As many of you know, INL is a leading national laboratory\n",
            "9.48 - 12.18 min\n",
            " in the areas of cybersecurity and 5G.  Today, we have an excellent panel  following the keynote speech discussing  the transformational role of 5G in the fourth Industrial  Revolution, also known as Industry 4.0.  We will have a special session tomorrow  with seven papers from our research collaborators  on this topic.  INL appreciates its active research collaboration  with VCU through Professor Milos Manic,  and certainly looking forward to our continued collaboration  with the academia to make secure use of 5G  and future wireless technologies.  It's my privilege now to introduce our keynote speaker,  Ms. Valerie M. Coffield.  She serves as the Chief Strategy Officer  of the Cybersecurity and Infrastructure Security Agency,  CISA.  Ms. Coffield serves as the principal policy  and strategic advisor to CISA leadership  and senior management, integrating strategy  across all the nation's organizations,  all the nation's organizations' mission areas,  and ensuring policy, strategy, and operational consistency  throughout the agency.  Prior to CISA, Ms. Coffield served  at the Federal Bureau of Investigations  for 22 years in a variety of roles.  She was the Deputy Assistant Director  for the Cybersecurities Branch within FBI Cyber Division,  where she led coordination and deployment of the division's  technical tools and capabilities  and oversaw cyber-related training,  recruiting, and hiring and budgeting for the division.  She also served in a senior executive role  as the Chief Staff of the Science and Technology Branch  as the Deputy Assistant Director  of the Digital Transformation Office,  where she engaged with interagency partners.  With that said, I now introduce,  I now turn the time over to Ms. Coffield.  Thank you.  Thank you so much.  And thank you so much for inviting me to speak here today  at the 24th IEEE International Conference  on Industrial Security.  And I really thank you for that kind introduction, Paul.  I'm jealous.  I'm hoping you're enjoying the great time of year  in sunny Florida.\n",
            "12.18 - 14.71 min\n",
            " I really wish I could be there in person with you.  Nevertheless, I'm excited to be addressing all of you today  about the importance of 5G security and resilience.  And as Paul mentioned, my name is Val Coffield  and I work for the Cybersecurity  and Infrastructure Security Agency, known as CISA,  and I'm the Chief Strategy Officer there.  And in my work as a Principal Policy  and Strategic Advisor to CISA Leadership and Management,  I'm charged with integrating strategy  across all of the organization's mission areas  and ensuring that the policy and strategy  and that there's operational consistency  throughout the entire agency.  Since I work in policy and strategy at a national level,  you can see why 5G's powerful capabilities  and inherent risks are extremely important to me and my job.  5G or the fifth generation mobile network  represents a complete transformation  of telecommunications networks,  introducing a wealth of benefits that will pave the way  for new capabilities and support connectivity  for applications like smart cities,  autonomous vehicles, and telemedicine.  Roughly every 10 years,  the next generation of mobile communication network  is released, bringing faster speeds  and increased capabilities.  The first generation or 1G of wireless networks  brought the very first cell phones,  2G brought improved coverage and texting,  3G introduced voice with data and internet,  and 4G long-term evolution delivered increased speeds  to keep up with mobile data demand.  And then there is 5G.  We know that 5G will transform the digital landscape  and serve as a catalyst for innovation,  new markets and economic growth.  As tens of billions of devices  are connected to the internet through 5G,  these connections will empower a vast array  of new and enhanced critical infrastructure services.  5G builds upon existing telecommunication infrastructure  by improving the bandwidth, capacity,  and reliability of wireless broadband services.  The goal is to meet increasing data  and communications requirements,  including the capacity for tens of billions  of connected devices that will make up  the internet of things.\n",
            "14.71 - 17.1 min\n",
            " Ultra low latency required  for critical near real-time communication  and faster speeds to support emerging technologies.  Now, I know that I don't really need to explain  the capabilities and history of 5G to this group.  So let me just give you an overview  of how CISA is involved in the risk management of 5G.  As new technologies both directly  and indirectly connect to 5G emerge,  CISA assesses that 5G will present opportunities  and challenges as its implementation  will introduce vulnerabilities related to supply chain,  deployment, network security,  and the loss of competition and trusted options.  5G's emphasis on software-defined networking  and the aggregation of the network  could potentially make these networks  more vulnerable to security threats,  particularly those originating  from a supply chain compromise.  At CISA, we look at 5G as a process of risk management.  In other words, we want to identify, analyze, assess,  and communicate the risks by accepting, avoiding,  transferring, or controlling it to an acceptable level  considering associated costs  and benefits of any actions taken.  Effective risk management improves  the quality of decision-making.  While risk often cannot be totally eliminated,  actions can usually be taken to control risk.  As you all may know or may not know,  CISA is a very new agency.  It was formally established in 2018  as America's cyber defense agency under the DHS umbrella.  We were established to serve as the national coordinator  for critical infrastructure, security, and resilience.  And we lead efforts to understand, manage,  and reduce both cyber and physical risks.  We do this by maintaining public-private partnerships  that are dedicated to sharing risk information,  working with and through our regional experts  who are responsible for onsite vulnerability assessments  and partnering with owners and operators to mitigate risk.  And we participate in national  and international level dialogues  to learn more about threats and share lessons learned.  CISA also works with government and industry  to identify, analyze, prioritize,  and manage the most significant strategic risks  to the nation's critical infrastructure.\n",
            "17.1 - 19.66 min\n",
            " Since the nation's critical infrastructure  is largely owned and operated by the private sector,  managing risks is a shared priority.  CISA creates an environment where the private sector,  government agencies, and key stakeholders  can collaborate, share expertise,  and coordinate risk reduction activities  to ensure critical infrastructure is secure and resilient  now and into the future.  In today's digitizing world,  as organizations are increasingly integrating  cyber systems into their operations,  they are also facing more diverse and sophisticated threats.  Cyber, physical, technological, or natural,  they may have cross-sector impacts.  The evolving risk landscape necessitates  an evolved response.  Because we lead the national effort  to understand and manage risk,  the rollout of 5G has been a focus area of ours.  Why?  Because the use of 5G components  manufactured by untrusted companies  could expose US entities to risks introduced  by malicious software and hardware, counterfeit components,  and component flaws caused by poor manufacturing processes  and maintenance procedures.  5G hardware, software, and services  provided by untrusted entities  can increase the risk of compromise  to the confidentiality, integrity,  and availability of network assets.  Even if US networks are secure,  US data that travels overseas  through untrusted telecommunications networks  is potentially at risk of interception,  manipulation, disruption, and destruction.  The effectiveness of 5G security enhancements  will in part depend on proper implementation  and configuration.  Despite security enhancements over previous generations,  it is unknown what new vulnerabilities  may be discovered in 5G networks.  Further, 5G builds upon previous generations  of wireless networks,  and will initially be integrated  into 4G long-term evolution networks  that contain some legacy vulnerabilities.  Untrusted companies may be less likely  to participate in interoperability efforts.\n",
            "19.66 - 21.96 min\n",
            " Custom 5G technologies  that do not meet interoperability standards  may be difficult to update, repair, and replace.  This potentially increases the lifecycle cost of the product  and delays 5G deployment  if the equipment requires replacement.  The US has placed an increased emphasis  on ensuring the security and integrity of 5G.  For example, security officials from over 30 countries,  including the US, met in Prague in May of 2019  to discuss guidelines  for securing the 5G network deployment.  The results of the engagement  was the deployment of the Prague proposals,  which identified recommendations  in four distinct categories in preparation of 5G rollout.  The first category was in policy,  the second in technology,  the third in economy,  and last in security, privacy, and resilience.  Building upon the recommendations  established by the Prague proposals,  the US developed the National Strategy to Secure 5G,  a strategic document that supports the goals  of the recently released National Cybersecurity Strategy  and expands on how the US government  will secure 5G infrastructure domestically and abroad.  And what's CISA doing about it?  Well, at CISA specifically,  we are leading the 5G risk management efforts  and helping shape the rollout  of this emerging critical infrastructure  so that the US can fully benefit  from all the advantages 5G connectivity promises to bring.  In doing so, we are dedicated to working  with our interagency, industry, and international partners  to manage the accompanying risks and challenges  to 5G implementation appropriately,  increasing its security and resilience at the design phase  and reducing national security risk  from an untrustworthy 5G network.  Through this approach,  we aim to ensure that there are policy, legal,  security, and safety frameworks in place  to fully leverage 5G technology  while managing its significant risks.  Our approach is also guided  by a series of strategic initiatives,  including supporting 5G policy and standards developments  by emphasizing security and resilience.\n",
            "21.97 - 24.35 min\n",
            " To prevent attempts by threat actors  to influence the design and architecture of 5G networks,  it is critical that 5G policies and standards  serve as the foundation for safe  and secure 5G implementation.  CISA is actively engaging with partners  in the promotion and development  of security best practices, standards,  and federal policies that remove  five untrustworthy 5G components from US networks.  Expanding situational awareness of 5G supply chain risks,  and we're also promoting security measures.  Between untrusted components,  vendors, equipments, and networks,  5G supply chain security is under constant threat.  CISA is collaborating with interagency,  industry, and international partners  to evaluate, prioritize, and communicate  5G supply chain risks to 5G infrastructure.  We're partnering with stakeholders  to strengthen and secure existing infrastructure  to support future 5G deployments.  First iterations of 5G deployments  are working alongside existing 4G  long-term evolution infrastructure and core networks.  This has the potential for malicious attacks  to exploit known vulnerabilities.  CISA is working with industry and government  to develop and communicate security enhancements  to support secure 5G deployments.  Where possible, leveraging research  and development capabilities that address 5G security  and vulnerability challenges.  We're encouraging innovation in the 5G marketplace  to foster trusted 5G vendors.  To foster a market of trusted 5G vendors,  CISA is working with its partners to communicate risk,  promote best practices, and establish methods  for evaluating the security of 5G vendors.  By communicating risk and vulnerabilities  to 5G components and technologies,  CISA will shape the 5G technological ecosystem,  maintaining the US's position as a global leader  in 5G deployment and creating a market environment  that prioritizes security and resilience.  And last but not least,  we are analyzing potential 5G use case  and sharing information  on identified risk management strategies.  To ensure the security and integrity of 5G,\n",
            "24.37 - 26.76 min\n",
            " CISA will communicate known vulnerabilities  and risk management strategies for use cases  associated with securing the nation's critical functions.  Specifically, the use of Internet of Things deliverables  outlined in the Botnet Report and the Council  to secure the digital economy IoT baseline.  At CISA, we have three principles  to support these strategic initiatives.  And these principles are focusing  on three core competencies.  First, risk management,  promoting secure and resilient 5G deployment  by leading efforts to identify, analyze,  prioritize, and manage risk.  Second, stakeholder engagement,  actively engaging federal, state, local,  tribal, and territorial industry associations,  academia, nonprofit, and international partners  to address 5G challenges.  And we also are providing lastly, technical assistance,  updating and developing instructional tools and services  to support stakeholders with the planning,  governance, operational, and technical aspects  of secure 5G deployments.  I wanna take a little bit of time  to talk about CISA's national critical functions.  5G will also support a host  of our national critical functions,  including autonomous vehicles, smart electric grids,  intelligent medicine, and military communications.  In other words, 5G will support functions  of the government and private sector out of many  that we deem so vital that their disruption,  corruption, or dysfunction would have a debilitating impact  on security, national economic security,  national public health or safety,  or any combination thereof.  In April of 2019, we published the initial set  of national critical functions,  which has since been completed by definition,  and since we have completed definitions  for each of the functions.  Obviously, our national critical functions  are increasingly relying on 5G components,  which is why securing these components  and emerging technology is of utmost importance.  If you aren't familiar  with our national critical functions or NCF efforts,  it was an initiative undertaken by our agency  in collaboration with government and industry partners\n",
            "26.76 - 29.2 min\n",
            " associated with all  of the 16 critical infrastructure sectors,  state, local, and tribal partners, and other stakeholders.  The set of NCFs are organized into four areas,  connect, distribute, manage, and supply,  which identify the connections by technologies  that enable critical communications and capabilities  to send and receive data,  like internet connectivity, like 5G.  Distribution methods that allow the movement of goods,  people, and utilities inside and outside the United States.  For example, electrical distribution  or cargo transportation.  Management processes that ensure our national security  and public health and safety,  like the management of hazardous materials  or national emergencies,  and the supply of materials, goods, and services  that secure our economy.  For example, clean water, housing,  and research and development.  The NCFs allow for a more robust prioritization  of critical infrastructure  and a more systematic approach to critical infrastructure  and a more systematic approach  to corresponding risk management activity.  They represent an evolution  to the critical infrastructure risk management framework  established in the National Infrastructure Protection Plan.  While the previous approach focused almost entirely  on entity level risk management,  as opposed to critical outcomes,  the NCF approach enables a richer understanding  of how entities come together to produce critical functions  and what assets, systems, networks, and technologies  underpin those functions.  By viewing risk through a functional lens,  we can ultimately add resilience and harden systems  across the critical infrastructure ecosystem  in a more targeted, prioritized, and strategic manner.  This allows for more holistically capturing  cross-cutting risks and associated dependencies  that may have cascading impacts within and across sectors.  I also wanna talk about the supply chain a little bit.  As I highlighted that we examined 5G  and other emerging technologies  impacting our national critical functions,  we realize that there is a lot of risk out there,  and to effectively manage these risks,  we need to prioritize our approach.\n",
            "29.2 - 31.62 min\n",
            " In these efforts, CISA has redoubled its focus  on working with the private sector,  developers, and international partners  to prioritize national security  in their decision-making process,  while also initiating partnerships  and programs to foster innovation.  We want to quit being reactive  to products that pose real and potential threats  and become more proactive  by stopping those potentially harmful,  overtly vulnerable products  from being developed in the first place.  My agency is especially concerned  about the cyber and data risks  from information and communications technology products  designed, developed, manufactured, or supplied  by companies operating under the control  or influence of a foreign government  whose national security interests  are averse to the United States.  We all know that there is a certain level of risk  with any technology that generates  or collects sensitive data,  such as industrial control systems,  but that risk increases dramatically  when the technology comes from a company  that could be persuaded or forced to abuse their access  on behalf of a foreign government  that does not share our constitutional norms and values  or operates without any meaningful checks  on its abilities to compel cooperation  with its intelligence services.  The United States government has serious concerns  about technologies that take American data  into the territory of a government that allows  or directs its intelligence services  to open the hood and peek around  and do whatever they want with that data.  Those concerns and that pragmatic worldview  are what led us to develop the framework  that I want to share with you today.  We are looking at supply chain risk management  in the context of procurement or acquisition  for our nation's supply chain.  We are looking at three interconnecting pieces  and that's the what, the where, and the who.  First, the what.  For this, we are looking at the technical aspects  of a product or service.\n",
            "31.62 - 33.71 min\n",
            " In doing so, we are examining a variety of concerns.  For example, we want to identify what functions  the product or services perform and how it operates,  whether the product receives software updates  from the supplier,  and whether there are technical mitigations  that could be instituted to detect malicious use  of the product, just to name a few.  Second, the where.  This is a legal piece.  When examining the legal piece of a technology or product,  it's important to ask several questions  about the country of origin.  For example, what extent do the foreign government's laws  or policies permit it to compel cooperation  with its intelligence activities?  Is there a meaningful independent judicial review?  And more broadly, what are the foreign government's  national security interests as they relate to your country?  And third, the who.  For this, we are looking at the relationship  a company may have with a foreign government.  It is imperative that we examine  whether a foreign government,  whether directly or indirectly,  holds a financial stake in the company.  It is also important to look at the extent  to which a company may be influenced  or susceptible to coercion.  We all know the saying,  the cybersecurity chain is only as strong  as its weakest link.  I believe this framework will help us ensure  that our country's supply chain excludes that weak link.  And I welcome you to think about these three questions  as you consider your organization's supply chain  and the integration of 5G into your networks.  I wanna shift a little bit  and touch on the importance of workforce.  A long-term and one of the most relevant goals for CISA  is to secure a more resilient future  for the American people  and accelerate our technological capabilities  by growing our national cybersecurity workforce.  Bottom line, as technologies are advancing,  like it has been in the realm of 5G,  we need our education and workforce  to keep up with these advances.  It's happening fast,  which is why this is one of CISA's top priorities.\n",
            "33.71 - 35.91 min\n",
            " This is why we are working with the academic sector  to promote cyber education for K through 12  and higher education students.  We are also growing the pipeline for future talents.  We want to ensure that we have a workforce  with a diverse way of thinking  about our unique cyber challenges,  like keeping 5G secure into the future.  And there's still approximately 700 open  cybersecurity positions currently in the US.  There are so many ways to grow the cybersecurity workforce  and CISA has joined forces with entities like cyber.org  and the Girl Scouts of the USA  to get students thinking about careers  in cybersecurity earlier,  just like if they wanted to be a doctor,  a teacher, or a lawyer.  I know our time together here is short.  So before I conclude, I want to take a step back  and provide a quick repap on where we are  in terms of the 5G rollout  and where we might be headed in the future.  Just imagine in the next 10 years,  will we learn the lessons of underinvestment  in 5G telecommunications technology  and ultimately dominate the 6G market?  Or will we cede to reality  in which authoritarian regime linked firms  become the center of gravity for innovating connectivity,  patenting their technologies  and progressively setting the telecom agenda of the future?  We will only succeed in fully securing 5G  if we invest aggressively in our talent  and in our education system,  if we invest in our alliances globally  for peace and freedom across governments,  as well as with the private sector,  and only if we stay true to the values  that bind us together  for collective security and prosperity.  These are not insurmountable challenges,  but they are by no means assured.  I would submit that technological innovation of the future  is no longer a guarantee of peace and prosperity,  but it's at its core.  To secure the independence of our nations,  the partnership and mutual respect we have for one another  and continued global cooperation,  technological development,  secure technological development\n",
            "35.91 - 38.12 min\n",
            " is a no-fail mission in and of itself.  Ultimately, our goal at CISA  is to strengthen the global critical infrastructure community  so that we are faster and smarter than our adversaries.  We wanna make sure that by continuing  to deepen collaboration across industry  and the international community,  we will raise the cost, time, and complexity thresholds  for successful attacks to the point  that they exceed the capabilities  of even the most advanced threat actors.  This must be a priority  with securing our most critical functions,  including, of course, the 5G networks.  Lastly, I wanna mention that 2022  was a banner year for 5G.  It was when most carriers switched over to the 5G network.  However, even now in 2023,  5G has faced some deployment challenges,  which is why the rollout continues in many aspects  all over the world.  Thank you again for inviting me  to be a part of such a wonderful conference,  and I'm happy to take any questions you might have now.  Thank you.  Ms. Caulfield, Valerie, thank you so much for your comments.  I'd like to, anybody out in the audience have any questions?  Arup?  Arup?  Thank you again so much, CSO Caulfield.  I think you gave us such a nice summary,  very clear picture of what your agency is focusing on.  I think I just want to check your thoughts  on one particular aspect of global collaboration.  And I think, as you know, the 5G, you're already doing it.  You mentioned the Prague meeting,  and you mentioned that in addition to securing  this critical infrastructure in the US,  also important to security overall architecture globally,  because everything is global right now.  Do you have any particular thoughts?  I'm looking at this conference,  it's an international academic conference mostly.  And I do think that collaborating,  in continuing the collaboration on the security aspects  of 5G with our partner countries and with the academia  would be really very strong input to your process, right?  Any particular thoughts on that?  No, I think that the academic community  plays a very important role.\n",
            "38.12 - 41.03 min\n",
            " And organizations like IEEE also obviously play  a tremendously important role,  because we believe that the standards that should control  and the organizing principles around these networks  need to be through these international communities,  these collaborative communities,  and not necessarily dictated by a single country's  perspective.  We feel like with that international collaboration  and the transparency that exists  in those types of standards,  that the standards that are developed and produced  through those type of organizations  are stronger and more secure.  And so, we want to continue to collaborate.  I know CISA, as well as many of my interagency partners  across the US government, oftentimes collaborate  and will continue to collaborate both with academia  and with our international partners.  Since as you mentioned,  this is such an international and global issue,  and that we also are working on trying to secure  more capability building across the nations,  working with like-minded countries  to be able to provide alternatives,  especially for the developing countries  where they might not be able to afford different options.  Well, thank you very much indeed  for your great presentation.  By the way, I'm not a communication guy,  neither am I a security guy,  but I have some comments on what you have presented  very nicely.  One is I was a bit surprised  that you had 700 jobs still open.  That seemed too little to me, right?  Maybe you mean right at the very top level, right?  Because I think every company needs a security guy,  you know, so really we need to educate them, we professors.  The second thing is just an experience.  We have to educate the public too.  It is just an experience,  maybe a bit could be relaxing if I tell you what it is.  I took a taxi from the airport to the hotel  and the taxi driver, I've tried to push in my car  through the credit card, you know, thing.  And he said, it doesn't work, it doesn't work.  And I said, what should I do?  So he gave me an iPhone with it, you know,  everything, everything, and he wanted to push it through.\n",
            "41.03 - 43.7 min\n",
            " And we had a great argument that I said, this is not secure.  I don't know what there is in that iPhone, right?  But in the end, I had to pay through that iPhone.  Thank you very much.  Are there any other questions?  Yes, sir.  So again, yes, I'm Wes Withrow.  Again, appreciate your time today.  And thank you, thank you for your presentation.  You know, one of the things that like in the world  that we're living in right now and talking about,  you know, zero trust and zero trust architecture, right?  And know that CIS is just getting ready to release  kind of the new version 2.0  of the zero trust maturity model as well too.  But in your time working  with our international coalition of partners,  have you seen, you know, the discussions  or had discussions with internationally  about the adoption of the principles of zero trust,  knowing that it's still very early in this country, right?  You know, we're kind of on our own five yard line  getting started with it.  But again, have you seen  and had discussions internationally about zero trust?  Oh, no, I think that there is, you know,  interest in zero trust.  I mean, most of our collaboration internationally  starts with a lot of our, you know,  our close allies like our Five Eyes partners.  And I do think that there is interest there in moving out.  And a lot of times, you know,  the US is a leader in security standards.  And so as we move out,  I think that there's very likelihood  that other of our allied, close allied partners  will start to do the same as well.  So I do think there's interest in moving in that direction.  Thank you.  Thank you.  Thank you for a great presentation.  My name is Regina Roos and I'm coming from Germany.  So I always like to hear what the other part of the world  is thinking in respect of this topic of cybersecurity.  However, I feel we have a big homework,  which is related to education.  And I'm wondering when globally the school systems  in primary school and kindergarten  will start to put cybersecurity on the agenda  because only then we can start\n",
            "43.7 - 46.2 min\n",
            " bringing the right mindset forward.  What are the planes in your organization  to support it as a social project?  No, thank you for that question.  I totally agree that we need to,  and we are trying, as I mentioned in my earlier speech,  to focus in on education at all levels.  So I agree with you that we need to start early.  And so we have been working with organizations  like co.org here in the US  to help with K through 12 curriculum,  but we're also working with universities  and helping to improve and increase cybersecurity education  at the university level as well.  For us, I know if we start at home,  a lot of times the US, once they start something,  can be a leader and promulgate that information  and best practices to our international colleagues.  But I agree with you that it's an important issue  and we are starting.  And I know too, I should note that the US recently released  its national cybersecurity strategy,  but there's gonna be a companion strategy  that focuses in on the national cybersecurity workforce.  So I would look for that strategic document.  I don't know exactly when it will be released publicly,  but the plan is to release it publicly.  And I think there'll be ideas in there  in that strategic document  of how we can improve K through 12 education.  Yeah, thank you very much for the very nice presentation  with clear and strong messages.  I'm working for ABB,  that is a large automation company for industries.  I do have a lot to work on the 5G for industrial automation.  I fully agree with you about your concern  about security and the residence of 5G infrastructure.  My question is, do you think the major vendors  or the 5G technology has taken the proper actions  to address this concern?  For example, in their standard, in their product,  in the fundamental technologies?  I mean, I think we've taken positive strides,  but I think there's always room for improvement.  One of our big initiatives this year  with our director, Director Easterly,  is to encourage not just in the 5G space,  but in general, really encouraging product designers  to think about security and building that in  as they are developing the product\n",
            "46.2 - 48.79 min\n",
            " and not as an after action bolt-on.  And we're really trying to also drive the best practice  of having security as something that you don't have  to opt into, but it's security by default.  So, really these principles of building products securely  at the onset and as well as driving this security principle  of having a secure by default.  And so these are things that I know our organization  is trying to promulgate throughout our meetings  with industry and also with our international colleagues  as well.  Sorry, more question.  Do you see any single point authority  that is coordinating these actions?  For example, is a 3GPP or NIST in US  is driving such activities?  I don't think that there's a single point within the US  that's driving the activity,  but I would say that NIST is definitely a partner of ours.  I think CISA is also working.  I think the newly created office  of the national cyber director is probably going  to become a focal point of activities like that,  but it's in a nascent state, right?  It's just been established,  but I do think that in the future,  they could be one of those focal points.  Thank you.  Ms. Coppil, this is Miloš.  I am with Virginia Commonwealth University  and a general chair of this conference.  I wanted first to thank you.  I know how difficult it is to speak via Zoom  when you cannot exactly see the audience,  you cannot see the faces, you cannot see reactions.  So as a teacher via Zoom,  I can totally relate with the challenges  of presenting this way.  So thank you very much for your time today.  I wanted to mention that in Virginia,  our governor was stating that there's  about 36,000 available positions in cyber.  And out of those, maybe 17,000 in cybersecurity.  And you know, many companies like Amazon  moved to Northern Virginia and so on.  So the demand is definitely there.  I just wanted to ask you maybe if you can chime in  on a role of AI and machine learning in cyber  and 5G, how do you see the future when it comes to AI?  Is AI taking over?\n",
            "48.79 - 51.27 min\n",
            " Are we in control of AI?  Do we have some kind of shared autonomy agreement with AI?  What's gonna happen in the future, your take?  Thank you very much.  No, thanks for that question.  AI is obviously a hot topic.  And I don't think we know all the answers,  but AI like most technology that's developed, right?  It in itself is benign.  It's neither good or bad.  And AI itself brings obviously lots of promise,  but obviously there's also peril involved.  And so I know that we are trying to thoughtfully work  through as a country on how we should be thinking about AI  and what governance processes need to be in place.  As I mentioned, with the release  of the National Cybersecurity Strategy,  there are also these supplemental strategies  that are being released.  And one strategy that the US government is working on  is a strategy in regards to AI.  It's obviously a clearly a transformational technology,  and it's already in so much of what we do right now.  And I think as it becomes,  as we develop and grow its uses,  we do need to think about  what are the proper governance structures  that need to be in place.  And so I don't have the answers  to what that structure is yet,  but I know that we are working through together  as a US government and together with our industry  and academic partners to think through this issue.  Valerie, thank you so much.  I think, were there any other questions?  Yes, sir.  Hi, thank you again for the presentation.  It was very informative.  Something you mentioned is kind of sticking with me though,  in terms of the manufacture of 5G components  from places that might not be so aligned  with US operations.  And in the past, there have been some cases  and in the past there have been,  I don't wanna name names,  but there have been some, I guess,  embargoes on certain types of products,  like cellular products on the market.  Do you foresee that there might be more of that coming  with the scope of 5G and it's touching everything now,\n",
            "51.29 - 53.91 min\n",
            " smart cities, telemedicine,  the stuff that you mentioned in your presentation?  And do you think that this will affect US operations  outside of the US?  I will say that there is a real interest in understanding,  I think, COVID sort of opened up a lot of our understanding  or opened our eyes to a lot of the risks  that exist with our supply chain on top of,  I think even before COVID,  we were definitely talking about what risks exist  in the 5G supply chain network.  I do think that there are going,  and since recently the CHIPS Act has been passed,  there's definitely a push in the US government  to bring to onshore more of our capabilities  and technical components.  I don't see in the near term  how that would be possible entirely,  but there is definitely a movement to try to do that  and build some indigenous onshore capabilities here.  But again, I think it's also working  with our international partners  in how we work together on ensuring  that we have a secure and resilient supply chain  where we feel like all the components  and pieces and software involved in whatever the product is  is something that we feel like we can rely on.  Valerie, it looks like that's all the questions for now.  Thank you so much for your time today  and you have a great rest of your day.  Thank you, and thanks so much for inviting me to speak.  I hope you have a good rest of the conference.  Thank you.\n",
            "53.91 - 55.77 min\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "# Vectorization using ChromaDB\n",
        "\n",
        "## Introduction to ChromaDB\n",
        "ChromaDB is a high-performance vector database used to transform our text data into a structured, queryable form. In this section, we'll explain what ChromaDB is and why it's useful in our pipeline.\n",
        "\n",
        "## Preprocessing Text for Vectorization\n",
        "Before we can vectorize our text data, it may need to be preprocessed. This section discusses any necessary preprocessing steps such as tokenization or normalization.\n",
        "\n",
        "## Vectorizing Text Data\n",
        "Once our text data is preprocessed, it's time to vectorize it using ChromaDB. We'll explain how to send our text data to ChromaDB, receive vectorized data in return, and store this data for future use."
      ],
      "metadata": {
        "id": "pElv8qIQ3vT8"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "## Process text and store in VectorDB"
      ],
      "metadata": {
        "id": "xAybb4HaMo-y"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "os.environ[\"OPENAI_API_KEY\"] = \"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\"\n",
        "embeddings = OpenAIEmbeddings(openai_api_key = os.environ[\"OPENAI_API_KEY\"])\n",
        "#vstore with metadata. Here we will store page numbers.\n",
        "vStore = Chroma.from_texts(documents, embeddings, metadatas=[{\"source\": s} for s in sources])\n",
        "#deciding model\n",
        "model_name = \"gpt-3.5-turbo\"\n",
        "\n",
        "retriever = vStore.as_retriever()\n",
        "retriever.search_kwargs = {'k':2}"
      ],
      "metadata": {
        "id": "TZO9mYT_OfcY"
      },
      "execution_count": 10,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "model = RetrievalQAWithSourcesChain.from_chain_type(llm=OpenAI(), chain_type=\"stuff\", retriever=retriever)"
      ],
      "metadata": {
        "id": "W6jspHYeGEbJ"
      },
      "execution_count": 11,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "\n",
        "# Querying with Langchain\n",
        "\n",
        "## Introduction to Langchain\n",
        "Langchain provides a natural language interface for querying our vector data. In this section, we'll provide an introduction to Langchain and explain how it fits into our pipeline.\n",
        "\n",
        "## Setting Up Langchain for Querying\n",
        "Before we can start querying, we need to set up Langchain. This section will guide you through the process of setting up Langchain to work with our vectorized data.\n",
        "\n",
        "## Formulating and Executing Queries\n",
        "With Langchain set up, we can now formulate and execute queries on our data. We'll walk you through the process of creating a query, sending it to Langchain, and interpreting the results."
      ],
      "metadata": {
        "id": "fQjQD2Al35Rg"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "## Q&A"
      ],
      "metadata": {
        "id": "SXDgtwBrkNOr"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "query = \"What is this video about?\"\n",
        "response = model({\"question\":query}, return_only_outputs=True)\n",
        "print('Answer :',response['answer'])\n",
        "print('Referred clip segments :',response['sources'])"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "14Sap5xtGLZG",
        "outputId": "72840fca-8ae6-425b-c72d-b5d5a3c8b81b"
      },
      "execution_count": 12,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Answer :  This video is about assessing the risks associated with 5G technology and the US government's efforts to secure a more resilient future for the American people by growing the national cyber security workforce. \n",
            "\n",
            "Referred clip segments : 33.71 - 35.91 min, 53.91 - 55.77 min\n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "query = \"Summarise and list the key points addressed with a brief explanation of each.\"\n",
        "response = model({\"question\":query}, return_only_outputs=True)\n",
        "print('Answer :',response['answer'])\n",
        "print('Referred clip segments :',response['sources'])"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "zQYXQmXAJlTI",
        "outputId": "603b1879-a87d-4ae1-95ba-67197990a629"
      },
      "execution_count": 14,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Answer :  \n",
            "The key points addressed in the content are: \n",
            "1) Examining a variety of concerns when integrating 5G into networks, such as what functions the product or services perform and how it operates, whether the product receives software updates from the supplier, and whether there are technical mitigations that could be instituted to detect malicious use of the product. \n",
            "2) Examining the legal piece of a technology or product, such as what extent do the foreign government's laws or policies permit it to compel cooperation with its intelligence activities, is there a meaningful independent judicial review, and what are the foreign government's national security interests. \n",
            "3) Examining the relationship a company may have with a foreign government, such as whether a foreign government holds a financial stake in the company and to what extent a company may be influenced or susceptible to coercion. \n",
            "4) Securing a more resilient future for the American people and accelerating technological capabilities by growing the national cybersecurity workforce. \n",
            "5) Examining the National Critical Functions (NCFs) which includes connections by technologies that enable critical communications and capabilities to send and receive data, like 5G; distribution methods that allow the movement of goods, people, and utilities; management processes that ensure national security and public health and\n",
            "Referred clip segments : \n"
          ]
        }
      ]
    }
  ]
}