langchain_presentation.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Exploring LangChain: Components & RAG Walkthrough\n",
    "\n",
    "**Presenter:** Jack Tol  \n",
    "**Website:** [jacktol.net](https://jacktol.net)\n",
    "\n",
    "**Education:**  \n",
    "- Bachelor of Artificial Intelligence  \n",
    "  University of Technology Sydney  \n",
    "  *2023 - Present*\n",
    "\n",
    "**Overview:** Today, we'll delve into LangChain, exploring its key components and concluding with a walkthrough on leveraging Langchain's components and integrations to build a RAG prototype application.\n",
    "\n",
    "**Date:** 2024-06-29 \n",
    "\n",
    "**Agenda:**\n",
    "- Introduction to Langchain & Its Ecosystem\n",
    "- Reviewing a Few Langchain Components & A Brief Introduction to RAG\n",
    "- Live Textbook RAG Demonstration\n",
    "\n",
    "**Audience:** AI Enthusiasts, Developers, Researchers\n",
    "\n",
    "**Contact:** For inquiries or more information, please email [contact@jacktol.net](mailto:contact@jacktol.net) or visit [jacktol.net](https://jacktol.net).\n",
    "\n",
    "**Note:** This presentation will be recorded for future viewing.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Introduction to LangChain\n",
    "\n",
    "### The LangChain Ecosystem\n",
    "- [LangChain.com](https://www.langchain.com/) is the website for LangChain, a **software company** that creates various products to aid in the development, testing, evaluation, management, and deployment of LLM-powered AI applications.\n",
    "\n",
    "- The naming scheme adopted by LangChain can be a bit confusing. Let's start by differentiating between **LangChain the Company** and **LangChain the product**.\n",
    "\n",
    "- One of the main products offered by LangChain is also called LangChain. This framework contains multiple packages and aims to be a flexible, yet abstract AI toolkit that provides various modular components which work together to create LLM-powered software/applications. The LangChain Product will be the primary focus of today's presentation.\n",
    "\n",
    "#### The LangChain Product\n",
    "\n",
    "**LangChain** is a framework consisting of several packages:\n",
    "\n",
    "- **Langchain-Core**: Base abstractions of different components and provides a way to compose them together. No 3rd Party Integrations. Lightweight.\n",
    "- **Langchain-Community**: Community Maintained 3rd party integrations for various components.\n",
    "- **Partner Packages**: Popular integrations (e.g., langchain-openai, langchain-anthropic) are separated for better support.\n",
    "- **Langchain**: Contains chains, agents, and retrieval strategies for application architecture. All components are NOT specific to any one integration, but rather generic across all integrations.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "![](src/langchain_components.jpg)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### The LangSmith Product\n",
    "- **LangSmith**: Interpretability Developer platform for debugging, testing, evaluating, and monitoring LLM applications. RAG, CoT, Agentic Behavior.\n",
    "\n",
    "#### The LangServe Product\n",
    "- **LangServe**: A package which is built using FastAPI which makes it a bit easier deploy your Langchain chains as REST APIs for production-ready applications.\n",
    "\n",
    "### The Purpose of Langchain\n",
    "- **Langchain is a software framework** designed to simplify the integration of LLMs into applications by providing modular components.\n",
    "- **You can debug, test, evaluate, and deploy LLM applications** more efficiently using these various products and modular components.\n",
    "\n",
    "Langchain's library offers a variety of \"components\" that facilitate the development of LLM-related products and software. These components include:\n",
    "\n",
    "- LLMs\n",
    "- Embedding Models\n",
    "- Vector Stores\n",
    "- Document Loaders\n",
    "- Tools (Agentic Functions)\n",
    "\n",
    "...and much more.\n",
    "\n",
    "In this presentation, we'll explore some implementations of these components. We will conclude with a demonstration of a small-scale notebook-based RAG prototype, showcasing how these elements can be integrated and function cohesively."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Covering a few Langchain Components & A Brief Review of RAG\n",
    "\n",
    "There are far too many components within Langchain to share them all with you here, so I'll just be covering a few components and their implementations which I feel are the most important and intuitive to understand."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Document Loaders\n",
    "#### PDF Document Loader"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "from langchain.document_loaders import PyPDFLoader"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[Document(metadata={'source': 'example_data\\\\pdf_loader_example.pdf', 'page': 0}, page_content='Efficient World Models with Context-Aware Tokenization\\nVincent Micheli* 1Eloi Alonso* 1Franc ¸ois Fleuret1\\nAbstract\\nScaling up deep Reinforcement Learning (RL)\\nmethods presents a significant challenge. Follow-\\ning developments in generative modelling, model-\\nbased RL positions itself as a strong contender.\\nRecent advances in sequence modelling have led\\nto effective transformer-based world models, al-\\nbeit at the price of heavy computations due to\\nthe long sequences of tokens required to accu-\\nrately simulate environments. In this work, we\\npropose ∆-IRIS, a new agent with a world model\\narchitecture composed of a discrete autoencoder\\nthat encodes stochastic deltas between time steps\\nand an autoregressive transformer that predicts\\nfuture deltas by summarizing the current state\\nof the world with continuous tokens. In the\\nCrafter benchmark, ∆-IRIS sets a new state of\\nthe art at multiple frame budgets, while being\\nan order of magnitude faster to train than pre-\\nvious attention-based approaches. We release\\nour code and models at https://github.com/\\nvmicheli/delta-iris .\\n1. Introduction\\nDeep Reinforcement Learning (RL) methods have recently\\ndelivered impressive results (Ye et al., 2021; Hafner et al.,\\n2023; Schwarzer et al., 2023) in traditional benchmarks\\n(Bellemare et al., 2013; Tassa et al., 2018). In light of the\\nevermore complex domains tackled by the latest generations\\nof generative models (Rombach et al., 2022; Achiam et al.,\\n2023), the prospect of training agents in more ambitious\\nenvironments (Kanervisto et al., 2022) may hold signifi-\\ncant appeal. However, that leap forward poses a serious\\nchallenge: deep RL architectures have been comparatively\\nsmaller and less sample-efficient than their (self-)supervised\\ncounterparts. In contrast, more intricate environments ne-\\ncessitate models with greater representational power and\\nhave higher data requirements.\\n*Equal contribution1University of Geneva, Switzerland. Corre-\\nspondence to: <first.last@unige.ch >.\\nProceedings of the 41stInternational Conference on Machine\\nLearning , Vienna, Austria. PMLR 235, 2024. Copyright 2024 by\\nthe author(s).Model-based RL (MBRL) (Sutton & Barto, 2018) is hy-\\npothesized to be the key for scaling up deep RL agents\\n(LeCun, 2022). Indeed, world models (Ha & Schmidhuber,\\n2018) offer a diverse range of capabilities: lookahead search\\n(Schrittwieser et al., 2020; Ye et al., 2021), learning in imag-\\nination (Sutton, 1991; Hafner et al., 2023), representation\\nlearning (Schwarzer et al., 2021; D’Oro et al., 2023), and un-\\ncertainty estimation (Pathak et al., 2017; Sekar et al., 2020).\\nIn essence, MBRL shifts the focus from the RL problem to\\na generative modelling problem, where the development of\\nan accurate world model significantly simplifies policy train-\\ning. In particular, policies learnt in the imagination of world\\nmodels are freed from sample efficiency constraints, a com-\\nmon limitation of RL agents that is magnified in complex\\nenvironments with slow rollouts.\\nRecently, the IRIS agent (Micheli et al., 2023) achieved\\nstrong results in the Atari 100k benchmark (Bellemare et al.,\\n2013; Kaiser et al., 2020). IRIS introduced a world model\\ncomposed of a discrete autoencoder and an autoregressive\\ntransformer, casting dynamics learning as a sequence mod-\\nelling problem where the transformer composes over time a\\nvocabulary of image tokens built by the autoencoder. This\\napproach opened up avenues for future model-based meth-\\nods to capitalize on advances in generative modelling (Ville-\\ngas et al., 2022; Achiam et al., 2023), and has already been\\nadopted beyond its original domain (comma.ai, 2023; Hu\\net al., 2023). However, in its current form, scaling IRIS to\\nmore complex environments is computationally prohibitive.\\nIndeed, such an endeavor requires a large number of tokens\\nto encode visually challenging frames. Besides, sophisti-\\ncated dynamics may require to store numerous time steps\\nin memory to reason about the past, ultimately making the\\nimagination procedure excessively slow. Hence, under these\\nconstraints, maintaining a favorable imagined-to-collected\\ndata ratio is practically infeasible.\\nIn the present work, we introduce ∆-IRIS, a new agent\\ncapable of scaling to visually complex environments with\\nlengthier time horizons. ∆-IRIS encodes frames by attend-\\ning to the ongoing trajectory of observations and actions,\\neffectively describing stochastic deltas between time steps.\\nThis enriched conditioning scheme drastically reduces the\\nnumber of tokens to encode frames, offloads the determinis-\\ntic aspects of world modelling to the autoencoder, and lets\\nthe autoregressive transformer focus on stochastic dynamics.\\n1arXiv:2406.19320v1  [cs.LG]  27 Jun 2024'),\n",
       " Document(metadata={'source': 'example_data\\\\pdf_loader_example.pdf', 'page': 1}, page_content='Efficient World Models with Context-Aware Tokenization\\nz1\\n1 . . .zKI\\n1z1\\n2 . . .zKI\\n2\\nx1ˆx1\\nx2ˆx2\\nx0 a0 z1\\n1 . . . zK\\n1x1 a1 z1\\n2 . . . zK\\n2\\nx1 x2 x0 a0 a1 a2ˆx1 ˆx2\\nEncoderDecoder\\nFigure 1. Discrete autoencoder of IRIS (Micheli et al., 2023) (left) and ∆-IRIS (right). IRIS encodes and decodes frames independently,\\nmeaning that zthas to carry all the information necessary to reconstruct xt. On the other hand, ∆-IRIS’ encoder and decoder are\\nconditioned on past frames and actions, thus ztonly has to capture what has changed and that cannot be inferred from actions, i.e. the\\nstochastic delta. This conditioning scheme enables us to drastically reduce the number of tokens required to encode a frame with minimal\\nloss (K≪KI), which is critical to speed up the autoregressive transformer that predicts future tokens.\\nNonetheless, substituting the sequence of absolute image\\ntokens with a sequence of ∆-tokens makes the task of the\\nautoregressive model more arduous. In order to predict the\\nnext transition, it may only reason over previous ∆-tokens,\\nand thus faces the challenge of integrating over multiple\\ntime steps as a way to form a representation of the current\\nstate of the world. To resolve this issue, we modify the\\nsequence of the autoregressive model by interleaving contin-\\nuous I-tokens, that summarize successive world states with\\nframe embeddings, and discrete ∆-tokens.\\nIn the Crafter benchmark (Hafner, 2022), ∆-IRIS exhibits\\nfavorable scaling properties: the agent solves 17 out of\\n22 tasks after 10M frames of data collection, supersedes\\nDreamerV3 (Hafner et al., 2023) at multiple frame bud-\\ngets, and trains 10 times faster than IRIS. In addition, we\\ninclude results in the sample-efficient setting with Atari\\ngames. Through experiments, we provide evidence that ∆-\\nIRIS learns to disentangle the deterministic and stochastic\\naspects of world modelling. Moreover, we conduct ab-\\nlations to validate the new conditioning schemes for the\\nautoencoder and transformer models.\\n2. Method\\nWe consider a Partially Observable Markov Decision Pro-\\ncess ( POMDP ) (Sutton & Barto, 2018). The transition,\\nreward, and episode termination dynamics are captured\\nby the conditional distributions p(xt+1|x≤t, a≤t)and\\np(rt, dt|x≤t, a≤t), where xt∈ X =R3×h×wis an im-\\nage observation, at∈ A ={1, . . . , A }a discrete action,\\nrt∈Ra scalar reward, and dt∈ {0,1}indicates episode\\ntermination. The reinforcement learning objective is to finda policy pπ(at|x≤t, a<t)that maximizes the expected sum\\nof rewards Eπ[P\\nt≥0γtrt], with discount factor γ∈(0,1).\\nLearning in imagination (Sutton, 1991; Sutton & Barto,\\n2018) consists of 3 stages that are repeated alternatively:\\nexperience collection, world model learning, and policy\\nimprovement. Strikingly, the agent learns behaviours purely\\nwithin its world model, and real experience is only leveraged\\nto learn the environment dynamics.\\nIn the vein of IRIS (Micheli et al., 2023), our world model\\nis composed of a discrete autoencoder (Van Den Oord et al.,\\n2017) and an autoregressive transformer (Vaswani et al.,\\n2017; Radford et al., 2019), albeit with new conditioning\\nschemes and architectures. We first expose IRIS’ world\\nmodel in Section 2.1, then present ∆-IRIS’ autoencoder and\\nautoregressive model in Sections 2.2 and 2.3, respectively.\\nFinally, we describe the policy improvement phase in Sec-\\ntion 2.4. Appendix A gives a detailed breakdown of model\\narchitectures and hyperparameters.\\n2.1. Background: IRIS\\nHigh-dimensional images are converted into tokens with a\\ndiscrete autoencoder (EI, DI)(Van Den Oord et al., 2017).\\nThe encoder EI:Rh×w×3→ {1, . . . , N I}KImaps an in-\\nput image xtintoKItokens from a vocabulary of size NI.\\nThe discretization is done by picking the index of the vector\\nin the vocabulary embedding table that is closest to the en-\\ncoder output yt∈RKI×d. The KItokens are then decoded\\nback into an image with DI:{1, . . . , N I}KI→Rh×w×3.\\nThis discrete autoencoder is trained with L1reconstruction,\\nperceptual (Esser et al., 2021) and commitment losses (Van\\nDen Oord et al., 2017) computed on collected frames.\\n2'),\n",
       " Document(metadata={'source': 'example_data\\\\pdf_loader_example.pdf', 'page': 2}, page_content='Efficient World Models with Context-Aware Tokenization\\n˜x0a0 z1\\n1z2\\n1 . . . zK−1\\n1zK\\n1˜x1a1 z1\\n2z2\\n2 . . . zK−1\\n2zK\\n2˜x2a2 z1\\n3z2\\n3 . . . zK−1\\n3zK\\n3ˆz1\\n1ˆz2\\n1 . . . ˆzK−1\\n1ˆzK\\n1ˆz1\\n2ˆz2\\n2 . . . ˆzK−1\\n2ˆzK\\n2ˆz1\\n3ˆz2\\n3 . . . ˆzK−1\\n3ˆzK\\n3ˆr0ˆd0\\nˆr1ˆd1\\nˆr2ˆd2\\nx0 x1 x2\\nFigure 2. Unrolling dynamics over time. At each time step (separated by dashed lines), the GPT-like autoregressive transformer Gpredicts\\nthe∆-tokens for the next frame, as well as the reward and a potential episode termination. Its input sequence consists of action tokens,\\n∆-tokens, and I-tokens, namely continuous image embeddings that alleviate the need to attend to past ∆-tokens for world modelling.\\nMore specifically, an initial frame x0is embedded into I-token ˜x0. From ˜x0anda0,Gpredicts the reward ˆr0, episode termination\\nˆd0∈ {0,1}, and in an autoregressive manner ˆz1= (ˆz1\\n1, . . . , ˆzK\\n1), the∆-tokens for the next frame. Note that, during the imagination\\nprocedure, the next frame (stripped box) is computed by the decoder Dbased on previous frames, actions, and the ∆-tokens generated by\\nG, i.e.x1=D(x0, a0,ˆz1).\\nThe transformer GImodels the environment dynamics by\\noperating over an input sequence of image and action tokens\\n(z1\\n0, . . . , zKI\\n0, a0, z1\\n1, . . . , zKI\\n1, a1, . . . , z1\\nt, . . . , zKI\\nt, at).\\nImage and action tokens are embedded with learnt\\nlookup tables. At each time step, GIpredicts\\nthe transition, reward, and termination distributions:\\npGI(ˆzt+1|z≤t, a≤t)withˆzk\\nt+1∼pGI(ˆzk\\nt+1|z≤t, a≤t, z<k\\nt+1),\\npGI(ˆrt|z≤t, a≤t), and pGI(ˆdt|z≤t, a≤t). The model is\\ntrained with a cross-entropy loss on segments sampled from\\npast experience.\\nAt a high level, the autoencoder builds a vocabulary of image\\ntokens to encode each frame, and the transformer captures\\nthe environment dynamics by autoregressively composing\\nthe vocabulary over time. As a result, this world model\\nis capable of attending to previous time steps to make its\\npredictions, and models the joint law of future latent states.\\n2.2. Disentangling deterministic and stochastic\\ndynamics\\nIRIS (Micheli et al., 2023) encodes frames independently,\\nmaking no assumption about temporal redundancy within\\ntrajectories. One major drawback of this general formu-\\nlation is that, in environments with visually challenging\\nframes, a large number of tokens is required to encode\\nframes losslessly. Consequently, computations with the dy-\\nnamics model become increasingly prohibitive, as the atten-\\ntion mechanism scales quadratically with sequence length.\\nTherefore, limiting computation under such a trade-off may\\nresult in degraded performance (Micheli et al. (2023) app.\\nE)One possible solution to achieve fast world modelling with\\nminimal loss is to condition the autoencoder on previous\\nframes and actions. Intuitively, encoding a frame given pre-\\nvious frames consists in describing what has changed, the\\ndelta, between successive time steps. In many environments,\\nthe delta between frames is often much simpler to describe\\nthan the frames themselves. As a matter of fact, when\\nthe transition function is deterministic, adding previous ac-\\ntions to the conditioning of the decoder results in a world\\nmodel, without the need to encode any information between\\ntime steps. However, most environments of interest feature\\nstochastic dynamics, and apart from aleatoric uncertainty,\\narchitectural limitations such as the agent’s memory may\\ninduce additional epistemic uncertainty. Hence, the delta\\nbetween two time steps usually consists of deterministic and\\nstochastic components.\\nFor instance, an agent moving from one square to another\\nin a grid-like environment when pressing movement keys\\ncan be seen as a deterministic component of the transition.\\nOn the other hand, the sudden apparition of an enemy in\\na nearby square is a random event. Interestingly, only the\\nstochastic features of a transition should be encoded, and\\nthe autoencoder could directly learn to model the determin-\\nistic dynamics, which do not require the expressivity and\\nability to handle multimodality of an autoregressive model.\\nTherefore, when autoenconding frames by conditioning on\\nprevious frames and actions, a frame encoding may only\\nconsist of a handful of ∆-tokens, instead of a large number\\nof image tokens describing frames independently.\\nSection 3.4 provides empirical evidence that ∆-IRIS’ autoen-\\ncoder learns to encode frames in such fashion, and Figure 1\\nillustrates the new conditioning scheme of the autoencoder.\\n3'),\n",
       " Document(metadata={'source': 'example_data\\\\pdf_loader_example.pdf', 'page': 3}, page_content='Efficient World Models with Context-Aware Tokenization\\n∆-tokens sampled randomly\\n∆-tokens sampled by the autoregressive transformer\\nt= 0 t= 4 t= 5 t= 9 t= 10 t= 12\\nFigure 3. Evidence of dynamics disentanglement. Two trajectories are imagined with different ways of generating ∆-tokens. In the top\\ntrajectory, ∆-tokens are sampled randomly. In the bottom trajectory, the autoregressive transformer predicts future ∆-tokens. The same\\nstarting frame ( t= 0) and sequence of actions are used. With random ∆-tokens, the deterministic aspects of the dynamics (layout,\\nmovement, items, crafting) are still properly modelled, but the stochastic dynamics (mobs, health indicators) become problematic. For\\ninstance, the agent successfully cuts down a tree between t= 4andt= 5, and uses wood planks to build a crafting table between t= 10\\nandt= 12 . We observe that these dynamics are modelled in the same way whether ∆-tokens are sampled randomly or not. However, in\\nthe top trajectory, large quantities of cows appear and disappear from the screen incoherently, whereas the bottom trajectory does not\\ndisplay such erratic patterns. This experiment shows that ∆-IRIS encodes stochastic deltas between time steps with ∆-tokens, and its\\ndecoder handles the deterministic aspects of world modelling. Appendix F contains additional examples.\\nMore formally, for any set Y, we denote Sn(Y) =Sn\\ni=1Yi\\nthe set of tuples of elements from Yof maximum length\\nn, andS(Y) =S∞(Y). LetZ={1, . . . , N }a vocab-\\nulary of discrete tokens. Given past images and actions\\n(x0, a0, . . . , x t−1, at−1), the encoder E:S(X ×A )×X →\\nZKconverts an image xtintozt= (z1\\nt, . . . , zK\\nt), a se-\\nquence of Kdiscrete ∆-tokens. The encoder is param-\\neterized by a Convolutional Neural Network ( CNN) (Le-\\nCun et al., 1989). Actions are embedded with a learnt\\nlookup table and concatenated channel-wise with frames.\\nWe use vector quantization (Van Den Oord et al., 2017;\\nEsser et al., 2021) with factorized and normalized codes\\n(Yu et al., 2021) to discretize the encoder’s continuous\\noutputs. The CNN decoder D:S(X × A )× ZK→ X\\nreconstructs an image ˆxtfrom past frames, actions and ∆-\\ntokens (x0, a0, . . . , x t−1, at−1, zt). Action and ∆-tokens\\nare embedded with learnt lookup tables, and concatenated\\nchannel-wise with feature maps obtained by forwarding\\nframes through an auxiliary CNN.\\nThe discrete autoencoder is trained on previously collected\\ntrajectories with a weighted combination of L1,L2and\\nmax-pixel (Anand et al., 2022) reconstruction losses, as\\nwell as a commitment loss (Van Den Oord et al., 2017). The\\ncodebook is updated with an exponential moving average\\n(Razavi et al., 2019) and we use a straight-through estimator\\n(Bengio et al., 2013) to enable backpropagation.2.3. Modelling stochastic dynamics\\nWhile it should be possible to predict future ∆-tokens, given\\na starting image, past actions and ∆-tokens, we found this\\ntask much more difficult than simply predicting future image\\ntokens, given past image tokens and actions, as in IRIS.\\nTo better understand why this is the case, let us consider\\nanother example: in a grid environment, ∆-tokens may\\ndescribe the unpredictable movement of an enemy, ran-\\ndomly jumping from one square to another at every time\\nstep. Based on the initial enemy location and after only a\\nfew time steps, it becomes increasingly difficult to predict\\nif the enemy and the agent are located on the same square,\\nwhich could trigger a battle and make the enemy disap-\\npear. Indeed, situating the two entities involves reasoning\\nabout the initial observation, and integrating over all of the\\nprevious action and ∆-tokens, which may have a complex\\ndependence structure.\\nTo address this problem, we alter the sequence of the dynam-\\nics model by interleaving continuous I-tokens, in reference\\nto MPEG’s I-frames (Richardson, 2004), and discrete ∆-\\ntokens. I-tokens alleviate the need of integrating over past\\n∆-tokens to form a representation of the current state of\\nthe world, i.e. they deploy a “soft” Markov blanket for the\\nprediction of the next ∆-tokens.\\n4'),\n",
       " Document(metadata={'source': 'example_data\\\\pdf_loader_example.pdf', 'page': 4}, page_content='Efficient World Models with Context-Aware Tokenization\\nTable 1. Returns, number of parameters, and frames collected per second (FPS) for the methods considered. We compute FPS as the total\\nnumber of environment frames collected divided by the training duration. ∆-IRIS outperforms DreamerV3 for larger frame budgets, and\\nis 10x faster than IRIS (64 tokens).\\nMethod Return @1M Return @5M Return @10M #Parameters FPS\\n∆-IRIS 7.7 (0.5) 15.4 (0.4) 16.1 (0.1) 25M 20\\nDreamerV3 XL 9.2(0.3) 14.2 (0.2) 15.1 (0.3) 200M 30\\nIRIS (64 tokens) 5.5 (0.7) - - 48M 2\\n∆-IRIS w/o I-tokens 6.6 (0.2) 10.4 (0.5) 12.6 (0.8) 24M 22\\nDreamerV3 M 6.2 (0.5) 12.6 (0.7) 13.7 (0.8) 37M 40\\nIRIS (16 tokens) 4.4 (0.1) - - 50M 6\\n 4 6 8 10 12 14 16 18\\n1M 2M 3M 4M 5M 6M7M8M9MReturn\\nNumber of framesDreamerV3 MDreamerV3 XLIRIS (16 tokens)IRIS (64 tokens)Δ-IRIS w/o I-tokensΔ-IRIS\\nFigure 4. Returns at multiple frame budgets in the Crafter bench-\\nmark. ∆-IRIS achieves higher returns than DreamerV3 beyond\\n3M frames, and surpasses IRIS for all frame budgets considered.\\nRemoving I-tokens from the input sequence of the autoregressive\\ntransformer significantly hurts performance.\\nWe obtain I-tokens by forwarding frames through an aux-\\niliary CNN at each time step. They are not produced by a\\ndiscrete autoencoder. Since I-tokens are not predicted by the\\nmodel but rather enrich its conditioning, there are no incen-\\ntives to include a lossy discretization operator or to optimize\\na reconstruction loss. Instead, they are optimized end-to-\\nend with the learning objectives of the dynamics model.\\nWith this improved conditioning, the dynamics model per-\\nceives the ongoing trajectory with a mixture of continuous\\nand discrete representations, while making its predictions\\nautoregressively in a discrete space.\\nFigure 2 displays the input sequence of the dynam-\\nics model and the quantities it predicts. Given a se-\\nquence of past I-tokens, action tokens, and ∆-tokens\\n(˜x0, a0, z1\\n1, . . . , zK\\n1, . . . , ˜xt−1, at−1, z1\\nt, . . . , zk\\nt), the dy-\\nnamics model Goutputs a categorical distribution on Z\\nfor the next ∆-token ˆzk+1\\nt∼pG(ˆzk+1\\nt|˜x<t, z<t, a<t, z≤k\\nt).\\nIt also predicts distributions for rewards pG(ˆrt|˜x≤tz≤t, a≤t)\\nand episode terminations pG(ˆdt|˜x≤t, z≤t, a≤t).Gis parameterized by a stack of transformer encoder layers\\nwith causal self-attention (Vaswani et al., 2017; Radford\\net al., 2019). It is trained with a cross-entropy loss for transi-\\ntion and termination predictions, and we follow DreamerV3\\n(Hafner et al., 2023) in using discrete regression with two-\\nhot targets and symlog scaling for reward prediction (Imani\\n& White, 2018).\\n2.4. Policy improvement\\nDuring the policy improvement phase, the policy πlearns\\nin the imagination POMDP of its world model, composed of\\nthe autoencoder (E, D)and the dynamics model G.\\nAt time step t, the policy observes a reconstructed image\\nobservation ˆxtand samples action at∼π(at|ˆx≤t). The\\nworld model then predicts the reward ˆrt, the episode end ˆdt,\\nand the next observation ˆxt+1=D(ˆx≤t,ˆa≤t,ˆz≤t,ˆzt+1),\\nwithˆzt+1∼pG(ˆzt+1|ˆx≤t,ˆa≤t,ˆz≤t). The imagination pro-\\ncedure is initialized with a real observation x0sampled from\\npast experience, and is rolled out for Hsteps. The procedure\\nstops if an episode termination is predicted before reaching\\nthe imagination horizon.\\nWe employ to a large extent the actor-critic training method\\nused for IRIS (Micheli et al., 2023). A value baseline is\\ntrained to predict λ-returns (Sutton & Barto, 2018) with the\\nsame discrete regression objective as for reward prediction.\\nThe policy optimizes the REINFORCE with value baseline\\n(Sutton & Barto, 2018) learning objective over imagined\\ntrajectories. Exploration is encouraged by adding an entropy\\nmaximization term to the policy’s objective.\\n3. Experiments\\nIn our experiments, we consider the Crafter benchmark\\n(Hafner, 2022) to illustrate ∆-IRIS’ ability to scale to a\\nvisually rich environment with large frame budgets. Be-\\nsides, we also include Atari 100k games (Bellemare et al.,\\n2013; Kaiser et al., 2020) in Appendix C to showcase the\\nperformance and speed of our agent in the sample-efficient\\nsetting.\\n5')]"
      ]
     },
     "execution_count": 2,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "pdf_loader_example = PyPDFLoader(\"example_data\\pdf_loader_example.pdf\")\n",
    "pages =  pdf_loader_example.load()\n",
    "pages[:5]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### CSV Document Loader"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [],
   "source": [
    "from langchain.document_loaders import CSVLoader"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[Document(metadata={'source': 'example_data\\\\metadata_parsed_arxiv.csv', 'row': 0}, page_content=\"document_id: 0809.0182\\nauthors: [{'keyname': 'Xie', 'forenames': 'Z. Y.'}, {'keyname': 'Jiang', 'forenames': 'H. C.'}, {'keyname': 'Chen', 'forenames': 'Q. N.'}, {'keyname': 'Weng', 'forenames': 'Z. Y.'}, {'keyname': 'Xiang', 'forenames': 'T.'}]\\ntitle: Second Renormalization of Tensor-Network States\"),\n",
       " Document(metadata={'source': 'example_data\\\\metadata_parsed_arxiv.csv', 'row': 1}, page_content=\"document_id: 0810.0725\\nauthors: [{'keyname': 'Shadrin', 'forenames': 'S.'}]\\ntitle: BCOV theory via Givental group action on cohomological field theories\"),\n",
       " Document(metadata={'source': 'example_data\\\\metadata_parsed_arxiv.csv', 'row': 2}, page_content=\"document_id: 0909.0800\\nauthors: [{'keyname': 'Shadrin', 'forenames': 'Sergey'}, {'keyname': 'Zvonkine', 'forenames': 'Dimitri'}]\\ntitle: A group action on Losev-Manin cohomological field theories\"),\n",
       " Document(metadata={'source': 'example_data\\\\metadata_parsed_arxiv.csv', 'row': 3}, page_content=\"document_id: 0911.2157\\nauthors: [{'keyname': 'Herrero', 'forenames': 'R.'}, {'keyname': 'Pi', 'forenames': 'F.'}, {'keyname': 'Rius', 'forenames': 'J.'}, {'keyname': 'Orriols', 'forenames': 'G.'}]\\ntitle: About the oscillatory possibilities of the dynamical systems\"),\n",
       " Document(metadata={'source': 'example_data\\\\metadata_parsed_arxiv.csv', 'row': 4}, page_content=\"document_id: 0912.5115\\nauthors: [{'keyname': 'Buryak', 'forenames': 'A.'}, {'keyname': 'Shadrin', 'forenames': 'S.'}]\\ntitle: A new proof of Faber's intersection number conjecture\")]"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "csv_loader_example = CSVLoader(\"example_data\\metadata_parsed_arxiv.csv\", encoding=\"utf-8\")\n",
    "csv_data = csv_loader_example.load()\n",
    "csv_data[:5]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Chat Models"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [],
   "source": [
    "from langchain_anthropic.chat_models import ChatAnthropic "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [],
   "source": [
    "claude_llm = ChatAnthropic(model=\"claude-3-5-sonnet-20240620\",\n",
    "                           temperature=0,\n",
    "                           max_tokens=1024,\n",
    "                           timeout=None,)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "\"The meaning of life is a profound philosophical question that has been debated by thinkers, philosophers, and religious leaders throughout history. There is no single, universally accepted answer, as the meaning of life can be highly personal and subjective. Some common perspectives include:\\n\\n1. Happiness and well-being: Finding joy, contentment, and fulfillment in life.\\n\\n2. Love and relationships: Forming meaningful connections with others.\\n\\n3. Personal growth: Continuously learning, improving, and developing as an individual.\\n\\n4. Contribution to society: Making a positive impact on the world and helping others.\\n\\n5. Self-realization: Discovering and fulfilling one's potential and purpose.\\n\\n6. Religious or spiritual fulfillment: Following a particular faith or spiritual path.\\n\\n7. Experiencing and appreciating existence: Living in the present moment and finding beauty in the world.\\n\\n8. Creating meaning: Actively constructing purpose and significance in one's own life.\\n\\n9. Survival and reproduction: Fulfilling biological imperatives.\\n\\n10. Seeking knowledge and understanding: Pursuing truth and wisdom.\\n\\nUltimately, the meaning of life is a personal journey of discovery, and individuals may find different answers based on their experiences, beliefs, and values. It's important to note that many people combine multiple perspectives to create their own unique understanding of life's meaning.\""
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "claude_response = claude_llm.invoke(\"What is the meaning of life?\")\n",
    "claude_response.content"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Vector Stores"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [],
   "source": [
    "from langchain_pinecone import PineconeVectorStore\n",
    "from langchain_openai import OpenAIEmbeddings"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [],
   "source": [
    "embedding_model = OpenAIEmbeddings(model=\"text-embedding-3-small\")\n",
    "arxiv_example_index = PineconeVectorStore.from_existing_index(embedding=embedding_model, index_name=\"arxiv-example-index\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[Document(metadata={'document_id': '2406.17358'}, page_content='Uniform stability of the damped wave equation with a confining potential in the Euclidean space by Antoine Prouff'),\n",
       " Document(metadata={'document_id': '2406.17348'}, page_content='Exact controllability to eigensolutions of the heat equation via bilinear controls on two-dimensional domains by Rémi Buffe, Alessandro Duca'),\n",
       " Document(metadata={'document_id': '2406.17449'}, page_content='Pulsating string solution and stability in two parameter $\\\\chi$-deformed background by Rashmi R Nayak, Nibedita Padhi, Manoranjan Samal'),\n",
       " Document(metadata={'document_id': '2406.17293'}, page_content='Improved wave function for heavy-light mesons in QCD potential model approach and parameterization of the Cornell potential by Abdul Aziz, Sabyasachi Roy, Atri DEshamukhya'),\n",
       " Document(metadata={'document_id': '2406.17498'}, page_content='Multi-solitons of one-dimensional Boussinesq equation by Vicente Alvarez, Amin Esfahani')]"
      ]
     },
     "execution_count": 13,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "search_results = arxiv_example_index.similarity_search(query=\"Uniform stability of the damped wave equation with a confining potential in the Euclidean space\", k=5)\n",
    "search_results"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### A Brief Review of RAG\n",
    "\n",
    "- RAG Stands for Retrieval Augmented Generation.\n",
    "- A RAG Pipeline consists of a few different parts, which all work together to allow a language model to generate reliable and accurate information based on retrieved context.\n",
    "- We pass through the users query and the retrieved context dynamically to a language model by filling in a prompt template.\n",
    "- The prompt template leverages a LLMs ability to follow system level instructions when directed and the fact that we can pass through content as variables to the model.\n",
    "\n",
    "\n",
    "#### Prompt Template Example\n",
    "```\n",
    "Instructions:\n",
    "Your job is to answer the user's query using only the provided context.\n",
    "Keep your answer grounded in the facts of the provided context.\n",
    "If the context does not contain the facts needed to answer the user's query, return: \"I do not have enough information available to accurately answer the question.\"\n",
    "\n",
    "Context:\n",
    "{context}\n",
    "\n",
    "Query:\n",
    "{user_query}\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### How does the context actually get retrieved?\n",
    "- We take our external documents (PDFs, CSVs, TXTs etc.) and use **some heuristic** to split the text (perhaps splitting after *n* number of characters or based on the tags in a programming language if we are splitting code text).\n",
    "- If our splits are too small, say, if we decided to split on sentences, we can combine a few sentences together to create a ***chunk*** of text.\n",
    "- We take our chunks of text and pass them through a ***sentence embedding*** model, and we upload the embedding vectors to a vector database.\n",
    "- Now, when a user enters a query, if we encode that query using the same embedding model, we can then perform a ***cosine similarity*** search between all the vectors in the database with respect to the users query vector, and then retrieve the ***top-k*** most similar chunks of text.\n",
    "- If we save the retrieved context to a `context` variable, we can easily send that through to the prompt template in addition to the users query and let the model generate the response."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### RAG Pipeline Diagram\n",
    "![](src/rag_pipeline.png)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Textbook RAG Demonstration Using Langchain\n",
    "### Part 1: Loading, Processing, & Uploading Our Documents\n",
    "\n",
    "Let's first quickly take a look at the Textbook PDF we are going to be using for this example, in addition to the Pinecone Vector Store where we will be uploading our chunks to.\n",
    "\n",
    "Understanding any high-level framework always begins by looking at and understanding the imports you'll be using.\n",
    "\n",
    "Looking at our imports we see that we first import our 'Document Loader/Transformer' components, which in this case is `PyPDFLoader` & `RecurseiveCharacterSplitter` respectively.\n",
    "\n",
    "Next, we import our \"Embedding Model & Vector Store\" components, which in this case is the `HuggingFaceEmbeddings` & `PineconeVectorStore` respectively.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [],
   "source": [
    "from langchain.document_loaders import PyPDFLoader\n",
    "from langchain.text_splitter import RecursiveCharacterTextSplitter\n",
    "    \n",
    "from langchain_huggingface import HuggingFaceEmbeddings\n",
    "from langchain_pinecone import PineconeVectorStore"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's setup our PDF Loader, passing through the path for the textbook, and our Text Splitter, passing through our desired text splitting settings."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [],
   "source": [
    "pdf_loader = PyPDFLoader(\"Machine Learning - An Algorithmic Perspective, Second Edition, by Stephen Marsland.pdf\")\n",
    "text_splitter = RecursiveCharacterTextSplitter(chunk_size=500, chunk_overlap=50, length_function=len)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "And now running our PDF Loader & Splitter with the convenient `load_and_split` method."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [],
   "source": [
    "chunks = pdf_loader.load_and_split(text_splitter=text_splitter)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Inspecting our chunks, we can see that each chunk is stored in a list, with a `page_content` component containing the actual text of the chunk, and also a `metadata` component which stores a dictionary of metadata for each chunk, including the name of the original source of the chunk and also the page from which it came from."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[Document(metadata={'source': 'Machine Learning - An Algorithmic Perspective, Second Edition, by Stephen Marsland.pdf', 'page': 0}, page_content='Chapman & Hall/CRC \\nMachine Learning & Pattern Recognition SeriesChapman & Hall/CRC \\nMachine Learning & Pattern Recognition Series\\nMachine Learning MACHINE \\nLEARNING\\nAn Algorithmic Perspective\\nSecond Edition\\nMarsland\\nStephen Marsland\\n• Access online or download to your smartphone, tablet or PC/Mac\\n• Search the full text of this and other titles you own\\n• Make and share notes and highlights\\n• Copy and paste text and figures for use in your own documents'),\n",
       " Document(metadata={'source': 'Machine Learning - An Algorithmic Perspective, Second Edition, by Stephen Marsland.pdf', 'page': 0}, page_content='• Customize your view by changing font size and layoutWITH VITALSOURCE®\\nEBOOKsecond editionMachine Learning: An Algorithmic Perspective, Second Edition  helps you understand \\nthe algorithms of machine learning. It puts you on a path toward mastering the relevant \\nmathematics and statistics as well as the necessary programming and experimentation.\\nNew to the Second Edition\\n•  Two new chapters on deep belief networks and Gaussian processes'),\n",
       " Document(metadata={'source': 'Machine Learning - An Algorithmic Perspective, Second Edition, by Stephen Marsland.pdf', 'page': 0}, page_content='•  Reorganization of the chapters to make a more natural flow of content\\n•  Revision of the support vector machine material, including a simple implementation for \\nexperiments\\n•  New material on random forests, the perceptron convergence theorem, accuracy \\nmethods, and conjugate gradient optimization for the multi-layer perceptron\\n•  Additional discussions of the Kalman and particle filters\\n•  Improved code, including better use of naming conventions in Python'),\n",
       " Document(metadata={'source': 'Machine Learning - An Algorithmic Perspective, Second Edition, by Stephen Marsland.pdf', 'page': 0}, page_content='The text strongly encourages you to practice with the code. Each chapter includes detailed \\nexamples along with further reading and problems. All of the Python code used to create the \\nexamples is available on the author’s website. \\nFeatures\\n•  Reflects recent developments in machine learning, including the rise of deep belief \\nnetworks\\n•  Presents the necessary preliminaries, including basic probability and statistics\\n•  Discusses supervised learning using neural networks'),\n",
       " Document(metadata={'source': 'Machine Learning - An Algorithmic Perspective, Second Edition, by Stephen Marsland.pdf', 'page': 0}, page_content='•  Covers dimensionality reduction, the EM algorithm, nearest neighbor methods, optimal \\ndecision boundaries, kernel methods, and optimization\\n•  Describes evolutionary learning, reinforcement learning, tree-based learners, and \\nmethods to combine the predictions of many learners\\n•  Examines the importance of unsupervised learning, with a focus on the self-organizing \\nfeature map \\n•  Explores modern, statistically based approaches to machine learning\\nK18981\\nwww.crcpress.com\\nMachine Learning')]"
      ]
     },
     "execution_count": 17,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "chunks[:5]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "As a sanity check, we can also inspect the length of the `chunks` list and see if it makes sense, and in this case it does, especially since we set our chunk size to 500 characters."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "2285"
      ]
     },
     "execution_count": 18,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "len(chunks)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now let's set up two variables, the `embedding_model`and our `textbook_vector_store`, these are generally set up as global variables as many different parts of your codebase will use them.\n",
    "\n",
    "We set the `embedding_model` to a `HuggingFaceEmbeddings` object, and our `textbook_vector_store` to a `PineconeVectorStore` object, using the `from_existing_index` method, in which we can pass through our embedding model, in addition to the index name, which we set to the name of our Pinecone index which we have set up."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [],
   "source": [
    "embedding_model = HuggingFaceEmbeddings()\n",
    "\n",
    "textbook_vector_store = PineconeVectorStore.from_existing_index(embedding=embedding_model, index_name=\"textbook-vector-store\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's now parse both the text and metadata component of each chunk, parsing each of these components to their own list, taking special attention to parse the metadata as a dictionary as Pinecone expects metadata to be formatted that way as we'll see shortly."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {},
   "outputs": [],
   "source": [
    "texts = [chunk.page_content for chunk in chunks]\n",
    "metadatas = [{'page': chunk.metadata['page'] + 1, 'document': chunk.metadata['source']} for chunk in chunks]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "And as we would expect, when we look within the `texts` list, we see the text component of the chunks. And ditto for the our `metadatas` list, where we see the metadata, including the page number & source file name for that particular chunk, formatted as a dictionary. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "['Chapman & Hall/CRC \\nMachine Learning & Pattern Recognition SeriesChapman & Hall/CRC \\nMachine Learning & Pattern Recognition Series\\nMachine Learning MACHINE \\nLEARNING\\nAn Algorithmic Perspective\\nSecond Edition\\nMarsland\\nStephen Marsland\\n• Access online or download to your smartphone, tablet or PC/Mac\\n• Search the full text of this and other titles you own\\n• Make and share notes and highlights\\n• Copy and paste text and figures for use in your own documents',\n",
       " '• Customize your view by changing font size and layoutWITH VITALSOURCE®\\nEBOOKsecond editionMachine Learning: An Algorithmic Perspective, Second Edition  helps you understand \\nthe algorithms of machine learning. It puts you on a path toward mastering the relevant \\nmathematics and statistics as well as the necessary programming and experimentation.\\nNew to the Second Edition\\n•  Two new chapters on deep belief networks and Gaussian processes',\n",
       " '•  Reorganization of the chapters to make a more natural flow of content\\n•  Revision of the support vector machine material, including a simple implementation for \\nexperiments\\n•  New material on random forests, the perceptron convergence theorem, accuracy \\nmethods, and conjugate gradient optimization for the multi-layer perceptron\\n•  Additional discussions of the Kalman and particle filters\\n•  Improved code, including better use of naming conventions in Python',\n",
       " 'The text strongly encourages you to practice with the code. Each chapter includes detailed \\nexamples along with further reading and problems. All of the Python code used to create the \\nexamples is available on the author’s website. \\nFeatures\\n•  Reflects recent developments in machine learning, including the rise of deep belief \\nnetworks\\n•  Presents the necessary preliminaries, including basic probability and statistics\\n•  Discusses supervised learning using neural networks',\n",
       " '•  Covers dimensionality reduction, the EM algorithm, nearest neighbor methods, optimal \\ndecision boundaries, kernel methods, and optimization\\n•  Describes evolutionary learning, reinforcement learning, tree-based learners, and \\nmethods to combine the predictions of many learners\\n•  Examines the importance of unsupervised learning, with a focus on the self-organizing \\nfeature map \\n•  Explores modern, statistically based approaches to machine learning\\nK18981\\nwww.crcpress.com\\nMachine Learning']"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/plain": [
       "[{'page': 1,\n",
       "  'document': 'Machine Learning - An Algorithmic Perspective, Second Edition, by Stephen Marsland.pdf'},\n",
       " {'page': 1,\n",
       "  'document': 'Machine Learning - An Algorithmic Perspective, Second Edition, by Stephen Marsland.pdf'},\n",
       " {'page': 1,\n",
       "  'document': 'Machine Learning - An Algorithmic Perspective, Second Edition, by Stephen Marsland.pdf'},\n",
       " {'page': 1,\n",
       "  'document': 'Machine Learning - An Algorithmic Perspective, Second Edition, by Stephen Marsland.pdf'},\n",
       " {'page': 1,\n",
       "  'document': 'Machine Learning - An Algorithmic Perspective, Second Edition, by Stephen Marsland.pdf'}]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "from IPython.display import display\n",
    "\n",
    "display(texts[:5])\n",
    "display(metadatas[:5])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We are now finally at the point where we can embed and then upload our chunks. We use the `from_texts` method on our `textbook_vector_store`, passing through our `texts`, `embedding_model`, `metadatas`, and `index_name`. Suprisingly, due to the amazing power and abstraction given to us by Langchain, this one line of code embeds and uploads all 2285 chunks of our textbook."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<langchain_pinecone.vectorstores.PineconeVectorStore at 0x1e80f5cdc90>"
      ]
     },
     "execution_count": 23,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "textbook_vector_store.from_texts(\n",
    "    texts=texts,\n",
    "    embedding=embedding_model,\n",
    "    metadatas=metadatas,\n",
    "    index_name=\"textbook-vector-store\"\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's quickly make a function to conduct a similarity search on the vector store with respect to a user query we enter, setting the number of items to retrieve (k) to 5, printing out the retrieved context chunk and the corresponding page number and source document metadata. We use the `similarity_search` method on our `textbook_vector_store`, passing through our query and k number as parameters, loop through the results, and print the relevant information."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {},
   "outputs": [],
   "source": [
    "def retrieve_and_display(query, k=5):\n",
    "    results = textbook_vector_store.similarity_search(query=query, k=k)\n",
    "    for result in results:\n",
    "        print(f\"Text: {result.page_content}\\n\")\n",
    "        print(f\"Document: {result.metadata['document']}\\n\")\n",
    "        print(f\"Page: {int(result.metadata['page'])}\")\n",
    "        print(\"\\n\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Wow! Look at that, we see the context which was retrieved, which is by default ordered by descending with respect to the cosine similarity between the chunk and the user query, in addition to the name of the source document of that chunk and the page number from which the chunk came from so we can always cross-check the information."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Text: neural networks so that they can do something useful.\n",
      "The question we need to think about ﬁrst is how our neurons can learn. We are going to\n",
      "look atsupervised learning for the next few chapters, which means that the algorithms will\n",
      "learn by example: the dataset that we learn from has the correct output values associated\n",
      "with each datapoint. At ﬁrst sight this might seem pointless, since if you already know the\n",
      "\n",
      "Document: Machine Learning - An Algorithmic Perspective, Second Edition, by Stephen Marsland.pdf\n",
      "\n",
      "Page: 64\n",
      "\n",
      "\n",
      "Text: a well-known introduction to neural networks:\n",
      "•D.E. Rumelhart, G.E. Hinton, and R.J. Williams. Learning internal representations\n",
      "by back-propagating errors. Nature, 323(99):533–536, 1986a.\n",
      "•D.E. Rumelhart, J.L. McClelland, and the PDP Research Group, editors. Parallel\n",
      "Distributed Processing . MIT Press, Cambridge, MA, 1986b.\n",
      "•R. Lippmann. An introduction to computing with neural nets. IEEE ASSP Magazine ,\n",
      "pages 4–22, 1987.\n",
      "\n",
      "Document: Machine Learning - An Algorithmic Perspective, Second Edition, by Stephen Marsland.pdf\n",
      "\n",
      "Page: 129\n",
      "\n",
      "\n",
      "Text: to look at the neural network solution proposed by Rumelhart, Hinton, and McClelland,\n",
      "the Multi-layer Perceptron (MLP), which is still one of the most commonly used machine\n",
      "learning methods around. The MLP is one of the most common neural networks in use. It\n",
      "is often treated as a ‘black box’, in that people use it without understanding how it works,\n",
      "which often results in fairly poor results. Getting to the stage where we understand how it\n",
      "\n",
      "Document: Machine Learning - An Algorithmic Perspective, Second Edition, by Stephen Marsland.pdf\n",
      "\n",
      "Page: 94\n",
      "\n",
      "\n",
      "Text: 3.2 NEURAL NETWORKS\n",
      "One thing that is probably fairly obvious is that one neuron isn’t that interesting. It doesn’t\n",
      "do very much, except ﬁre or not ﬁre when we give it inputs. In fact, it doesn’t even learn.\n",
      "If we feed in the same set of inputs over and over again, the output of the neuron never\n",
      "varies—it either ﬁres or does not. So to make the neuron a little more interesting we need\n",
      "to work out how to make it learn, and then we need to put sets of neurons together into\n",
      "\n",
      "Document: Machine Learning - An Algorithmic Perspective, Second Edition, by Stephen Marsland.pdf\n",
      "\n",
      "Page: 64\n",
      "\n",
      "\n",
      "Text: In terms of learning about a set of data we have now reached the stage that neural\n",
      "networks were up to in 1969. Then, two researchers, Minsky and Papert, published a book\n",
      "called “Perceptrons.” The purpose of the book was to stimulate neural network research\n",
      "by discussing the learning capabilities of the Perceptron, and showing what the network\n",
      "could and could not learn. Unfortunately, the book had another eﬀect: it eﬀectively killed\n",
      "\n",
      "Document: Machine Learning - An Algorithmic Perspective, Second Edition, by Stephen Marsland.pdf\n",
      "\n",
      "Page: 76\n",
      "\n",
      "\n"
     ]
    }
   ],
   "source": [
    "retrieve_and_display(\"tell me about neural networks\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "And lastly, let's just put this into a function, to make it easy to load, process, & upload different documents a bit more efficiently."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### First let's do our imports and define the two global variables we need"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {},
   "outputs": [],
   "source": [
    "from langchain.document_loaders import PyPDFLoader\n",
    "from langchain.text_splitter import RecursiveCharacterTextSplitter\n",
    "    \n",
    "from langchain_huggingface import HuggingFaceEmbeddings\n",
    "from langchain_pinecone import PineconeVectorStore\n",
    "\n",
    "embedding_model = HuggingFaceEmbeddings()\n",
    "textbook_vector_store = PineconeVectorStore.from_existing_index(embedding=embedding_model, index_name=\"textbook-vector-store\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {},
   "outputs": [],
   "source": [
    "def process_textbook_to_vector_store():\n",
    "    pdf_loader = PyPDFLoader(\"Machine Learning - An Algorithmic Perspective, Second Edition, by Stephen Marsland.pdf\")\n",
    "    text_splitter = RecursiveCharacterTextSplitter(chunk_size=500, chunk_overlap=50, length_function=len)\n",
    "    \n",
    "    chunks = pdf_loader.load_and_split(text_splitter=text_splitter)\n",
    "    \n",
    "    texts = [chunk.page_content for chunk in chunks]\n",
    "    metadatas = [{'page': chunk.metadata['page'] + 1, 'document': chunk.metadata['source']} for chunk in chunks]\n",
    "    \n",
    "    textbook_vector_store.from_texts(\n",
    "        texts=texts,\n",
    "        embedding=embedding_model,\n",
    "        metadatas=metadatas,\n",
    "        index_name=\"textbook-vector-store\"\n",
    "    )\n",
    "    print(f\"Sucessfully Uploaded {len(texts)} Texts & {len(metadatas)} Metadatas to the Pinecone Textbook Vector Store.\")\n",
    "    return"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Sucessfully Uploaded 2285 Texts & 2285 Metadatas to the Pinecone Textbook Vector Store.\n"
     ]
    }
   ],
   "source": [
    "process_textbook_to_vector_store()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "##### Exercise: Try this on your own documents, notes or textbooks. Additionally, see if you can get it to work with Langchain's `PyPDFDirectoryLoader`."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Wonderful, we can see everything is working as expected. It's now time to move onto the next part."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Part 2: Creating the RAG Component\n",
    "\n",
    "Now, that we have gone through the process of loading, splitting, embedding and uploading our textbook to the vector database, and confirmed it's working as expected, it's time to connect this system up to a language model, so we can create our Question-Answering RAG Assistant.\n",
    "\n",
    "Like before, let's inspect our imports to make sure we understand the tools we'll be using. Our first import from the `langchain_openai` integration, where we import the `ChatOpenAI` module. Our second import is from the `langchain_core` parent module, where we specifically capture the `prompts` sub-module in which we import the `ChatPromptTemplate` class.\n",
    "\n",
    "We covered the `HuggingFaceEmbeddings` and `PineconeVectorStore` imports previously, so we won't worry about covering it here again."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "metadata": {},
   "outputs": [],
   "source": [
    "from langchain_openai import ChatOpenAI\n",
    "from langchain_core.prompts import ChatPromptTemplate\n",
    "\n",
    "from langchain_huggingface import HuggingFaceEmbeddings\n",
    "from langchain_pinecone import PineconeVectorStore"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Next, we need to create our prompt template. The `ChatPromptTemplate` class provides a variety of flexible prompt templates which accommodates to various situations. When using the `ChatOpenAI` module, its most common to use the `from_messages` method, which formats system messages, human messages, and AI messages in an easy-to-understand list format which OpenAI LLM's expect.\n",
    "\n",
    "We set a pretty specific system message, making sure to instruct the LLM to answer the users question only using the provided context, and if the context doesn't contain the answer, to let the user know this, instead of just hallucinating a response. We also, just pass through a `context` and `query` variable, which will get filled in with information at run time."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "metadata": {},
   "outputs": [],
   "source": [
    "template = ChatPromptTemplate.from_messages([\n",
    "    (\"system\", \"Using ONLY the provided context, answer the user's query. If the provided context doesn't contain the answer, then return 'I don't have enough information to accurately answer the question.'\"),\n",
    "    (\"system\", \"Context: {context}\"),\n",
    "    (\"human\", \"User Query: {query}\"),\n",
    "    (\"ai\", \"Answer:\")\n",
    "])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now we simply setup our inference model, in this case setting a `llm` variable to a `ChatOpenAI()` model, passing through our model name, tempature, max tokens, and timeout. There are many, many more parameters you can pass in here to customize your model, but this simple setup covers pretty much all the essentials."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "metadata": {},
   "outputs": [],
   "source": [
    "llm = ChatOpenAI(\n",
    "    model=\"gpt-4o\",\n",
    "    temperature=0.3,\n",
    "    max_tokens=None,\n",
    "    timeout=None,\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now we need a mechanism to capture a user query, saving it to a variable `user_query`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "metadata": {},
   "outputs": [],
   "source": [
    "user_query = input(\"Please Enter Your Query: \")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Even though we defined the embedding model and vector store variables earlier, since we are going to want this section to be independent to the last section, we will redefine them here for clarification. Its helpful to think of the earlier part as the \"Document Processing & Uploading Pipeline\" and this section as the \"Query, Retrieve, & Generate Pipeline\".\n",
    "\n",
    "Each should work independently, as we may want upload many, many more textbooks in the future, but the mechanism to query that vector store and generate a response will be the same."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 34,
   "metadata": {},
   "outputs": [],
   "source": [
    "embedding_model = HuggingFaceEmbeddings()\n",
    "textbook_vector_store = PineconeVectorStore.from_existing_index(embedding=embedding_model, index_name=\"textbook-vector-store\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now, just like before, we'll use the `similarity_search` method on out vector store to conduct a similarity search with respect to the user query, passing through a k value of 10 instead of 5 to improve the RAG performance and reliability of the system. We will set the retrieved context to a `retrieved_contexts` variable.\n",
    "\n",
    "Inspecting the format of the retrieved context, we see its in the same format as earlier. We need to loop through all the retrieved context, parsing and extracting the page_content to a context list."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 35,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[Document(metadata={'document': 'Machine Learning - An Algorithmic Perspective, Second Edition, by Stephen Marsland.pdf', 'page': 64.0}, page_content='neural networks so that they can do something useful.\\nThe question we need to think about ﬁrst is how our neurons can learn. We are going to\\nlook atsupervised learning for the next few chapters, which means that the algorithms will\\nlearn by example: the dataset that we learn from has the correct output values associated\\nwith each datapoint. At ﬁrst sight this might seem pointless, since if you already know the'),\n",
       " Document(metadata={'document': 'Machine Learning - An Algorithmic Perspective, Second Edition, by Stephen Marsland.pdf', 'page': 64.0}, page_content='neural networks so that they can do something useful.\\nThe question we need to think about ﬁrst is how our neurons can learn. We are going to\\nlook atsupervised learning for the next few chapters, which means that the algorithms will\\nlearn by example: the dataset that we learn from has the correct output values associated\\nwith each datapoint. At ﬁrst sight this might seem pointless, since if you already know the'),\n",
       " Document(metadata={'document': 'Machine Learning - An Algorithmic Perspective, Second Edition, by Stephen Marsland.pdf', 'page': 129.0}, page_content='a well-known introduction to neural networks:\\n•D.E. Rumelhart, G.E. Hinton, and R.J. Williams. Learning internal representations\\nby back-propagating errors. Nature, 323(99):533–536, 1986a.\\n•D.E. Rumelhart, J.L. McClelland, and the PDP Research Group, editors. Parallel\\nDistributed Processing . MIT Press, Cambridge, MA, 1986b.\\n•R. Lippmann. An introduction to computing with neural nets. IEEE ASSP Magazine ,\\npages 4–22, 1987.'),\n",
       " Document(metadata={'document': 'Machine Learning - An Algorithmic Perspective, Second Edition, by Stephen Marsland.pdf', 'page': 129.0}, page_content='a well-known introduction to neural networks:\\n•D.E. Rumelhart, G.E. Hinton, and R.J. Williams. Learning internal representations\\nby back-propagating errors. Nature, 323(99):533–536, 1986a.\\n•D.E. Rumelhart, J.L. McClelland, and the PDP Research Group, editors. Parallel\\nDistributed Processing . MIT Press, Cambridge, MA, 1986b.\\n•R. Lippmann. An introduction to computing with neural nets. IEEE ASSP Magazine ,\\npages 4–22, 1987.'),\n",
       " Document(metadata={'document': 'Machine Learning - An Algorithmic Perspective, Second Edition, by Stephen Marsland.pdf', 'page': 94.0}, page_content='to look at the neural network solution proposed by Rumelhart, Hinton, and McClelland,\\nthe Multi-layer Perceptron (MLP), which is still one of the most commonly used machine\\nlearning methods around. The MLP is one of the most common neural networks in use. It\\nis often treated as a ‘black box’, in that people use it without understanding how it works,\\nwhich often results in fairly poor results. Getting to the stage where we understand how it'),\n",
       " Document(metadata={'document': 'Machine Learning - An Algorithmic Perspective, Second Edition, by Stephen Marsland.pdf', 'page': 94.0}, page_content='to look at the neural network solution proposed by Rumelhart, Hinton, and McClelland,\\nthe Multi-layer Perceptron (MLP), which is still one of the most commonly used machine\\nlearning methods around. The MLP is one of the most common neural networks in use. It\\nis often treated as a ‘black box’, in that people use it without understanding how it works,\\nwhich often results in fairly poor results. Getting to the stage where we understand how it'),\n",
       " Document(metadata={'document': 'Machine Learning - An Algorithmic Perspective, Second Edition, by Stephen Marsland.pdf', 'page': 64.0}, page_content='3.2 NEURAL NETWORKS\\nOne thing that is probably fairly obvious is that one neuron isn’t that interesting. It doesn’t\\ndo very much, except ﬁre or not ﬁre when we give it inputs. In fact, it doesn’t even learn.\\nIf we feed in the same set of inputs over and over again, the output of the neuron never\\nvaries—it either ﬁres or does not. So to make the neuron a little more interesting we need\\nto work out how to make it learn, and then we need to put sets of neurons together into'),\n",
       " Document(metadata={'document': 'Machine Learning - An Algorithmic Perspective, Second Edition, by Stephen Marsland.pdf', 'page': 64.0}, page_content='3.2 NEURAL NETWORKS\\nOne thing that is probably fairly obvious is that one neuron isn’t that interesting. It doesn’t\\ndo very much, except ﬁre or not ﬁre when we give it inputs. In fact, it doesn’t even learn.\\nIf we feed in the same set of inputs over and over again, the output of the neuron never\\nvaries—it either ﬁres or does not. So to make the neuron a little more interesting we need\\nto work out how to make it learn, and then we need to put sets of neurons together into'),\n",
       " Document(metadata={'document': 'Machine Learning - An Algorithmic Perspective, Second Edition, by Stephen Marsland.pdf', 'page': 76.0}, page_content='In terms of learning about a set of data we have now reached the stage that neural\\nnetworks were up to in 1969. Then, two researchers, Minsky and Papert, published a book\\ncalled “Perceptrons.” The purpose of the book was to stimulate neural network research\\nby discussing the learning capabilities of the Perceptron, and showing what the network\\ncould and could not learn. Unfortunately, the book had another eﬀect: it eﬀectively killed'),\n",
       " Document(metadata={'document': 'Machine Learning - An Algorithmic Perspective, Second Edition, by Stephen Marsland.pdf', 'page': 76.0}, page_content='In terms of learning about a set of data we have now reached the stage that neural\\nnetworks were up to in 1969. Then, two researchers, Minsky and Papert, published a book\\ncalled “Perceptrons.” The purpose of the book was to stimulate neural network research\\nby discussing the learning capabilities of the Perceptron, and showing what the network\\ncould and could not learn. Unfortunately, the book had another eﬀect: it eﬀectively killed')]"
      ]
     },
     "execution_count": 35,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "retrieved_contexts = textbook_vector_store.similarity_search(query=user_query, k=10)\n",
    "retrieved_contexts"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "And that's exactly what we do here. We simply loop through each doc in `retrieved_contexts` and using a list comprehension, extract the parsed context to a new context list.\n",
    "\n",
    "Looking at the context list, this looks much cleaner now."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 36,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "['neural networks so that they can do something useful.\\nThe question we need to think about ﬁrst is how our neurons can learn. We are going to\\nlook atsupervised learning for the next few chapters, which means that the algorithms will\\nlearn by example: the dataset that we learn from has the correct output values associated\\nwith each datapoint. At ﬁrst sight this might seem pointless, since if you already know the',\n",
       " 'neural networks so that they can do something useful.\\nThe question we need to think about ﬁrst is how our neurons can learn. We are going to\\nlook atsupervised learning for the next few chapters, which means that the algorithms will\\nlearn by example: the dataset that we learn from has the correct output values associated\\nwith each datapoint. At ﬁrst sight this might seem pointless, since if you already know the',\n",
       " 'a well-known introduction to neural networks:\\n•D.E. Rumelhart, G.E. Hinton, and R.J. Williams. Learning internal representations\\nby back-propagating errors. Nature, 323(99):533–536, 1986a.\\n•D.E. Rumelhart, J.L. McClelland, and the PDP Research Group, editors. Parallel\\nDistributed Processing . MIT Press, Cambridge, MA, 1986b.\\n•R. Lippmann. An introduction to computing with neural nets. IEEE ASSP Magazine ,\\npages 4–22, 1987.',\n",
       " 'a well-known introduction to neural networks:\\n•D.E. Rumelhart, G.E. Hinton, and R.J. Williams. Learning internal representations\\nby back-propagating errors. Nature, 323(99):533–536, 1986a.\\n•D.E. Rumelhart, J.L. McClelland, and the PDP Research Group, editors. Parallel\\nDistributed Processing . MIT Press, Cambridge, MA, 1986b.\\n•R. Lippmann. An introduction to computing with neural nets. IEEE ASSP Magazine ,\\npages 4–22, 1987.',\n",
       " 'to look at the neural network solution proposed by Rumelhart, Hinton, and McClelland,\\nthe Multi-layer Perceptron (MLP), which is still one of the most commonly used machine\\nlearning methods around. The MLP is one of the most common neural networks in use. It\\nis often treated as a ‘black box’, in that people use it without understanding how it works,\\nwhich often results in fairly poor results. Getting to the stage where we understand how it',\n",
       " 'to look at the neural network solution proposed by Rumelhart, Hinton, and McClelland,\\nthe Multi-layer Perceptron (MLP), which is still one of the most commonly used machine\\nlearning methods around. The MLP is one of the most common neural networks in use. It\\nis often treated as a ‘black box’, in that people use it without understanding how it works,\\nwhich often results in fairly poor results. Getting to the stage where we understand how it',\n",
       " '3.2 NEURAL NETWORKS\\nOne thing that is probably fairly obvious is that one neuron isn’t that interesting. It doesn’t\\ndo very much, except ﬁre or not ﬁre when we give it inputs. In fact, it doesn’t even learn.\\nIf we feed in the same set of inputs over and over again, the output of the neuron never\\nvaries—it either ﬁres or does not. So to make the neuron a little more interesting we need\\nto work out how to make it learn, and then we need to put sets of neurons together into',\n",
       " '3.2 NEURAL NETWORKS\\nOne thing that is probably fairly obvious is that one neuron isn’t that interesting. It doesn’t\\ndo very much, except ﬁre or not ﬁre when we give it inputs. In fact, it doesn’t even learn.\\nIf we feed in the same set of inputs over and over again, the output of the neuron never\\nvaries—it either ﬁres or does not. So to make the neuron a little more interesting we need\\nto work out how to make it learn, and then we need to put sets of neurons together into',\n",
       " 'In terms of learning about a set of data we have now reached the stage that neural\\nnetworks were up to in 1969. Then, two researchers, Minsky and Papert, published a book\\ncalled “Perceptrons.” The purpose of the book was to stimulate neural network research\\nby discussing the learning capabilities of the Perceptron, and showing what the network\\ncould and could not learn. Unfortunately, the book had another eﬀect: it eﬀectively killed',\n",
       " 'In terms of learning about a set of data we have now reached the stage that neural\\nnetworks were up to in 1969. Then, two researchers, Minsky and Papert, published a book\\ncalled “Perceptrons.” The purpose of the book was to stimulate neural network research\\nby discussing the learning capabilities of the Perceptron, and showing what the network\\ncould and could not learn. Unfortunately, the book had another eﬀect: it eﬀectively killed']"
      ]
     },
     "execution_count": 36,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "context = [doc.page_content for doc in retrieved_contexts]\n",
    "context"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now that we have the user query and the context, we are ready to populate our prompt template with these values. So using the `invoke` method on our template, we pass through our `user_query` and `context` variables through to their respective key.\n",
    "\n",
    "We set this finalized prompt to the `prompt_value` variable."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 37,
   "metadata": {},
   "outputs": [],
   "source": [
    "prompt_value = template.invoke({\n",
    "    \"query\": user_query,\n",
    "    \"context\": context\n",
    "})"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "And now, that our prompt template has been populated, we are ready to pass through our prompt to the LLM using the `invoke` method on our `llm`, passing through our `prompt_value` which contains our prompt. We capture the response in the `response` variable, and print out the response.\n",
    "\n",
    "Wow! Look at that. We got the response from the LLM, we just need to parse the content of the response and then we have for all intents and purposes created a RAG Pipeline in Langchain."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 38,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "AIMessage(content='Neural networks are computational models inspired by the human brain. They consist of interconnected neurons that process information by responding to external inputs. A single neuron isn\\'t very interesting on its own, as it only fires or doesn\\'t fire when given inputs and doesn\\'t learn. To make neurons more useful, they need to be able to learn and be put together into networks.\\n\\nOne of the most common neural networks in use is the Multi-layer Perceptron (MLP), proposed by Rumelhart, Hinton, and McClelland. The MLP is often treated as a \\'black box\\' by users who don\\'t understand how it works, which can lead to poor results. Understanding how neural networks learn is crucial, and supervised learning is a common method where algorithms learn by example from a dataset with correct output values associated with each datapoint.\\n\\nFor further reading, some well-known introductions to neural networks include:\\n- D.E. Rumelhart, G.E. Hinton, and R.J. Williams\\' \"Learning internal representations by back-propagating errors\" (Nature, 1986)\\n- D.E. Rumelhart, J.L. McClelland, and the PDP Research Group\\'s \"Parallel Distributed Processing\" (MIT Press, 1986)\\n- R. Lippmann\\'s \"An introduction to computing with neural nets\" (IEEE ASSP Magazine, 1987)', additional_kwargs={'refusal': None}, response_metadata={'token_usage': {'completion_tokens': 279, 'prompt_tokens': 1173, 'total_tokens': 1452}, 'model_name': 'gpt-4o-2024-05-13', 'system_fingerprint': 'fp_157b3831f5', 'finish_reason': 'stop', 'logprobs': None}, id='run-42fc4bdb-313b-4434-b066-1be954702327-0', usage_metadata={'input_tokens': 1173, 'output_tokens': 279, 'total_tokens': 1452})"
      ]
     },
     "execution_count": 38,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "response = llm.invoke(prompt_value)\n",
    "response"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Extracting out the content, we get our final response from the LLM which is now our question answered using only chunks of retrieved context which we have just uploaded a few minutes prior. Beautiful."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 39,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'Neural networks are computational models inspired by the human brain. They consist of interconnected neurons that process information by responding to external inputs. A single neuron isn\\'t very interesting on its own, as it only fires or doesn\\'t fire when given inputs and doesn\\'t learn. To make neurons more useful, they need to be able to learn and be put together into networks.\\n\\nOne of the most common neural networks in use is the Multi-layer Perceptron (MLP), proposed by Rumelhart, Hinton, and McClelland. The MLP is often treated as a \\'black box\\' by users who don\\'t understand how it works, which can lead to poor results. Understanding how neural networks learn is crucial, and supervised learning is a common method where algorithms learn by example from a dataset with correct output values associated with each datapoint.\\n\\nFor further reading, some well-known introductions to neural networks include:\\n- D.E. Rumelhart, G.E. Hinton, and R.J. Williams\\' \"Learning internal representations by back-propagating errors\" (Nature, 1986)\\n- D.E. Rumelhart, J.L. McClelland, and the PDP Research Group\\'s \"Parallel Distributed Processing\" (MIT Press, 1986)\\n- R. Lippmann\\'s \"An introduction to computing with neural nets\" (IEEE ASSP Magazine, 1987)'"
      ]
     },
     "execution_count": 39,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "parsed_response = response.content\n",
    "parsed_response"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "For a bit of fun, let's inspect the prompt actually sent to the model...\n",
    "\n",
    "Oh, that's not very helpful"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 40,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "ChatPromptValue(messages=[SystemMessage(content=\"Using ONLY the provided context, answer the user's query. If the provided context doesn't contain the answer, then return 'I don't have enough information to accurately answer the question.'\"), SystemMessage(content=\"Context: ['neural networks so that they can do something useful.\\\\nThe question we need to think about ﬁrst is how our neurons can learn. We are going to\\\\nlook atsupervised learning for the next few chapters, which means that the algorithms will\\\\nlearn by example: the dataset that we learn from has the correct output values associated\\\\nwith each datapoint. At ﬁrst sight this might seem pointless, since if you already know the', 'neural networks so that they can do something useful.\\\\nThe question we need to think about ﬁrst is how our neurons can learn. We are going to\\\\nlook atsupervised learning for the next few chapters, which means that the algorithms will\\\\nlearn by example: the dataset that we learn from has the correct output values associated\\\\nwith each datapoint. At ﬁrst sight this might seem pointless, since if you already know the', 'a well-known introduction to neural networks:\\\\n•D.E. Rumelhart, G.E. Hinton, and R.J. Williams. Learning internal representations\\\\nby back-propagating errors. Nature, 323(99):533–536, 1986a.\\\\n•D.E. Rumelhart, J.L. McClelland, and the PDP Research Group, editors. Parallel\\\\nDistributed Processing . MIT Press, Cambridge, MA, 1986b.\\\\n•R. Lippmann. An introduction to computing with neural nets. IEEE ASSP Magazine ,\\\\npages 4–22, 1987.', 'a well-known introduction to neural networks:\\\\n•D.E. Rumelhart, G.E. Hinton, and R.J. Williams. Learning internal representations\\\\nby back-propagating errors. Nature, 323(99):533–536, 1986a.\\\\n•D.E. Rumelhart, J.L. McClelland, and the PDP Research Group, editors. Parallel\\\\nDistributed Processing . MIT Press, Cambridge, MA, 1986b.\\\\n•R. Lippmann. An introduction to computing with neural nets. IEEE ASSP Magazine ,\\\\npages 4–22, 1987.', 'to look at the neural network solution proposed by Rumelhart, Hinton, and McClelland,\\\\nthe Multi-layer Perceptron (MLP), which is still one of the most commonly used machine\\\\nlearning methods around. The MLP is one of the most common neural networks in use. It\\\\nis often treated as a ‘black box’, in that people use it without understanding how it works,\\\\nwhich often results in fairly poor results. Getting to the stage where we understand how it', 'to look at the neural network solution proposed by Rumelhart, Hinton, and McClelland,\\\\nthe Multi-layer Perceptron (MLP), which is still one of the most commonly used machine\\\\nlearning methods around. The MLP is one of the most common neural networks in use. It\\\\nis often treated as a ‘black box’, in that people use it without understanding how it works,\\\\nwhich often results in fairly poor results. Getting to the stage where we understand how it', '3.2 NEURAL NETWORKS\\\\nOne thing that is probably fairly obvious is that one neuron isn’t that interesting. It doesn’t\\\\ndo very much, except ﬁre or not ﬁre when we give it inputs. In fact, it doesn’t even learn.\\\\nIf we feed in the same set of inputs over and over again, the output of the neuron never\\\\nvaries—it either ﬁres or does not. So to make the neuron a little more interesting we need\\\\nto work out how to make it learn, and then we need to put sets of neurons together into', '3.2 NEURAL NETWORKS\\\\nOne thing that is probably fairly obvious is that one neuron isn’t that interesting. It doesn’t\\\\ndo very much, except ﬁre or not ﬁre when we give it inputs. In fact, it doesn’t even learn.\\\\nIf we feed in the same set of inputs over and over again, the output of the neuron never\\\\nvaries—it either ﬁres or does not. So to make the neuron a little more interesting we need\\\\nto work out how to make it learn, and then we need to put sets of neurons together into', 'In terms of learning about a set of data we have now reached the stage that neural\\\\nnetworks were up to in 1969. Then, two researchers, Minsky and Papert, published a book\\\\ncalled “Perceptrons.” The purpose of the book was to stimulate neural network research\\\\nby discussing the learning capabilities of the Perceptron, and showing what the network\\\\ncould and could not learn. Unfortunately, the book had another eﬀect: it eﬀectively killed', 'In terms of learning about a set of data we have now reached the stage that neural\\\\nnetworks were up to in 1969. Then, two researchers, Minsky and Papert, published a book\\\\ncalled “Perceptrons.” The purpose of the book was to stimulate neural network research\\\\nby discussing the learning capabilities of the Perceptron, and showing what the network\\\\ncould and could not learn. Unfortunately, the book had another eﬀect: it eﬀectively killed']\"), HumanMessage(content='User Query: tell me about neural networks'), AIMessage(content='Answer:')])"
      ]
     },
     "execution_count": 40,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "prompt_value"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's parse this so it looks a bit nicer..."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 41,
   "metadata": {},
   "outputs": [],
   "source": [
    "messages = prompt_value.messages\n",
    "\n",
    "system_instructions = messages[0].content\n",
    "context_message = messages[1].content\n",
    "user_query_message = messages[2].content\n",
    "llm_response = parsed_response"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 42,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "SYSTEM INSTRUCTIONS:\n",
      "Using ONLY the provided context, answer the user's query. If the provided context doesn't contain the answer, then return 'I don't have enough information to accurately answer the question.'\n",
      "CONTEXT SENT TO MODEL:\n",
      "Context: ['neural networks so that they can do something useful.\\nThe question we need to think about ﬁrst is how our neurons can learn. We are going to\\nlook atsupervised learning for the next few chapters, which means that the algorithms will\\nlearn by example: the dataset that we learn from has the correct output values associated\\nwith each datapoint. At ﬁrst sight this might seem pointless, since if you already know the', 'neural networks so that they can do something useful.\\nThe question we need to think about ﬁrst is how our neurons can learn. We are going to\\nlook atsupervised learning for the next few chapters, which means that the algorithms will\\nlearn by example: the dataset that we learn from has the correct output values associated\\nwith each datapoint. At ﬁrst sight this might seem pointless, since if you already know the', 'a well-known introduction to neural networks:\\n•D.E. Rumelhart, G.E. Hinton, and R.J. Williams. Learning internal representations\\nby back-propagating errors. Nature, 323(99):533–536, 1986a.\\n•D.E. Rumelhart, J.L. McClelland, and the PDP Research Group, editors. Parallel\\nDistributed Processing . MIT Press, Cambridge, MA, 1986b.\\n•R. Lippmann. An introduction to computing with neural nets. IEEE ASSP Magazine ,\\npages 4–22, 1987.', 'a well-known introduction to neural networks:\\n•D.E. Rumelhart, G.E. Hinton, and R.J. Williams. Learning internal representations\\nby back-propagating errors. Nature, 323(99):533–536, 1986a.\\n•D.E. Rumelhart, J.L. McClelland, and the PDP Research Group, editors. Parallel\\nDistributed Processing . MIT Press, Cambridge, MA, 1986b.\\n•R. Lippmann. An introduction to computing with neural nets. IEEE ASSP Magazine ,\\npages 4–22, 1987.', 'to look at the neural network solution proposed by Rumelhart, Hinton, and McClelland,\\nthe Multi-layer Perceptron (MLP), which is still one of the most commonly used machine\\nlearning methods around. The MLP is one of the most common neural networks in use. It\\nis often treated as a ‘black box’, in that people use it without understanding how it works,\\nwhich often results in fairly poor results. Getting to the stage where we understand how it', 'to look at the neural network solution proposed by Rumelhart, Hinton, and McClelland,\\nthe Multi-layer Perceptron (MLP), which is still one of the most commonly used machine\\nlearning methods around. The MLP is one of the most common neural networks in use. It\\nis often treated as a ‘black box’, in that people use it without understanding how it works,\\nwhich often results in fairly poor results. Getting to the stage where we understand how it', '3.2 NEURAL NETWORKS\\nOne thing that is probably fairly obvious is that one neuron isn’t that interesting. It doesn’t\\ndo very much, except ﬁre or not ﬁre when we give it inputs. In fact, it doesn’t even learn.\\nIf we feed in the same set of inputs over and over again, the output of the neuron never\\nvaries—it either ﬁres or does not. So to make the neuron a little more interesting we need\\nto work out how to make it learn, and then we need to put sets of neurons together into', '3.2 NEURAL NETWORKS\\nOne thing that is probably fairly obvious is that one neuron isn’t that interesting. It doesn’t\\ndo very much, except ﬁre or not ﬁre when we give it inputs. In fact, it doesn’t even learn.\\nIf we feed in the same set of inputs over and over again, the output of the neuron never\\nvaries—it either ﬁres or does not. So to make the neuron a little more interesting we need\\nto work out how to make it learn, and then we need to put sets of neurons together into', 'In terms of learning about a set of data we have now reached the stage that neural\\nnetworks were up to in 1969. Then, two researchers, Minsky and Papert, published a book\\ncalled “Perceptrons.” The purpose of the book was to stimulate neural network research\\nby discussing the learning capabilities of the Perceptron, and showing what the network\\ncould and could not learn. Unfortunately, the book had another eﬀect: it eﬀectively killed', 'In terms of learning about a set of data we have now reached the stage that neural\\nnetworks were up to in 1969. Then, two researchers, Minsky and Papert, published a book\\ncalled “Perceptrons.” The purpose of the book was to stimulate neural network research\\nby discussing the learning capabilities of the Perceptron, and showing what the network\\ncould and could not learn. Unfortunately, the book had another eﬀect: it eﬀectively killed']\n",
      "ENTERED USER QUERY:\n",
      "User Query: tell me about neural networks\n",
      "RESPONSE FROM LLM:\n",
      "Neural networks are computational models inspired by the human brain. They consist of interconnected neurons that process information by responding to external inputs. A single neuron isn't very interesting on its own, as it only fires or doesn't fire when given inputs and doesn't learn. To make neurons more useful, they need to be able to learn and be put together into networks.\n",
      "\n",
      "One of the most common neural networks in use is the Multi-layer Perceptron (MLP), proposed by Rumelhart, Hinton, and McClelland. The MLP is often treated as a 'black box' by users who don't understand how it works, which can lead to poor results. Understanding how neural networks learn is crucial, and supervised learning is a common method where algorithms learn by example from a dataset with correct output values associated with each datapoint.\n",
      "\n",
      "For further reading, some well-known introductions to neural networks include:\n",
      "- D.E. Rumelhart, G.E. Hinton, and R.J. Williams' \"Learning internal representations by back-propagating errors\" (Nature, 1986)\n",
      "- D.E. Rumelhart, J.L. McClelland, and the PDP Research Group's \"Parallel Distributed Processing\" (MIT Press, 1986)\n",
      "- R. Lippmann's \"An introduction to computing with neural nets\" (IEEE ASSP Magazine, 1987)\n"
     ]
    }
   ],
   "source": [
    "print(f\"SYSTEM INSTRUCTIONS:\\n{system_instructions}\")\n",
    "print(f\"CONTEXT SENT TO MODEL:\\n{context_message}\")\n",
    "print(f\"ENTERED USER QUERY:\\n{user_query_message}\")\n",
    "print(f\"RESPONSE FROM LLM:\\n{llm_response}\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Time to put it in a function"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### First let's do our imports and define the two global variables we need"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 44,
   "metadata": {},
   "outputs": [],
   "source": [
    "from langchain_huggingface import HuggingFaceEmbeddings\n",
    "from langchain_pinecone import PineconeVectorStore\n",
    "\n",
    "from langchain_openai import ChatOpenAI\n",
    "from langchain_core.prompts import ChatPromptTemplate\n",
    "\n",
    "embedding_model = HuggingFaceEmbeddings()\n",
    "textbook_vector_store = PineconeVectorStore.from_existing_index(embedding=embedding_model, index_name=\"textbook-vector-store\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 45,
   "metadata": {},
   "outputs": [],
   "source": [
    "def textbook_rag():\n",
    "\n",
    "    user_query = input(\"Please Enter Your Query: \")\n",
    "\n",
    "    template = ChatPromptTemplate.from_messages([\n",
    "        (\"system\", \"Using ONLY the provided context, answer the user's query. If the provided context doesn't contain the answer, then return 'I don't have enough information to accurately answer the question'\"),\n",
    "        (\"system\", \"Context: {context}\"),\n",
    "        (\"human\", \"User Query: {query}\"),\n",
    "        (\"ai\", \"Answer:\")\n",
    "    ])\n",
    "\n",
    "    llm = ChatOpenAI(\n",
    "        model=\"gpt-4o\",\n",
    "        temperature=0.3,\n",
    "        max_tokens=None,\n",
    "        timeout=None,\n",
    "    )\n",
    "\n",
    "    retrieved_contexts = textbook_vector_store.similarity_search(query=user_query, k=10)\n",
    "    context = [doc.page_content for doc in retrieved_contexts]\n",
    "\n",
    "    prompt_value = template.invoke({\n",
    "        \"query\": user_query,\n",
    "        \"context\": context\n",
    "    })\n",
    "\n",
    "    response = llm.invoke(prompt_value)\n",
    "    parsed_response = response.content\n",
    "    print(parsed_response)\n",
    "    return"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 47,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Neural networks are systems of interconnected neurons that can learn and perform tasks. They consist of multiple neurons that can fire or not fire based on inputs. To make neurons more interesting and useful, they need to be able to learn and be organized into sets. One common type of neural network is the Multi-layer Perceptron (MLP), proposed by Rumelhart, Hinton, and McClelland, which is widely used in machine learning. Neural networks can learn from datasets with correct output values associated with each datapoint through supervised learning.\n"
     ]
    }
   ],
   "source": [
    "textbook_rag()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Concluding Remarks\n",
    "Perfect, we have now created a complete, albeit simple RAG implementation using only Langchain components/integrations, starting at external document loading, splitting, embedding, and uploading, and then conducting on the spot retrieval based on a user query to a GPT-4o inference model, printing out the results.\n",
    "\n",
    "Please use this notebook as a base, and in your own time, experiment by adding components, swapping out components, and creating some cool applications!\n",
    "\n",
    "Thank you for listening!"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.11.6"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}