chapters/welcome.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "tags": []
   },
   "source": [
    "# Welcome to AI-Powered Search"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "tags": []
   },
   "source": [
    "These notebooks accompany the book [AI-Powered Search](http://aiPoweredSearch.com) by Trey Grainger, Doug Turnbull, and Max Irwin (Manning Publications, 2024). \n",
    "\n",
    "You can find the book at https://aiPoweredSearch.com, including an electronic copy enabling you to link directly to these notebooks as you read.\n",
    "\n",
    "The examples here are designed for use when following along with the book and not as standalone reference materials, as most of the context and explanation behind the code examples exists in these book only.\n",
    "\n",
    "These notebooks are intended to run within a Docker container (`aips-notebooks`) alongside other containers which may be running your favorite supported search engine or vector database listed in the top-level **engines** folder. To build and start the Docker container, just run:\n",
    "\n",
    "```\n",
    "git clone https://github.com/treygrainger/ai-powered-search.git\n",
    "cd ai-powered-search\n",
    "docker compose up\n",
    "```\n",
    "\n",
    "The book's code examples work with many different search engines and vector databases. By default, the book uses Apache Solr as the primary storage, matching, and ranking engine, but you can easily swap in your your favorite engine by running the `set_engine` function here: [engines/supported-engines.ipynb](engines/supported-engines.ipynb).\n",
    "\n",
    "If you find the examples not running as expected. Please ensure you have built the entire AI-Powered Search project and are running it per the instructions at:\n",
    "http://github.com/treygrainger/ai-powered-search\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Run Healthcheck:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "import sys; sys.path.append(\"..\"); from aips import healthcheck\n",
    "healthcheck()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "\n",
    "\n",
    "\n",
    "## Table of Contents\n",
    "\n",
    "----\n",
    "### Part 1: Modern Search Relevance\n",
    "---\n",
    "\n",
    "#### Ch1 - Introducting AI-powered Search\n",
    "- No notebooks\n",
    "\n",
    "#### Ch2 - Working with Natural Language\n",
    "- No notebooks\n",
    "\n",
    "#### Ch3 - Ranking and Content-based Relevance\n",
    "1. [Vectors and Text Similarity](ch03/1.vectors-and-text-similarity.ipynb)\n",
    "2. [Controlling Relevance](ch03/2.controlling-relevance.ipynb)\n",
    "\n",
    "#### Ch4 - Crowdsourced Relevance\n",
    "1. [Setting up the Retrotech Dataset](ch04/1.setting-up-the-retrotech-dataset.ipynb)\n",
    "2. [Signals Boosting](ch04/2.signals-boosting.ipynb)\n",
    "\n",
    "---\n",
    "### Part 2: Learning Domain-specific Intent\n",
    "---\n",
    "\n",
    "#### Ch5 - Knowledge Graph Learning\n",
    "1. [Automatic Extraction of Knowledge Graphs from Content](ch05/1.open-information-extraction.ipynb)\n",
    "2. [Setting up the Knowledge Graph Datasets](ch05/2.index-datasets.ipynb)\n",
    "3. [Working with Semantic Knowledge Graphs](ch05/3.semantic-knowledge-graph.ipynb)\n",
    "\n",
    "#### Ch6 - Using Context to Learn Domain-specific Language\n",
    "1. [Query Classification and Disambiguation with Semantic Knowledge Graphs](ch06/1.skg-classification-disambiguation.ipynb)\n",
    "2. [Related Keywords Detection from Signals](ch06/2.related-keywords-from-signals.ipynb)\n",
    "3. [Spelling Correction from Signals](ch06/3.spell-correction.ipynb)\n",
    "\n",
    "#### Ch7 - Interpreting Query Intent Through Semantic Search \n",
    "1. [Setting up the Reviews Dataset](ch07/1.index-datasets.ipynb)\n",
    "2. [Semantic Search Application](ch07/2.semantic-search.ipynb)\n",
    "\n",
    "---\n",
    "### Part 3: Reflected Intelligence\n",
    "---\n",
    "\n",
    "#### Ch8 - Signals Boosting Models\n",
    "1. [Signals Boosting Models](ch08/1.signals-boosting.ipynb)\n",
    "\n",
    "#### Ch9 - Personalized Search\n",
    "1. [Personalized Search](ch09/1.personalization.ipynb)\n",
    "\n",
    "#### Ch10 - Learning to Rank for Generalizable Search Relevance\n",
    "1. [Setting up TheMovieDB Dataset](ch10/1.setup-the-movie-db.ipynb)\n",
    "2. [Judgments and Feature Logging](ch10/2.judgments-and-logging.ipynb)\n",
    "3. [Feature Normalization and Pairwise Transform](ch10/3.pairwise-transform.ipynb)\n",
    "4. [Train and Evaluate The Model](ch10/4.train-and-evaluate-the-model.ipynb)\n",
    "\n",
    "#### Ch11 - Automating Learning to Rank with Click Models\n",
    "1. [Setting up Search Sessions Data](ch11/0.setup.ipynb)\n",
    "2. [Click Through Rate Model](ch11/1.click-through-rate-judgments.ipynb)\n",
    "3. [Using SDBN Click Model To Overcome Position Bias](ch11/2.sdbn-judgments-to-overcome-position-bias.ipynb)\n",
    "4. [Dealing with Confidence Bias](ch11/3.sdbn-confidence-bias.ipynb)\n",
    "5. [End to End Automated LTR](ch11/4.end-to-end-auto-ltr.ipynb)\n",
    "\n",
    "#### Ch12 - Overcoming Ranking Bias through Active Learning\n",
    "1. [Setting up Search Sessions Data](ch12/0.setup.ipynb)\n",
    "2. [AB Testing to Active Learning](ch12/1.ab-testing-to-active-learning.ipynb)\n",
    "\n",
    "---\n",
    "### Part 4: The Search Frontier\n",
    "---\n",
    "\n",
    "#### Ch13 - Semantic Search with Dense Vectors\n",
    "1. [Setting up the Outdoors Dataset](ch13/1.setting-up-the-outdoors-dataset.ipynb)\n",
    "2. [Introduction to Transformers](ch13/2.introduction-to-transformers.ipynb)\n",
    "3. [Natural Language Autocomplete](ch13/3.natural-language-autocomplete.ipynb)\n",
    "4. [Semantic Search with Dense Vector Embeddings](ch13/4.semantic-search.ipynb)\n",
    "\n",
    "#### Ch14 - Question Answering with a Fine-tuned Large Language Model\n",
    "1. [Question Answering Visualizer](ch14/1.question-answering-visualizer.ipynb)\n",
    "2. [Question Answering Data Preparation](ch14/2.question-answering-CPU-data-preparation.ipynb)\n",
    "3. [Question Answering Fine Tuning](ch14/3.question-answering-GPU-fine-tuning.ipynb)\n",
    "4. [Question Answering Demo Application](ch14/4.question-answering-CPU-demo-application.ipynb)\n",
    "\n",
    "#### Ch15 - Foundation Models and Emerging Search Paradigms\n",
    "1. [LLM Exploration](ch15/1.llm-exploration.ipynb)\n",
    "1. [Multimodal Search](ch15/2.multimodal-search.ipynb)\n"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.10.9"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}