Initial import

.gitignore

@@ -0,0 +1,5 @@
+.DS_Store
+.ipynb_checkpoints
+data/tmp/
+data/audio/tmp
+models/tmp

README.md

@@ -0,0 +1,31 @@
+
+<br />
+<div align="center">
+  <h2 align="center">AI on a Microbudget</h2>
+
+  <a href="https://github.com/clstaudt/audiomariner">
+    <img src="fig/logo.png" alt="Logo" width="140" height="140">
+  </a>
+
+
+  <p align="justify">
+<i>In recent years, the AI field has pursued ever larger models, trained at “eye-watering” cost. In this talk we explore ideas for the rest of us, the GPU-poor. We’ll show you how to make do with less – less computing power, less person power, less data – while still building powerful models.</i>
+  </p>
+</div>
+
+
+## Part 1: Methods of Machine Learning Miniaturization
+
+Current progress in AI has seen remarkable capabilities emerging from simple prediction tasks – if we scale them massively. Surprisingly, we get sparks of reasoning and intelligence in a model that was trained to do little more than masked word prediction. Since that realization the AI field has pursued ever larger models, trained at “eye-watering” cost. If scaling is all you need – does it follow that, in practice, money is all you need?
+
+In this talk we explore ideas for the rest of us, the GPU-poor. Taking examples from language processing and computer vision, we’ll show you how to make do with less – less computing power, less person power, less data – while still building powerful models. We will introduce a set of methods and open source tools for the efficient reuse and miniaturization of models, including transfer learning and fine-tuning, knowledge distillation, and model quantization. We will also discuss how to choose efficient model architectures, and investigate ways in which small and specialized models can outperform large models. Our talk aims to provide an overview for ML practitioners, draws from our combined project experience, and is accompanied by a repository of code examples to get you started with building AI on a microbudget.
+
+📙 [**Companion Notebooks: Table of Contents**](notebooks/ai-on-a-microbudget.ipynb)
+
+
+## Setup
+
+
+
+---
+_This material is licensed under a [Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International (CC BY-NC-SA 4.0)](https://creativecommons.org/licenses/by-nc-sa/4.0/). Copyright © 2024 [Christian Staudt](https://clstaudt.me), [Katharina Rasch](https://krasch.io)_

notebooks/ai-on-a-microbudget.ipynb

@@ -0,0 +1,85 @@
+{
+ "cells": [
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "# AI on a Microbudget "
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "**In recent years, the AI field has pursued ever larger models, trained at “eye-watering” cost. In this talk we explore ideas for the rest of us, the GPU-poor. We’ll show you how to make do with less – less computing power, less person power, less data – while still building powerful models.**"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## Part 1: Methods of Machine Learning Miniaturization\n",
+    "\n",
+    "Current progress in AI has seen remarkable capabilities emerging from simple prediction tasks – if we scale them massively. Surprisingly, we get sparks of reasoning and intelligence in a model that was trained to do little more than masked word prediction. Since that realization the AI field has pursued ever larger models, trained at “eye-watering” cost. If scaling is all you need – does it follow that, in practice, money is all you need?\n",
+    "\n",
+    "In this talk we explore ideas for the rest of us, the GPU-poor. Taking examples from language processing and computer vision, we’ll show you how to make do with less – less computing power, less person power, less data – while still building powerful models. We will introduce a set of methods and open source tools for the efficient reuse and miniaturization of models, including transfer learning and fine-tuning, model distillation, and model quantization. We will also discuss how to choose efficient model architectures, and investigate ways in which small and specialized models can outperform large models. Our talk aims to provide an overview for ML practitioners, draws from our combined project experience, and is accompanied by a repository of code examples to get you started with building AI on a microbudget."
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "1. [🧠**Transfer Learning**](aioamb-transfer-learning.ipynb)<br>\n",
+    "    _Save on training resources by reusing the knowledge of pre-trained models_<br>\n",
+    "\n",
+    "    **Example Workflows**<br>\n",
+    "    \n",
+    "    A. [**Adapting a Pre-Trained Convolutional Neural Network for Custom Image Classification**](./aioamb-transfer-learning-image-classification.ipynb)\n",
+    "    \n",
+    "\n",
+    "2. [📊 **Quantization**]()<br>\n",
+    "    _Smaller models through lossy compression of model weights_<br>\n",
+    "\n",
+    "    **Example Workflows**<br>\n",
+    "\n",
+    "    A. [**Running a Large Language Model with Different Levels of Quantization**](./aioamb-quantization-llm.ipynb)\n",
+    "\n",
+    "3. [⚗️ **Model Distillation**]()<br>\n",
+    "    _Refining complex models into simpler, efficient versions_<br>\n",
+    "\n",
+    "    **Example Workflows**<br>\n",
+    "\n",
+    "    A. [**Faster Speech Transcription through Model Distillation**](./aioamb-distillation-whisper.ipynb)\n"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "---\n",
+    "_This notebook is licensed under a [Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International (CC BY-NC-SA 4.0)](https://creativecommons.org/licenses/by-nc-sa/4.0/). Copyright © 2024 [Christian Staudt](https://clstaudt.me), [Katharina Rasch](https://krasch.io)_"
+   ]
+  }
+ ],
+ "metadata": {
+  "kernelspec": {
+   "display_name": "ai-on-a-microbudget",
+   "language": "python",
+   "name": "python3"
+  },
+  "language_info": {
+   "codemirror_mode": {
+    "name": "ipython",
+    "version": 3
+   },
+   "file_extension": ".py",
+   "mimetype": "text/x-python",
+   "name": "python",
+   "nbconvert_exporter": "python",
+   "pygments_lexer": "ipython3",
+   "version": "3.10.12"
+  }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 2
+}