This is a proposal for an initial structure of public repositories for educational material and demos. The main idea here is to make available a set of 15+ notebooks with end-to-end experiments split into subjects according to different topics. This way, we would have smaller repos (with no more than 5 experiments) - avoiding the current scenario of having to download a single big repo to run any experiment, but also without having too many different repos to give maintenance.
- Using AI Studio in 6 Steps
- Deep Learning in AI Studio
- Integrating with NGC
- Gen AI with Galileo and AIS
Below, we find a description of each specific subject/repository, as well as the intended demos/tutorials to be included on each one
- Currently saved on ai-studio fundamentals folder
This repo would have a different structure than the other ones. Five different notebooks would be used to illustrate different foundational features of AI Studio, in separate tutorials. These notebooks are:
- Iris classification: One of the most traditional examples in ML, this notebook will be used to illustrate the most simple usage of AI Studio (section 1)
- Movie experiment: This notebook is an example of a recommendation system, which can be used to show features as Data Fabric, ML Flow and Tensorboard monitoring and model deployment.
- Tale of two cities: A nice example for different data visualization techniques, can also be used to demonstrate data fabric and installation of libraries/customization of environments
- MNIST classification: End-to-end introdutory example of Computer Vision with AI Studio
- Spam Classification: End-to-end introdutory example of Natural Language Processing with AI Studio
- classification/iris
- Needs to change the load_data, to use sklearn one
- What is a project on AI Studio, and how does it work?
- How to create a simple project?
- How to add a simple Workspace inside a project (Minimal vs Data Science workspace)
- How to connect to a Github Repository
- How to access your notebook inside the workspace
- What are the local folders?
- Introduce Movie experiment example
- Introduce tale of two cities project
- How to add local folders to my project
- How to access these local folders from inside the workspace
- How to add cloud folders to my project
- Why should you restart your workspace to access data fabric
- Show data visualization in previous examples
- Use movie experiment example to show monitoring
- Can we change TB logging to use tensorboard library instead of TF
- Data visualization tools included
- Using MLFlow to monitoring
- Using Tensorboard to monitoring
- Use the same notebooks in previous sessions
- Try to run them on minimal workspace, to show how to show the effects on environment
- Installing libraries with PIP
- Custom workspaces/environments
- Using conda environments manually
- Use movie experiment example to show Model Service (make sure it works)
- Create a quick UI later
- Logging and registering models in MLFlow
- Deploying a service (swagger interface)
- Adding a UI to the service
- MNIST (change Keras to scikit learn, so we do not use Tensorflow)
- SpamClassification
- Use MNIST to show how to work with images
- Use Spam classification to show how to work with text
- Select in the future
- Briefly explain the extra notebooks
- Folder: deep-learning-in-ais
Starting in this second subject, each individual demo/tutorial is associated with a single notebook (and auxiliary files). In this section we will have 4 examples on how to use Tensorflow and Pytorch inside AI Studio, using GPU resources and our Deep Learning workspaces to easily put in practice to process images and language.
- Basic Image Classification notebook
- Use Deep Learning image to work with a Image Classification example
- Use Data from datafabric
- Ensure that MLFlow/Tensorboard are being used in the code
- Ensure that multiple runs are made, with different configurations, to allow comparison
- Ensure that GPU is being used
- Super resolution example
- Use Deep Learning image and the super resolution problem
- Use cloud data from Data Fabric
- Ensure that MLFlow/Tensorboard are being used
- Deploy a super resolution service with UI
- Shakespeare example
- Explain basic character generation using statistical patterns
- Bert QA
- Explain basic usage of Hugging Face and transformers
- Folder: ngc-integration
Here, we will aggregate the demos that use NGC resources, to show how to use them to our use cases
- Rapids/Pandas Stock Demo
- Show how Rapids can accelerate data operations done in pandas
- Rapids OpenCellID example
- Expand Rapids acceleration to Data visualization of geo processing
- Audio translation examples
- Nemo Framework image and how to use it in AI Studio
- Download models using NGC integration
- Running the models inside notebook
- Publishing a service using the models
This actually is the same repository as the templates for Prometheus
- Prometheus chatbot template
- Creating a chatbot with langchain
- Using OpenAI model
- Evaluating experiment with Galileo Evaluate
- Using feedbacks from Galileo Evaluate to improve prompt
- Prometheus chatbot template
- Instrumenting the code with Galileo Observe
- Monitoring the code with Galileo Observe interface
- Instrumenting the code with Galileo Protect
- Deploying the model locally
- Monitoring Galileo Protect errors and alerts
- Prometheus summarization template
- Creating a custom pipeline for summarization
- Using multiple data connectors
- Using locally deployed model
- Custom chains on Galileo Evaluate
- Custom scorers on Galileo Evaluate
- Deploying the service and adding Observe and Protect
- Prometheus code generation example
- Explain the content of this example
- Prometheus text generation example
- Explain the content of this example