Open-source ML observability course

Welcome to the free Open-source ML observability course from Evidently AI.

📌 Useful links

• Newsletter. Sign up to receive course updates and be notified when the next cohort starts.
• Course materials. All 40 lessons with videos, course notes, and code examples are publicly available.
• Code examples. Are published in this GitHub repository.
• YouTube playlist. Subscribe to the course YouTube playlist.
• Discord community. Join the community to ask questions and chat with others.

🙋 How to participate?

• Learn at your own pace. We published all 40 lessons with videos, course notes, and code examples.
• Join the course cohort. To submit assignments and earn a certificate of completion, you must enroll in the course cohort. Sign up to save your seat and be notified when the next cohort starts.

📈 What the course is about?

This course is a deep dive into ML model observability and monitoring.

We explore different types of evaluations, from data quality to data drift, and how this fits in the model lifecycle. We also cover the engineering aspect of ML observability and how to integrate it with your ML services and pipelines.

👩‍💻 Who is it for?

This course is useful to professionals who have dealt with ML models in production and those preparing to deploy ML models:

• Data scientists,
• ML engineers,
• Technical product managers,
• Analysts.

🏆 Will I get a course certificate?

To earn a certificate, you must successfully complete all the assignments.

Note that the option to receive the certificate is available only to those who participate in the course cohort. The next cohort will take place in 2024. Sign up to get updates when it starts.

👩‍🎓 Course syllabus

ML observability course is organized into six modules. You can follow the complete course syllabus or pick only the modules that are most relevant to you.

📚 Module 1. Introduction to ML monitoring and observability

📈 Module 2. ML monitoring metrics: model quality, data quality, data drift

🔡 Module 3. ML monitoring for unstructured data: NLP, LLM, and embeddings

🏗 Module 4. Designing effective ML monitoring

✅ Module 5. ML pipeline validation and testing

📊 Module 6. Deploying an ML monitoring dashboard

💻 Are there any prerequisites?

There are both theoretical and code-focused modules that require knowledge of Python. We will walk you through the code, but you can skip these parts and still learn a lot.

💬 What if I need help?

Join our Discord #-ml-observability-course channel to chat with fellow learners and get support from the course team.

💌 Is there a newsletter?

Yes, sign up to receive course updates and be notified when the next cohort starts.

🧠 Our approach

• Blend of theory and practice. The course combines key concepts of ML observability and monitoring with practice-oriented tasks.
• Practical code examples. We provide end-to-end deployment blueprints and walk you through the code examples.
• Focus on open-source. The course is built upon open-source tools to make ML observability accessible to all.
• The course is free and open to everyone. All course videos are public so you can rewatch them anytime.