From 8f17d2e2336fafd23f989c5b257141b8f25f64ee Mon Sep 17 00:00:00 2001 From: Nikita Menshikov Date: Mon, 9 Jun 2025 11:30:31 +0300 Subject: [PATCH 01/56] add team folder and nessesary basics --- content/docs/2025/Kolobok/_index.md | 4 ++++ content/docs/2025/Kolobok/week1.md | 0 2 files changed, 4 insertions(+) create mode 100644 content/docs/2025/Kolobok/_index.md create mode 100644 content/docs/2025/Kolobok/week1.md diff --git a/content/docs/2025/Kolobok/_index.md b/content/docs/2025/Kolobok/_index.md new file mode 100644 index 000000000..a6e6f3f63 --- /dev/null +++ b/content/docs/2025/Kolobok/_index.md @@ -0,0 +1,4 @@ +--- +bookCollapseSection: true +title: "Kolobok" +--- \ No newline at end of file diff --git a/content/docs/2025/Kolobok/week1.md b/content/docs/2025/Kolobok/week1.md new file mode 100644 index 000000000..e69de29bb From a281830ec592a93bd12f3631b935d69e93f166ae Mon Sep 17 00:00:00 2001 From: Nikita Menshikov <74507368+NikitaMensh@users.noreply.github.com> Date: Mon, 9 Jun 2025 16:38:44 +0300 Subject: [PATCH 02/56] Update week1.md --- content/docs/2025/Kolobok/week1.md | 73 ++++++++++++++++++++++++++++++ 1 file changed, 73 insertions(+) diff --git a/content/docs/2025/Kolobok/week1.md b/content/docs/2025/Kolobok/week1.md index e69de29bb..36b591c29 100644 --- a/content/docs/2025/Kolobok/week1.md +++ b/content/docs/2025/Kolobok/week1.md @@ -0,0 +1,73 @@ +--- +title: "Week #1" +--- + +# Week #1 + +## Project description + +### Project name: *Kolobok* + +**Code repository**: [https://github.com/IU-Capstone-Project-2025/Kolobok](https://github.com/IU-Capstone-Project-2025/Kolobok) + +** + +### **Team Members** + +| Team Member | Telegram Alias | Email Address | Track | Responsibilities | +|-----------------------------------------|------------------|------------------------------------|---------------------------------------------|------------------------------------------------------------------------------------------| +| [Nikita Menshikov (Lead)] | [@NikitaMensh] | [n.menshikov@innopolis.university] | [Project Manager] | [Team management, reports writing, customer communication, work environment maintenance] | +| [Nikita Zagainov] | [@V1adych] | [n.zagainov@innopolis.university] | [ML engineer] | [Core models research & development] | +| [Vladislav Strelkov] | [@motrooo] | [v.strelkov@innopolis.university] | [DevOps] | [Product deployment, CI/CD] | +| [Sergey Aitov] | [@SerggAidd] | [s.aitov@innopolis.university] | [Annotator, backend] | [Dataset labelling + establishing backend logic] | +| [Darya Stepanova] | [@darriyano] | [d.stepanova@innopolis.university] | [UX designer] | [Construction and verification of telegram bot scenarious] | +| [Ekaterina Petrova] | [@vougeress] | [e.petrova@innopolis.university] | [Annotator, backend] | [Dataset labelling + establishing backend logic] | +| [Dmitry Tetkin] | [@dimasik057] | [d.tetkin@innopolis.university] | [Frontend] | [Implementing telegram bot to communicate with the user] | + + +## Brainstorming + +### Ideas during brainstorming + +*Put here your ranked ideas with short description* + +### Brief market research / problem validation + +*Put here a brief market research/problem validation for 1–2 ideas* + + +## Basic requirements + +### Target users and their primary needs + +*...* + +### User stories + +*...* + +### Initial scope + +*...* + + +## Tech-stack + +*...* + +## *Something else you want to add* + +*Feel free to add anything else that you consider important to your report* + + +# Weekly commitments + +## Individual contribution of each participant + +*...* + +## Confirmation of the code's operability + +We confirm that the code in the main branch: +- [ ] In working condition. +- [ ] Run via docker-compose (or another alternative described in the `README.md`). From 68ae668c95fd9457f3a75586268a4cd52f954f48 Mon Sep 17 00:00:00 2001 From: Nikita Menshikov <74507368+NikitaMensh@users.noreply.github.com> Date: Mon, 9 Jun 2025 17:46:01 +0300 Subject: [PATCH 03/56] Added project description --- content/docs/2025/Kolobok/week1.md | 23 +++++++++++++---------- 1 file changed, 13 insertions(+), 10 deletions(-) diff --git a/content/docs/2025/Kolobok/week1.md b/content/docs/2025/Kolobok/week1.md index 36b591c29..c70055446 100644 --- a/content/docs/2025/Kolobok/week1.md +++ b/content/docs/2025/Kolobok/week1.md @@ -9,20 +9,23 @@ title: "Week #1" ### Project name: *Kolobok* **Code repository**: [https://github.com/IU-Capstone-Project-2025/Kolobok](https://github.com/IU-Capstone-Project-2025/Kolobok) +Many car service centers offer tire trade-ins, but pricing used tires is slow and error-prone. It typically requires manual measurement of tread depth and a subjective evaluation of spike wear, which is not scalable. -** +This project uses machine learning to automate tire valuation. Users send two photos of a tire to a Telegram bot. The system then detects the brand and parameters, estimates tread depth, and analyzes the condition of spikes. + +The solution is designed for both car owners and retail employees, reducing manual workload and enabling remote consultations. Developed in response to a real-world need from a mid-sized car services. ### **Team Members** -| Team Member | Telegram Alias | Email Address | Track | Responsibilities | -|-----------------------------------------|------------------|------------------------------------|---------------------------------------------|------------------------------------------------------------------------------------------| -| [Nikita Menshikov (Lead)] | [@NikitaMensh] | [n.menshikov@innopolis.university] | [Project Manager] | [Team management, reports writing, customer communication, work environment maintenance] | -| [Nikita Zagainov] | [@V1adych] | [n.zagainov@innopolis.university] | [ML engineer] | [Core models research & development] | -| [Vladislav Strelkov] | [@motrooo] | [v.strelkov@innopolis.university] | [DevOps] | [Product deployment, CI/CD] | -| [Sergey Aitov] | [@SerggAidd] | [s.aitov@innopolis.university] | [Annotator, backend] | [Dataset labelling + establishing backend logic] | -| [Darya Stepanova] | [@darriyano] | [d.stepanova@innopolis.university] | [UX designer] | [Construction and verification of telegram bot scenarious] | -| [Ekaterina Petrova] | [@vougeress] | [e.petrova@innopolis.university] | [Annotator, backend] | [Dataset labelling + establishing backend logic] | -| [Dmitry Tetkin] | [@dimasik057] | [d.tetkin@innopolis.university] | [Frontend] | [Implementing telegram bot to communicate with the user] | +| Team Member | Telegram Alias | Email Address | Track | Responsibilities | +|---------------------------------------|----------------|----------------------------------|-------------------------------------------|----------------------------------------------------------------------------------------| +| Nikita Menshikov (Lead) | @NikitaMensh | n.menshikov@innopolis.university | Project manager | Team management, reports writing, customer communication, work environment maintenance | +| Nikita Zagainov | @V1adych | n.zagainov@innopolis.university | ML | Core models research & development | +| Vladislav Strelkov | @motrooo | v.strelkov@innopolis.university | DevOps | Product deployment, CI/CD | +| Sergey Aitov | @SerggAidd | s.aitov@innopolis.university | Backend, annotator | Dataset labelling + establishing backend logic | +| Darya Stepanova | @darriyano | d.stepanova@innopolis.university | UX designer | Construction and verification of telegram bot scenarious | +| Ekaterina Petrova | @vougeress | e.petrova@innopolis.university | Backend, annotator | Dataset labelling + establishing backend logic | +| Dmitry Tetkin | @dimasik057 | d.tetkin@innopolis.university | Frontend | Implementing telegram bot to communicate with the user | ## Brainstorming From 81f1d70ec0e490936f3a26524fafdc96d430b20e Mon Sep 17 00:00:00 2001 From: Nikita Menshikov <74507368+NikitaMensh@users.noreply.github.com> Date: Mon, 9 Jun 2025 17:46:17 +0300 Subject: [PATCH 04/56] Update week1.md --- content/docs/2025/Kolobok/week1.md | 1 + 1 file changed, 1 insertion(+) diff --git a/content/docs/2025/Kolobok/week1.md b/content/docs/2025/Kolobok/week1.md index c70055446..91c3faf18 100644 --- a/content/docs/2025/Kolobok/week1.md +++ b/content/docs/2025/Kolobok/week1.md @@ -9,6 +9,7 @@ title: "Week #1" ### Project name: *Kolobok* **Code repository**: [https://github.com/IU-Capstone-Project-2025/Kolobok](https://github.com/IU-Capstone-Project-2025/Kolobok) + Many car service centers offer tire trade-ins, but pricing used tires is slow and error-prone. It typically requires manual measurement of tread depth and a subjective evaluation of spike wear, which is not scalable. This project uses machine learning to automate tire valuation. Users send two photos of a tire to a Telegram bot. The system then detects the brand and parameters, estimates tread depth, and analyzes the condition of spikes. From 230f44828dfa8fc22785e6999b3eac204ba563b9 Mon Sep 17 00:00:00 2001 From: Nikita Menshikov <74507368+NikitaMensh@users.noreply.github.com> Date: Tue, 10 Jun 2025 18:37:12 +0300 Subject: [PATCH 05/56] Almost write week1.md --- content/docs/2025/Kolobok/week1.md | 79 ++++++++++++++++++------------ 1 file changed, 49 insertions(+), 30 deletions(-) diff --git a/content/docs/2025/Kolobok/week1.md b/content/docs/2025/Kolobok/week1.md index 91c3faf18..c9e373679 100644 --- a/content/docs/2025/Kolobok/week1.md +++ b/content/docs/2025/Kolobok/week1.md @@ -28,50 +28,69 @@ The solution is designed for both car owners and retail employees, reducing manu | Ekaterina Petrova | @vougeress | e.petrova@innopolis.university | Backend, annotator | Dataset labelling + establishing backend logic | | Dmitry Tetkin | @dimasik057 | d.tetkin@innopolis.university | Frontend | Implementing telegram bot to communicate with the user | - ## Brainstorming - ### Ideas during brainstorming - -*Put here your ranked ideas with short description* - -### Brief market research / problem validation - -*Put here a brief market research/problem validation for 1–2 ideas* - +1. **Tire Valuation System** (Chosen): + ML-powered analysis of tire condition through photos +2. **Virtual Car Detailing Platform**: + App for visualizing car modifications + service pricing +3. **Automated Maintenance Tracker**: + Service scheduling platform + maintenance history + +### Brief market research / problem validation (1st idea) +- No comprehensive solutions combining brand detection, tread measurement and spike analysis +- One brand detection [solution](https://www.griddynamics.com/blog/how-to-identify-vehicle-tires-using-deep-learning-visual-models) using CV was found, however, we have a better idea (need testing) +- 2 car services will be pilot platform for our solution ## Basic requirements - ### Target users and their primary needs - -*...* +1. **Car owners**: + Quickly value tires for sale/trade-in without visiting service center +2. **Retail employees**: + Accelerate tire evaluation process during customer interactions +3. **Service managers**: + Maintain consistent pricing routines across locations ### User stories - -*...* +1. As a car owner, I want to photograph my tires and get instant valuation of thread depth and spikes condition so I can sell them faster +2. As retail staff, I need automated tread measurements and spikes analysis to reduce human error in evaluations +3. As a service manager, I want remote consultation capabilities to scale operations and accelerata processes ### Initial scope - -*...* - +| IN Scope (MVP) | OUT of Scope | +|-----------------------------------------|------------------------------| +| Telegram bot interface | E-commerce integration | +| Tread depth measurement | Full tire marketplace | +| Spike condition analysis | Advanced damage detection | +| Brand/size detection | Multi-platform support | ## Tech-stack - -*...* - -## *Something else you want to add* - -*Feel free to add anything else that you consider important to your report* - +| Component | Technology | Justification | +|-----------------|------------------------------|----------------------------------------------------------------| +| Frontend | Python (python-telegram-bot) | Team member has an experience developing with this library | +| Backend | Python (FastAPI) | Async support, ML integration | +| ML | PyTorch, OpenCV | Industry standard for vision tasks | +| Deployment | Docker | Environment consistency, scalability | +| Annotation | Label Studio | Easy deployment on local server, rich annotation capabilities | + +## Additional notes +- Kanban board created for task tracking +- Initial research on tire measurement algorithms completed +- Team communication channels established (Telegram) # Weekly commitments - ## Individual contribution of each participant - -*...* +| Team Member | Contributions | +|---------------------|-------------------------------------------------------------------------------------------------------------------------------------------------------------| +| Nikita Menshikov | Project setup, report drafting, kanban board creation | +| Nikita Zagainov | ML research and feasibility analysis [[1](https://arxiv.org/pdf/2302.12242)] [[2](https://arxiv.org/pdf/2105.15203)] [[3](https://arxiv.org/pdf/1409.4842)] | +| Vladislav Strelkov | Docker setup | +| Sergey Aitov | 20 spike condition annotations | +| Darya Stepanova | Telegram bot user flow design | +| Ekaterina Petrova | 20 spike condition annotations | +| Dmitry Tetkin | "Hello World" Telegram bot implementation | ## Confirmation of the code's operability - We confirm that the code in the main branch: -- [ ] In working condition. -- [ ] Run via docker-compose (or another alternative described in the `README.md`). +- [x] In working condition +- [x] Run via docker-compose (as described in `README.md`) From 8512d8885db896e58670fb5df0cbb36916c8bba0 Mon Sep 17 00:00:00 2001 From: Nikita Menshikov <74507368+NikitaMensh@users.noreply.github.com> Date: Tue, 10 Jun 2025 18:38:22 +0300 Subject: [PATCH 06/56] Update week1.md --- content/docs/2025/Kolobok/week1.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/content/docs/2025/Kolobok/week1.md b/content/docs/2025/Kolobok/week1.md index c9e373679..02aa136cc 100644 --- a/content/docs/2025/Kolobok/week1.md +++ b/content/docs/2025/Kolobok/week1.md @@ -83,7 +83,7 @@ The solution is designed for both car owners and retail employees, reducing manu | Team Member | Contributions | |---------------------|-------------------------------------------------------------------------------------------------------------------------------------------------------------| | Nikita Menshikov | Project setup, report drafting, kanban board creation | -| Nikita Zagainov | ML research and feasibility analysis [[1](https://arxiv.org/pdf/2302.12242)] [[2](https://arxiv.org/pdf/2105.15203)] [[3](https://arxiv.org/pdf/1409.4842)] | +| Nikita Zagainov | ML research [[1](https://arxiv.org/pdf/2302.12242)] [[2](https://arxiv.org/pdf/2105.15203)] [[3](https://arxiv.org/pdf/1409.4842)] | | Vladislav Strelkov | Docker setup | | Sergey Aitov | 20 spike condition annotations | | Darya Stepanova | Telegram bot user flow design | From 5aa0eee7f3ed999c91e2ed3337c8895a8ed4c6ba Mon Sep 17 00:00:00 2001 From: Nikita Menshikov <74507368+NikitaMensh@users.noreply.github.com> Date: Tue, 10 Jun 2025 18:39:06 +0300 Subject: [PATCH 07/56] Update week1.md --- content/docs/2025/Kolobok/week1.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/content/docs/2025/Kolobok/week1.md b/content/docs/2025/Kolobok/week1.md index 02aa136cc..4f69e8c55 100644 --- a/content/docs/2025/Kolobok/week1.md +++ b/content/docs/2025/Kolobok/week1.md @@ -93,4 +93,4 @@ The solution is designed for both car owners and retail employees, reducing manu ## Confirmation of the code's operability We confirm that the code in the main branch: - [x] In working condition -- [x] Run via docker-compose (as described in `README.md`) +- [x] Run via docker-compose From 047e2b71339fdf4ea4c42da03a7fa551bea7dcf7 Mon Sep 17 00:00:00 2001 From: Nikita Menshikov <74507368+NikitaMensh@users.noreply.github.com> Date: Wed, 11 Jun 2025 14:00:06 +0300 Subject: [PATCH 08/56] Update week1.md --- content/docs/2025/Kolobok/week1.md | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/content/docs/2025/Kolobok/week1.md b/content/docs/2025/Kolobok/week1.md index 4f69e8c55..f0a92992f 100644 --- a/content/docs/2025/Kolobok/week1.md +++ b/content/docs/2025/Kolobok/week1.md @@ -49,12 +49,12 @@ The solution is designed for both car owners and retail employees, reducing manu 2. **Retail employees**: Accelerate tire evaluation process during customer interactions 3. **Service managers**: - Maintain consistent pricing routines across locations + Maintain consistent pricing routines across car service centers ### User stories 1. As a car owner, I want to photograph my tires and get instant valuation of thread depth and spikes condition so I can sell them faster -2. As retail staff, I need automated tread measurements and spikes analysis to reduce human error in evaluations -3. As a service manager, I want remote consultation capabilities to scale operations and accelerata processes +2. As retail staff, I want automated tread measurements and spikes analysis to reduce human error in evaluations +3. As a service manager, I want remote tire valuation to scale operations and accelerate processes ### Initial scope | IN Scope (MVP) | OUT of Scope | @@ -82,7 +82,7 @@ The solution is designed for both car owners and retail employees, reducing manu ## Individual contribution of each participant | Team Member | Contributions | |---------------------|-------------------------------------------------------------------------------------------------------------------------------------------------------------| -| Nikita Menshikov | Project setup, report drafting, kanban board creation | +| Nikita Menshikov | Project setup, report writing, kanban board creation | | Nikita Zagainov | ML research [[1](https://arxiv.org/pdf/2302.12242)] [[2](https://arxiv.org/pdf/2105.15203)] [[3](https://arxiv.org/pdf/1409.4842)] | | Vladislav Strelkov | Docker setup | | Sergey Aitov | 20 spike condition annotations | From e5de2eefb99fc3bc3f2c10f43bf43d076c834af7 Mon Sep 17 00:00:00 2001 From: Nikita Menshikov <74507368+NikitaMensh@users.noreply.github.com> Date: Wed, 11 Jun 2025 14:47:22 +0300 Subject: [PATCH 09/56] Added contributions --- content/docs/2025/Kolobok/week1.md | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/content/docs/2025/Kolobok/week1.md b/content/docs/2025/Kolobok/week1.md index f0a92992f..e48c9a456 100644 --- a/content/docs/2025/Kolobok/week1.md +++ b/content/docs/2025/Kolobok/week1.md @@ -82,12 +82,12 @@ The solution is designed for both car owners and retail employees, reducing manu ## Individual contribution of each participant | Team Member | Contributions | |---------------------|-------------------------------------------------------------------------------------------------------------------------------------------------------------| -| Nikita Menshikov | Project setup, report writing, kanban board creation | -| Nikita Zagainov | ML research [[1](https://arxiv.org/pdf/2302.12242)] [[2](https://arxiv.org/pdf/2105.15203)] [[3](https://arxiv.org/pdf/1409.4842)] | +| Nikita Menshikov | Project setup, report writing, [kanban board](https://github.com/orgs/IU-Capstone-Project-2025/projects/9) creation | +| Nikita Zagainov | ML [[research](https://github.com/IU-Capstone-Project-2025/Kolobok/commit/4a60648dd629b54b227625ab7d07d34c2b51bd86)] | | Vladislav Strelkov | Docker setup | -| Sergey Aitov | 20 spike condition annotations | -| Darya Stepanova | Telegram bot user flow design | -| Ekaterina Petrova | 20 spike condition annotations | +| Sergey Aitov | 20 spike condition [annotations](https://github.com/NikitaMensh/IU_Capstone_project_2025/blob/main/sergey_annotations_w1.png) [[sample](https://github.com/NikitaMensh/IU_Capstone_project_2025/blob/main/sergey_annotation_sample_w1.png)] | +| Darya Stepanova | Telegram bot user flow [[design](https://github.com/IU-Capstone-Project-2025/Kolobok/commit/038c5b913fdf6d97ffc62945ec45a03931973142)] | +| Ekaterina Petrova | 20 spike condition [annotations](https://github.com/NikitaMensh/IU_Capstone_project_2025/blob/main/kate_annotations_w1.png) [[sample](https://github.com/NikitaMensh/IU_Capstone_project_2025/blob/main/kate_annotation_sample_w1.png)] | | Dmitry Tetkin | "Hello World" Telegram bot implementation | ## Confirmation of the code's operability From 118ae84aae92642fbb16617b8edf638828974c53 Mon Sep 17 00:00:00 2001 From: Nikita Menshikov <74507368+NikitaMensh@users.noreply.github.com> Date: Wed, 11 Jun 2025 14:48:24 +0300 Subject: [PATCH 10/56] Update week1.md --- content/docs/2025/Kolobok/week1.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/content/docs/2025/Kolobok/week1.md b/content/docs/2025/Kolobok/week1.md index e48c9a456..b4a3f038d 100644 --- a/content/docs/2025/Kolobok/week1.md +++ b/content/docs/2025/Kolobok/week1.md @@ -83,10 +83,10 @@ The solution is designed for both car owners and retail employees, reducing manu | Team Member | Contributions | |---------------------|-------------------------------------------------------------------------------------------------------------------------------------------------------------| | Nikita Menshikov | Project setup, report writing, [kanban board](https://github.com/orgs/IU-Capstone-Project-2025/projects/9) creation | -| Nikita Zagainov | ML [[research](https://github.com/IU-Capstone-Project-2025/Kolobok/commit/4a60648dd629b54b227625ab7d07d34c2b51bd86)] | +| Nikita Zagainov | ML [research](https://github.com/IU-Capstone-Project-2025/Kolobok/commit/4a60648dd629b54b227625ab7d07d34c2b51bd86) | | Vladislav Strelkov | Docker setup | | Sergey Aitov | 20 spike condition [annotations](https://github.com/NikitaMensh/IU_Capstone_project_2025/blob/main/sergey_annotations_w1.png) [[sample](https://github.com/NikitaMensh/IU_Capstone_project_2025/blob/main/sergey_annotation_sample_w1.png)] | -| Darya Stepanova | Telegram bot user flow [[design](https://github.com/IU-Capstone-Project-2025/Kolobok/commit/038c5b913fdf6d97ffc62945ec45a03931973142)] | +| Darya Stepanova | Telegram bot user flow [design](https://github.com/IU-Capstone-Project-2025/Kolobok/commit/038c5b913fdf6d97ffc62945ec45a03931973142) | | Ekaterina Petrova | 20 spike condition [annotations](https://github.com/NikitaMensh/IU_Capstone_project_2025/blob/main/kate_annotations_w1.png) [[sample](https://github.com/NikitaMensh/IU_Capstone_project_2025/blob/main/kate_annotation_sample_w1.png)] | | Dmitry Tetkin | "Hello World" Telegram bot implementation | From 72e8a8ac36a7cc6c0573fe36f3b0e4679d9958b6 Mon Sep 17 00:00:00 2001 From: Nikita Menshikov <74507368+NikitaMensh@users.noreply.github.com> Date: Wed, 11 Jun 2025 14:59:28 +0300 Subject: [PATCH 11/56] Added frontend contribution --- content/docs/2025/Kolobok/week1.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/content/docs/2025/Kolobok/week1.md b/content/docs/2025/Kolobok/week1.md index b4a3f038d..dc7da155f 100644 --- a/content/docs/2025/Kolobok/week1.md +++ b/content/docs/2025/Kolobok/week1.md @@ -88,7 +88,7 @@ The solution is designed for both car owners and retail employees, reducing manu | Sergey Aitov | 20 spike condition [annotations](https://github.com/NikitaMensh/IU_Capstone_project_2025/blob/main/sergey_annotations_w1.png) [[sample](https://github.com/NikitaMensh/IU_Capstone_project_2025/blob/main/sergey_annotation_sample_w1.png)] | | Darya Stepanova | Telegram bot user flow [design](https://github.com/IU-Capstone-Project-2025/Kolobok/commit/038c5b913fdf6d97ffc62945ec45a03931973142) | | Ekaterina Petrova | 20 spike condition [annotations](https://github.com/NikitaMensh/IU_Capstone_project_2025/blob/main/kate_annotations_w1.png) [[sample](https://github.com/NikitaMensh/IU_Capstone_project_2025/blob/main/kate_annotation_sample_w1.png)] | -| Dmitry Tetkin | "Hello World" Telegram bot implementation | +| Dmitry Tetkin | "Hello World" Telegram bot [implementation](https://github.com/IU-Capstone-Project-2025/Kolobok/commit/f2784c39f27eb312e1c638c4c0465555951ac97c) | ## Confirmation of the code's operability We confirm that the code in the main branch: From 48f0e8a879e3457f94ce736864c5f49b242335f6 Mon Sep 17 00:00:00 2001 From: Nikita Menshikov <74507368+NikitaMensh@users.noreply.github.com> Date: Wed, 11 Jun 2025 16:05:17 +0300 Subject: [PATCH 12/56] Added contributions --- content/docs/2025/Kolobok/week1.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/content/docs/2025/Kolobok/week1.md b/content/docs/2025/Kolobok/week1.md index dc7da155f..09798bb24 100644 --- a/content/docs/2025/Kolobok/week1.md +++ b/content/docs/2025/Kolobok/week1.md @@ -84,7 +84,7 @@ The solution is designed for both car owners and retail employees, reducing manu |---------------------|-------------------------------------------------------------------------------------------------------------------------------------------------------------| | Nikita Menshikov | Project setup, report writing, [kanban board](https://github.com/orgs/IU-Capstone-Project-2025/projects/9) creation | | Nikita Zagainov | ML [research](https://github.com/IU-Capstone-Project-2025/Kolobok/commit/4a60648dd629b54b227625ab7d07d34c2b51bd86) | -| Vladislav Strelkov | Docker setup | +| Vladislav Strelkov | Docker setup [[1](https://github.com/IU-Capstone-Project-2025/Kolobok/blob/main/docker-compose.yml)], [[2](https://github.com/IU-Capstone-Project-2025/Kolobok/blob/main/tg_bot/.env)], [[3](https://github.com/IU-Capstone-Project-2025/Kolobok/blob/main/tg_bot/Dockerfile)] | | Sergey Aitov | 20 spike condition [annotations](https://github.com/NikitaMensh/IU_Capstone_project_2025/blob/main/sergey_annotations_w1.png) [[sample](https://github.com/NikitaMensh/IU_Capstone_project_2025/blob/main/sergey_annotation_sample_w1.png)] | | Darya Stepanova | Telegram bot user flow [design](https://github.com/IU-Capstone-Project-2025/Kolobok/commit/038c5b913fdf6d97ffc62945ec45a03931973142) | | Ekaterina Petrova | 20 spike condition [annotations](https://github.com/NikitaMensh/IU_Capstone_project_2025/blob/main/kate_annotations_w1.png) [[sample](https://github.com/NikitaMensh/IU_Capstone_project_2025/blob/main/kate_annotation_sample_w1.png)] | From 62dad1861106c588b596be9dacc8b3480be51ed7 Mon Sep 17 00:00:00 2001 From: Nikita Menshikov <74507368+NikitaMensh@users.noreply.github.com> Date: Wed, 11 Jun 2025 16:23:07 +0300 Subject: [PATCH 13/56] Make the links more visible --- content/docs/2025/Kolobok/week1.md | 12 ++++++------ 1 file changed, 6 insertions(+), 6 deletions(-) diff --git a/content/docs/2025/Kolobok/week1.md b/content/docs/2025/Kolobok/week1.md index 09798bb24..5866e8e84 100644 --- a/content/docs/2025/Kolobok/week1.md +++ b/content/docs/2025/Kolobok/week1.md @@ -82,13 +82,13 @@ The solution is designed for both car owners and retail employees, reducing manu ## Individual contribution of each participant | Team Member | Contributions | |---------------------|-------------------------------------------------------------------------------------------------------------------------------------------------------------| -| Nikita Menshikov | Project setup, report writing, [kanban board](https://github.com/orgs/IU-Capstone-Project-2025/projects/9) creation | -| Nikita Zagainov | ML [research](https://github.com/IU-Capstone-Project-2025/Kolobok/commit/4a60648dd629b54b227625ab7d07d34c2b51bd86) | +| Nikita Menshikov | Project setup, report writing, [[kanban board](https://github.com/orgs/IU-Capstone-Project-2025/projects/9)] creation | +| Nikita Zagainov | ML [[research](https://github.com/IU-Capstone-Project-2025/Kolobok/commit/4a60648dd629b54b227625ab7d07d34c2b51bd86)] | | Vladislav Strelkov | Docker setup [[1](https://github.com/IU-Capstone-Project-2025/Kolobok/blob/main/docker-compose.yml)], [[2](https://github.com/IU-Capstone-Project-2025/Kolobok/blob/main/tg_bot/.env)], [[3](https://github.com/IU-Capstone-Project-2025/Kolobok/blob/main/tg_bot/Dockerfile)] | -| Sergey Aitov | 20 spike condition [annotations](https://github.com/NikitaMensh/IU_Capstone_project_2025/blob/main/sergey_annotations_w1.png) [[sample](https://github.com/NikitaMensh/IU_Capstone_project_2025/blob/main/sergey_annotation_sample_w1.png)] | -| Darya Stepanova | Telegram bot user flow [design](https://github.com/IU-Capstone-Project-2025/Kolobok/commit/038c5b913fdf6d97ffc62945ec45a03931973142) | -| Ekaterina Petrova | 20 spike condition [annotations](https://github.com/NikitaMensh/IU_Capstone_project_2025/blob/main/kate_annotations_w1.png) [[sample](https://github.com/NikitaMensh/IU_Capstone_project_2025/blob/main/kate_annotation_sample_w1.png)] | -| Dmitry Tetkin | "Hello World" Telegram bot [implementation](https://github.com/IU-Capstone-Project-2025/Kolobok/commit/f2784c39f27eb312e1c638c4c0465555951ac97c) | +| Sergey Aitov | 20 spike condition [[annotations](https://github.com/NikitaMensh/IU_Capstone_project_2025/blob/main/sergey_annotations_w1.png)] [[sample](https://github.com/NikitaMensh/IU_Capstone_project_2025/blob/main/sergey_annotation_sample_w1.png)] | +| Darya Stepanova | Telegram bot user flow [[design](https://github.com/IU-Capstone-Project-2025/Kolobok/commit/038c5b913fdf6d97ffc62945ec45a03931973142)] | +| Ekaterina Petrova | 20 spike condition [[annotations](https://github.com/NikitaMensh/IU_Capstone_project_2025/blob/main/kate_annotations_w1.png)] [[sample](https://github.com/NikitaMensh/IU_Capstone_project_2025/blob/main/kate_annotation_sample_w1.png)] | +| Dmitry Tetkin | "Hello World" Telegram bot [[implementation](https://github.com/IU-Capstone-Project-2025/Kolobok/commit/f2784c39f27eb312e1c638c4c0465555951ac97c)] | ## Confirmation of the code's operability We confirm that the code in the main branch: From a7fb068514d380387d053cadc4fbe099681d98f6 Mon Sep 17 00:00:00 2001 From: Nikita Menshikov <74507368+NikitaMensh@users.noreply.github.com> Date: Wed, 11 Jun 2025 16:23:58 +0300 Subject: [PATCH 14/56] Update week1.md --- content/docs/2025/Kolobok/week1.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/content/docs/2025/Kolobok/week1.md b/content/docs/2025/Kolobok/week1.md index 5866e8e84..5e1502607 100644 --- a/content/docs/2025/Kolobok/week1.md +++ b/content/docs/2025/Kolobok/week1.md @@ -84,10 +84,10 @@ The solution is designed for both car owners and retail employees, reducing manu |---------------------|-------------------------------------------------------------------------------------------------------------------------------------------------------------| | Nikita Menshikov | Project setup, report writing, [[kanban board](https://github.com/orgs/IU-Capstone-Project-2025/projects/9)] creation | | Nikita Zagainov | ML [[research](https://github.com/IU-Capstone-Project-2025/Kolobok/commit/4a60648dd629b54b227625ab7d07d34c2b51bd86)] | -| Vladislav Strelkov | Docker setup [[1](https://github.com/IU-Capstone-Project-2025/Kolobok/blob/main/docker-compose.yml)], [[2](https://github.com/IU-Capstone-Project-2025/Kolobok/blob/main/tg_bot/.env)], [[3](https://github.com/IU-Capstone-Project-2025/Kolobok/blob/main/tg_bot/Dockerfile)] | -| Sergey Aitov | 20 spike condition [[annotations](https://github.com/NikitaMensh/IU_Capstone_project_2025/blob/main/sergey_annotations_w1.png)] [[sample](https://github.com/NikitaMensh/IU_Capstone_project_2025/blob/main/sergey_annotation_sample_w1.png)] | +| Vladislav Strelkov | Docker setup ([[1](https://github.com/IU-Capstone-Project-2025/Kolobok/blob/main/docker-compose.yml)], [[2](https://github.com/IU-Capstone-Project-2025/Kolobok/blob/main/tg_bot/.env)], [[3](https://github.com/IU-Capstone-Project-2025/Kolobok/blob/main/tg_bot/Dockerfile)]) | +| Sergey Aitov | 20 spike condition [[annotations](https://github.com/NikitaMensh/IU_Capstone_project_2025/blob/main/sergey_annotations_w1.png)] ([[sample](https://github.com/NikitaMensh/IU_Capstone_project_2025/blob/main/sergey_annotation_sample_w1.png)]) | | Darya Stepanova | Telegram bot user flow [[design](https://github.com/IU-Capstone-Project-2025/Kolobok/commit/038c5b913fdf6d97ffc62945ec45a03931973142)] | -| Ekaterina Petrova | 20 spike condition [[annotations](https://github.com/NikitaMensh/IU_Capstone_project_2025/blob/main/kate_annotations_w1.png)] [[sample](https://github.com/NikitaMensh/IU_Capstone_project_2025/blob/main/kate_annotation_sample_w1.png)] | +| Ekaterina Petrova | 20 spike condition [[annotations](https://github.com/NikitaMensh/IU_Capstone_project_2025/blob/main/kate_annotations_w1.png)] ([[sample](https://github.com/NikitaMensh/IU_Capstone_project_2025/blob/main/kate_annotation_sample_w1.png)]) | | Dmitry Tetkin | "Hello World" Telegram bot [[implementation](https://github.com/IU-Capstone-Project-2025/Kolobok/commit/f2784c39f27eb312e1c638c4c0465555951ac97c)] | ## Confirmation of the code's operability From 21a18b9b4565c94749c3c3a48c1225e6f805bbf3 Mon Sep 17 00:00:00 2001 From: Nikita Menshikov <74507368+NikitaMensh@users.noreply.github.com> Date: Sat, 14 Jun 2025 14:46:10 +0300 Subject: [PATCH 15/56] Update week1.md --- content/docs/2025/Kolobok/week1.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/content/docs/2025/Kolobok/week1.md b/content/docs/2025/Kolobok/week1.md index 5e1502607..bb5c81677 100644 --- a/content/docs/2025/Kolobok/week1.md +++ b/content/docs/2025/Kolobok/week1.md @@ -93,4 +93,4 @@ The solution is designed for both car owners and retail employees, reducing manu ## Confirmation of the code's operability We confirm that the code in the main branch: - [x] In working condition -- [x] Run via docker-compose +- [x] Run via method described in README.md From 7bb8fe401f2a07397142fedd09cf59db940cdcc8 Mon Sep 17 00:00:00 2001 From: Nikita Menshikov Date: Wed, 18 Jun 2025 19:16:07 +0300 Subject: [PATCH 16/56] week2 report --- content/docs/2025/Kolobok/week2.md | 182 +++++++++++++++++++++++++++++ 1 file changed, 182 insertions(+) create mode 100644 content/docs/2025/Kolobok/week2.md diff --git a/content/docs/2025/Kolobok/week2.md b/content/docs/2025/Kolobok/week2.md new file mode 100644 index 000000000..370e6d805 --- /dev/null +++ b/content/docs/2025/Kolobok/week2.md @@ -0,0 +1,182 @@ +--- +title: "Week 2" +--- + +# Week 2 Progress Report + +## Executive Summary + +This week marked significant progress across all project components. We successfully established the foundational architecture for our tire analysis system, with substantial advances in machine learning model development, backend API implementation, and user interface design. The team has demonstrated strong collaboration and technical execution, setting a solid foundation for the upcoming development phases. + +## Requirements Analysis + +### User Stories and Acceptance Criteria + +Our project addresses three core user needs through carefully defined acceptance criteria: + +#### 1. Tire Brand and Model Detection +**User Story:** As a user, I want to upload a photo of my tire so that the system identifies its brand and key specifications. + +**Acceptance Criteria:** +- System outputs brand name and size specification (e.g., "215/65 R16") +- Achieves accuracy above 90% for clear, well-lit photographs +- Gracefully handles unrecognized brands with appropriate user feedback + +#### 2. Tread Depth Measurement +**User Story:** As a user, I want to receive accurate tread depth measurements from my tire photos to assess wear condition. + +**Acceptance Criteria:** +- Mean Absolute Error (MAE) must not exceed 1mm +- Results displayed with 2 decimal place precision +- System provides warnings for poor image quality + +#### 3. Spike Condition Assessment +**User Story:** As a user, I want to know the condition of my tire spikes to understand wear patterns. + +**Acceptance Criteria:** +- Combined false positive and false negative rate below 10 +- Detects various spike conditions: worn, missing, loose, and re-nailed +- Provides clear count of good versus damaged spikes + +### Performance Benchmarks + +| Feature | Output Format | Performance Target | +|---------|---------------|-------------------| +| Tread Depth Estimation | Numeric value (mm), 2 decimal places | MAE ≤ 1mm | +| Spike Analysis | Count of good/bad spikes | FP + FN ≤ 10 | +| Brand Detection | Brand name and size string | ≥90% accuracy | + +## Technical Implementation Progress + +### Backend Development + +The FastAPI backend has been successfully scaffolded with a well-structured architecture: + +- **API Endpoints:** Two primary endpoints implemented (`/analyze/tread`, `/analyze/brand`) +- **Containerization:** Complete Docker setup with proper service orchestration +- **Security:** Bearer token authentication implemented across all endpoints +- **Integration Ready:** Backend prepared for ML model integration and Telegram bot connectivity + +The API follows RESTful principles with proper error handling for invalid tokens (401) and malformed images (400). + +### Frontend Development (Telegram Bot) + +The Telegram bot implementation has progressed beyond initial prototypes: + +- **User Flow:** Complete conversation handler with state management +- **User Experience:** Intuitive menu system with fallback mechanisms +- **Integration Status:** Ready for backend API connection + +The bot supports two main workflows: tire brand identification and tread depth analysis, with proper error handling and user feedback loops. + +### Machine Learning Pipeline + +#### Tread Depth Estimation Model +We've implemented a comprehensive regression model using CNN architecture: + +- **Architecture:** Multiple backbone options including Swin Transformer, EfficientNet-B7, Vision Transformer, DenseNet-201, and ConvNeXt +- **Training:** Custom PyTorch Dataset with proper data loading and augmentation +- **Metrics:** MSE loss function with MAE tracking for performance evaluation +- **Status:** Initial training completed with promising results + +#### Spike Quality Detection +Binary classification model for spike condition assessment: + +- **Model Type:** CNN-based classifier for good vs. bad spike detection +- **Loss Function:** CrossEntropy with class-wise accuracy metrics +- **Validation:** Comprehensive testing workflow implemented + +#### Brand Recognition System +OCR-based approach for tire sidewall text extraction: + +- **Preprocessing:** Image enhancement pipeline for sidewall flattening +- **Status:** Image processing methods validated, model integration pending + +### Dataset Development + +Our dataset has grown significantly through systematic annotation: + +- **Size:** Approximately 700 annotated images, 400 with spikes analysis +- **Annotation Quality:** Manual inspection for tread depth and spike condition +- **Diversity:** Real-world conditions with variable lighting and angles +- **Structure:** Organized folder structure with CSV metadata + +## User Experience Design + +### Conversation Flow + +The user journey begins with a welcoming interface that presents two primary options: + +**Path 1: Tire Brand and Model Detection** +1. User selects "Tire Brand and Model" option +2. System prompts for sidewall photo upload +3. Image validation checks (size, format, quality) +4. Processing with user feedback +5. Results display with confirmation options +6. Optional manual correction with feedback collection + +**Path 2: Tread Depth Analysis** +1. User selects "Tread Depth" option +2. System requests tread photo with visibility guidelines +3. Image validation and processing +4. Results showing depth measurement and spike count +5. Confirmation or manual input options +6. Feedback collection for model improvement + +### Technical Implementation Details + +#### Telegram Bot Architecture +- **Framework:** python-telegram-bot with ConversationHandler +- **States:** MENU, SIDE_PHOTO, SIDE_RESULT, SIDE_CUSTOM, TREAD_PHOTO, TREAD_RESULT, TREAD_CUSTOM +- **Commands:** /start for main menu, /cancel for conversation reset +- **Features:** Inline keyboards, image processing, text input handling + +#### API Specification +**Authentication:** Bearer token required for all endpoints +**Request Format:** JSON with base64-encoded image +**Endpoints:** +- `/api/v1/analyze_thread` - Tread depth and spike analysis +- `/api/v1/identify_tire` - Brand and model detection + +## Machine Learning Development + +### Training Infrastructure + +Our training notebook provides a complete ML pipeline for tread depth estimation: + +**Data Management:** +- CSV-based metadata with image paths and labels +- Custom ThreadDataset class for PyTorch integration +- 80/20 train-validation split with proper randomization + +**Model Architecture:** +- Multiple pre-trained backbone options +- Regression head for depth estimation +- Configurable hyperparameters and optimization + +**Training Process:** +- Mini-batch processing with DataLoader +- Real-time progress tracking +- Comprehensive logging and visualization +- Model checkpointing and evaluation + +**Evaluation Framework:** +- MAE-based performance assessment +- Prediction vs. ground truth analysis +- Model persistence for deployment + +## Team Contributions + +| Team Member | Primary Contributions | Deliverables | +|-------------|----------------------|--------------| +| Nikita Menshikov | Project documentation and user flow design | Comprehensive report and process diagrams | +| Nikita Zagainov | Machine learning model development | Training notebook and model implementations [[notebook](https://github.com/.../train.ipynb)] | +| Vladislav Strelkov | Infrastructure and deployment | Docker environment and service orchestration, tire annotation tool [[annotation tool](https://github.com/IU-Capstone-Project-2025/Kolobok/commit/ec520dcdb5376281db28b381b7304d03cb107230)] [[Docker setup](https://github.com/IU-Capstone-Project-2025/Kolobok/commit/c894595c8928c915249d5a11dc25195ba1534777)] [[docker-compose](https://github.com/IU-Capstone-Project-2025/Kolobok/commit/d34838a54e748eed72ff9adee51dd193c6f2c974)] | +| Sergey Aitov | Data annotation and API development | Labeled dataset and backend scaffolding [[labeled tires](https://disk.yandex.ru/d/WTfWlQqAaTR-ZA)] | +| Darya Stepanova | User experience design | Bot flow refinement and interface design [[figma](https://www.figma.com/design/llkM7QUkBNkwm0pit8qUhn/KolobokBoard?node-id=0-1&t=S58PeihoZSgsA2XF-1)] | +| Ekaterina Petrova | Dataset preparation and API support | Annotated images and backend assistance [[tires dataset](https://disk.yandex.ru/d/WTfWlQqAaTR-ZA)] | +| Dmitry Tetkin | Bot interaction logic | Telegram bot conversation handling | + +We confirm that all code in the main branch: +- [x] Compiles and runs without errors +- [x] Deploys successfully via docker-compose (via instructions in README.md) \ No newline at end of file From 144a39be8802cc7e0da5a4310d85a68ba5159436 Mon Sep 17 00:00:00 2001 From: Nikita Menshikov <74507368+NikitaMensh@users.noreply.github.com> Date: Wed, 18 Jun 2025 19:27:12 +0300 Subject: [PATCH 17/56] Update contributions --- content/docs/2025/Kolobok/week2.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/content/docs/2025/Kolobok/week2.md b/content/docs/2025/Kolobok/week2.md index 370e6d805..3cdee57c6 100644 --- a/content/docs/2025/Kolobok/week2.md +++ b/content/docs/2025/Kolobok/week2.md @@ -170,13 +170,13 @@ Our training notebook provides a complete ML pipeline for tread depth estimation | Team Member | Primary Contributions | Deliverables | |-------------|----------------------|--------------| | Nikita Menshikov | Project documentation and user flow design | Comprehensive report and process diagrams | -| Nikita Zagainov | Machine learning model development | Training notebook and model implementations [[notebook](https://github.com/.../train.ipynb)] | +| Nikita Zagainov | Machine learning model development | Training notebook and model implementations [[notebook](https://github.com/IU-Capstone-Project-2025/Kolobok/blob/main/ML/notebooks/train.ipynb)] | | Vladislav Strelkov | Infrastructure and deployment | Docker environment and service orchestration, tire annotation tool [[annotation tool](https://github.com/IU-Capstone-Project-2025/Kolobok/commit/ec520dcdb5376281db28b381b7304d03cb107230)] [[Docker setup](https://github.com/IU-Capstone-Project-2025/Kolobok/commit/c894595c8928c915249d5a11dc25195ba1534777)] [[docker-compose](https://github.com/IU-Capstone-Project-2025/Kolobok/commit/d34838a54e748eed72ff9adee51dd193c6f2c974)] | -| Sergey Aitov | Data annotation and API development | Labeled dataset and backend scaffolding [[labeled tires](https://disk.yandex.ru/d/WTfWlQqAaTR-ZA)] | +| Sergey Aitov | Data annotation and API development | Labeled dataset and backend scaffolding [[dummy endpoints](https://github.com/IU-Capstone-Project-2025/Kolobok/blob/main/backend/app.py)] [[tires dataset](https://disk.yandex.ru/d/WTfWlQqAaTR-ZA)] | | Darya Stepanova | User experience design | Bot flow refinement and interface design [[figma](https://www.figma.com/design/llkM7QUkBNkwm0pit8qUhn/KolobokBoard?node-id=0-1&t=S58PeihoZSgsA2XF-1)] | | Ekaterina Petrova | Dataset preparation and API support | Annotated images and backend assistance [[tires dataset](https://disk.yandex.ru/d/WTfWlQqAaTR-ZA)] | | Dmitry Tetkin | Bot interaction logic | Telegram bot conversation handling | We confirm that all code in the main branch: - [x] Compiles and runs without errors -- [x] Deploys successfully via docker-compose (via instructions in README.md) \ No newline at end of file +- [x] Deploys successfully via docker-compose (via instructions in README.md) From e4199085ae48a6225e73fc205229c6fb3da3bf67 Mon Sep 17 00:00:00 2001 From: Nikita Menshikov <74507368+NikitaMensh@users.noreply.github.com> Date: Wed, 18 Jun 2025 19:42:54 +0300 Subject: [PATCH 18/56] Update week2.md --- content/docs/2025/Kolobok/week2.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/content/docs/2025/Kolobok/week2.md b/content/docs/2025/Kolobok/week2.md index 3cdee57c6..1dc100c09 100644 --- a/content/docs/2025/Kolobok/week2.md +++ b/content/docs/2025/Kolobok/week2.md @@ -169,13 +169,13 @@ Our training notebook provides a complete ML pipeline for tread depth estimation | Team Member | Primary Contributions | Deliverables | |-------------|----------------------|--------------| -| Nikita Menshikov | Project documentation and user flow design | Comprehensive report and process diagrams | +| Nikita Menshikov | Project documentation and user flow design | Comprehensive report and [backlog](https://github.com/orgs/IU-Capstone-Project-2025/projects/9/views/1), setup labelling environment | | Nikita Zagainov | Machine learning model development | Training notebook and model implementations [[notebook](https://github.com/IU-Capstone-Project-2025/Kolobok/blob/main/ML/notebooks/train.ipynb)] | | Vladislav Strelkov | Infrastructure and deployment | Docker environment and service orchestration, tire annotation tool [[annotation tool](https://github.com/IU-Capstone-Project-2025/Kolobok/commit/ec520dcdb5376281db28b381b7304d03cb107230)] [[Docker setup](https://github.com/IU-Capstone-Project-2025/Kolobok/commit/c894595c8928c915249d5a11dc25195ba1534777)] [[docker-compose](https://github.com/IU-Capstone-Project-2025/Kolobok/commit/d34838a54e748eed72ff9adee51dd193c6f2c974)] | | Sergey Aitov | Data annotation and API development | Labeled dataset and backend scaffolding [[dummy endpoints](https://github.com/IU-Capstone-Project-2025/Kolobok/blob/main/backend/app.py)] [[tires dataset](https://disk.yandex.ru/d/WTfWlQqAaTR-ZA)] | | Darya Stepanova | User experience design | Bot flow refinement and interface design [[figma](https://www.figma.com/design/llkM7QUkBNkwm0pit8qUhn/KolobokBoard?node-id=0-1&t=S58PeihoZSgsA2XF-1)] | | Ekaterina Petrova | Dataset preparation and API support | Annotated images and backend assistance [[tires dataset](https://disk.yandex.ru/d/WTfWlQqAaTR-ZA)] | -| Dmitry Tetkin | Bot interaction logic | Telegram bot conversation handling | +| Dmitry Tetkin | Bot interaction logic | Telegram bot prototype | We confirm that all code in the main branch: - [x] Compiles and runs without errors From cc4541244413cfd2064ca8ea8cbee8415c766c46 Mon Sep 17 00:00:00 2001 From: Nikita Menshikov <74507368+NikitaMensh@users.noreply.github.com> Date: Wed, 18 Jun 2025 19:49:02 +0300 Subject: [PATCH 19/56] Update contributions --- content/docs/2025/Kolobok/week2.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/content/docs/2025/Kolobok/week2.md b/content/docs/2025/Kolobok/week2.md index 1dc100c09..f0c7d574e 100644 --- a/content/docs/2025/Kolobok/week2.md +++ b/content/docs/2025/Kolobok/week2.md @@ -169,7 +169,7 @@ Our training notebook provides a complete ML pipeline for tread depth estimation | Team Member | Primary Contributions | Deliverables | |-------------|----------------------|--------------| -| Nikita Menshikov | Project documentation and user flow design | Comprehensive report and [backlog](https://github.com/orgs/IU-Capstone-Project-2025/projects/9/views/1), setup labelling environment | +| Nikita Menshikov | Project documentation and user flow design | Comprehensive report and [backlog](https://github.com/orgs/IU-Capstone-Project-2025/projects/9/views/1), setup [labelling](https://github.com/NikitaMensh/IU_Capstone_project_2025/blob/main/photo_2025-06-17_10-52-36.jpg) [environment](https://github.com/NikitaMensh/IU_Capstone_project_2025/blob/main/photo_2025-06-17_10-52-37.jpg) | | Nikita Zagainov | Machine learning model development | Training notebook and model implementations [[notebook](https://github.com/IU-Capstone-Project-2025/Kolobok/blob/main/ML/notebooks/train.ipynb)] | | Vladislav Strelkov | Infrastructure and deployment | Docker environment and service orchestration, tire annotation tool [[annotation tool](https://github.com/IU-Capstone-Project-2025/Kolobok/commit/ec520dcdb5376281db28b381b7304d03cb107230)] [[Docker setup](https://github.com/IU-Capstone-Project-2025/Kolobok/commit/c894595c8928c915249d5a11dc25195ba1534777)] [[docker-compose](https://github.com/IU-Capstone-Project-2025/Kolobok/commit/d34838a54e748eed72ff9adee51dd193c6f2c974)] | | Sergey Aitov | Data annotation and API development | Labeled dataset and backend scaffolding [[dummy endpoints](https://github.com/IU-Capstone-Project-2025/Kolobok/blob/main/backend/app.py)] [[tires dataset](https://disk.yandex.ru/d/WTfWlQqAaTR-ZA)] | From 14fe217c643c6bda392391766b134de8c285b97e Mon Sep 17 00:00:00 2001 From: Nikita Menshikov <74507368+NikitaMensh@users.noreply.github.com> Date: Wed, 18 Jun 2025 19:50:05 +0300 Subject: [PATCH 20/56] Update week2.md --- content/docs/2025/Kolobok/week2.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/content/docs/2025/Kolobok/week2.md b/content/docs/2025/Kolobok/week2.md index f0c7d574e..77d9de0e5 100644 --- a/content/docs/2025/Kolobok/week2.md +++ b/content/docs/2025/Kolobok/week2.md @@ -172,9 +172,9 @@ Our training notebook provides a complete ML pipeline for tread depth estimation | Nikita Menshikov | Project documentation and user flow design | Comprehensive report and [backlog](https://github.com/orgs/IU-Capstone-Project-2025/projects/9/views/1), setup [labelling](https://github.com/NikitaMensh/IU_Capstone_project_2025/blob/main/photo_2025-06-17_10-52-36.jpg) [environment](https://github.com/NikitaMensh/IU_Capstone_project_2025/blob/main/photo_2025-06-17_10-52-37.jpg) | | Nikita Zagainov | Machine learning model development | Training notebook and model implementations [[notebook](https://github.com/IU-Capstone-Project-2025/Kolobok/blob/main/ML/notebooks/train.ipynb)] | | Vladislav Strelkov | Infrastructure and deployment | Docker environment and service orchestration, tire annotation tool [[annotation tool](https://github.com/IU-Capstone-Project-2025/Kolobok/commit/ec520dcdb5376281db28b381b7304d03cb107230)] [[Docker setup](https://github.com/IU-Capstone-Project-2025/Kolobok/commit/c894595c8928c915249d5a11dc25195ba1534777)] [[docker-compose](https://github.com/IU-Capstone-Project-2025/Kolobok/commit/d34838a54e748eed72ff9adee51dd193c6f2c974)] | -| Sergey Aitov | Data annotation and API development | Labeled dataset and backend scaffolding [[dummy endpoints](https://github.com/IU-Capstone-Project-2025/Kolobok/blob/main/backend/app.py)] [[tires dataset](https://disk.yandex.ru/d/WTfWlQqAaTR-ZA)] | +| Sergey Aitov | Data annotation and API development | Labeled dataset (350 images collected) and backend scaffolding [[dummy endpoints](https://github.com/IU-Capstone-Project-2025/Kolobok/blob/main/backend/app.py)] [[tires dataset](https://disk.yandex.ru/d/WTfWlQqAaTR-ZA)] | | Darya Stepanova | User experience design | Bot flow refinement and interface design [[figma](https://www.figma.com/design/llkM7QUkBNkwm0pit8qUhn/KolobokBoard?node-id=0-1&t=S58PeihoZSgsA2XF-1)] | -| Ekaterina Petrova | Dataset preparation and API support | Annotated images and backend assistance [[tires dataset](https://disk.yandex.ru/d/WTfWlQqAaTR-ZA)] | +| Ekaterina Petrova | Dataset preparation and API support | Annotated images (350 collected) and backend assistance [[tires dataset](https://disk.yandex.ru/d/WTfWlQqAaTR-ZA)] | | Dmitry Tetkin | Bot interaction logic | Telegram bot prototype | We confirm that all code in the main branch: From 2b4b1cb5d87f125e04e0d5eed157f48290e4bb26 Mon Sep 17 00:00:00 2001 From: Nikita Menshikov <74507368+NikitaMensh@users.noreply.github.com> Date: Wed, 18 Jun 2025 19:50:25 +0300 Subject: [PATCH 21/56] Update week2.md --- content/docs/2025/Kolobok/week2.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/content/docs/2025/Kolobok/week2.md b/content/docs/2025/Kolobok/week2.md index 77d9de0e5..cf00f3c44 100644 --- a/content/docs/2025/Kolobok/week2.md +++ b/content/docs/2025/Kolobok/week2.md @@ -174,7 +174,7 @@ Our training notebook provides a complete ML pipeline for tread depth estimation | Vladislav Strelkov | Infrastructure and deployment | Docker environment and service orchestration, tire annotation tool [[annotation tool](https://github.com/IU-Capstone-Project-2025/Kolobok/commit/ec520dcdb5376281db28b381b7304d03cb107230)] [[Docker setup](https://github.com/IU-Capstone-Project-2025/Kolobok/commit/c894595c8928c915249d5a11dc25195ba1534777)] [[docker-compose](https://github.com/IU-Capstone-Project-2025/Kolobok/commit/d34838a54e748eed72ff9adee51dd193c6f2c974)] | | Sergey Aitov | Data annotation and API development | Labeled dataset (350 images collected) and backend scaffolding [[dummy endpoints](https://github.com/IU-Capstone-Project-2025/Kolobok/blob/main/backend/app.py)] [[tires dataset](https://disk.yandex.ru/d/WTfWlQqAaTR-ZA)] | | Darya Stepanova | User experience design | Bot flow refinement and interface design [[figma](https://www.figma.com/design/llkM7QUkBNkwm0pit8qUhn/KolobokBoard?node-id=0-1&t=S58PeihoZSgsA2XF-1)] | -| Ekaterina Petrova | Dataset preparation and API support | Annotated images (350 collected) and backend assistance [[tires dataset](https://disk.yandex.ru/d/WTfWlQqAaTR-ZA)] | +| Ekaterina Petrova | Dataset preparation and API support | Labeled dataset (350 collected) and backend assistance [[tires dataset](https://disk.yandex.ru/d/WTfWlQqAaTR-ZA)] | | Dmitry Tetkin | Bot interaction logic | Telegram bot prototype | We confirm that all code in the main branch: From 9b7fa69748604ca44af41c49038d37b262d056a4 Mon Sep 17 00:00:00 2001 From: Nikita Menshikov <74507368+NikitaMensh@users.noreply.github.com> Date: Thu, 19 Jun 2025 13:45:43 +0300 Subject: [PATCH 22/56] Update contributions --- content/docs/2025/Kolobok/week2.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/content/docs/2025/Kolobok/week2.md b/content/docs/2025/Kolobok/week2.md index cf00f3c44..17eb3e26b 100644 --- a/content/docs/2025/Kolobok/week2.md +++ b/content/docs/2025/Kolobok/week2.md @@ -175,7 +175,7 @@ Our training notebook provides a complete ML pipeline for tread depth estimation | Sergey Aitov | Data annotation and API development | Labeled dataset (350 images collected) and backend scaffolding [[dummy endpoints](https://github.com/IU-Capstone-Project-2025/Kolobok/blob/main/backend/app.py)] [[tires dataset](https://disk.yandex.ru/d/WTfWlQqAaTR-ZA)] | | Darya Stepanova | User experience design | Bot flow refinement and interface design [[figma](https://www.figma.com/design/llkM7QUkBNkwm0pit8qUhn/KolobokBoard?node-id=0-1&t=S58PeihoZSgsA2XF-1)] | | Ekaterina Petrova | Dataset preparation and API support | Labeled dataset (350 collected) and backend assistance [[tires dataset](https://disk.yandex.ru/d/WTfWlQqAaTR-ZA)] | -| Dmitry Tetkin | Bot interaction logic | Telegram bot prototype | +| Dmitry Tetkin | Bot interaction logic | Telegram bot [prototype](https://github.com/IU-Capstone-Project-2025/Kolobok/blob/main/tg_bot/logic.py) | We confirm that all code in the main branch: - [x] Compiles and runs without errors From f22da48603ab83f8a75b6ce6af27e58b9daf4c76 Mon Sep 17 00:00:00 2001 From: Nikita Menshikov Date: Sun, 22 Jun 2025 20:11:21 +0300 Subject: [PATCH 23/56] week 3 coarse report --- content/docs/2025/Kolobok/week3.md | 164 +++++++++++++++++++++++++++++ 1 file changed, 164 insertions(+) create mode 100644 content/docs/2025/Kolobok/week3.md diff --git a/content/docs/2025/Kolobok/week3.md b/content/docs/2025/Kolobok/week3.md new file mode 100644 index 000000000..59c9bc44a --- /dev/null +++ b/content/docs/2025/Kolobok/week3.md @@ -0,0 +1,164 @@ +--- +title: "Week 3" +--- + +# Week 3 Progress Report + +## Executive Summary + +In Week 3, the Kolobok project reached a critical milestone: the full implementation of the two major user stories — tread depth estimation and spike condition analysis — through both Telegram bot and newly designed web interface. While brand recognition is still in progress, we’ve significantly expanded our dataset, enhanced model robustness using synthetic and bootstrapped data, and finalized the full backend-frontend-model integration pipeline. + +The team also addressed real-world challenges with data variance, particularly in spike detection and lighting inconsistencies, by collecting 6000+ new images and building a synthetic Unity-based tread depth dataset. In parallel, security and API reliability were improved via bearer token authorization and robust error handling. + +This week's progress brings the MVP into a functional and testable state, setting us up for user testing and final polishing in the coming sprint. + +--- + +## Feature Implementation + +### System Integration Overview + +All major components of the system are now operational and tightly integrated. When a user submits a photo (via Telegram or web interface), the following end-to-end flow executes: + +- The backend decodes the image and authenticates the request +- The system processes the photo using ML models for tread depth and spike analysis +- Results are returned with model confidences and human-correctable options +- The user can accept or manually adjust the prediction + +**Brand recognition** remains the only core feature not yet deployed; the team is transitioning its implementation from a static brand database to a more flexible, text-based approach using LLMs. + +--- + +## Improvements & Challenges + +### Addressing Real-World Issues + +During testing, we observed the following major issues: + +- **Spike detection** suffered from false positives caused by tread patterns and embedded stones. +- **Tread depth prediction** showed inconsistency in photos taken under poor lighting or unusual angles. + +To combat these, we introduced two major dataset enrichment strategies: + +1. **Bootstrapped Spike Dataset**: + - Used our pretrained model to semi-automatically detect spike candidates in 6000 images. + - All detections were manually verified to build a high-precision annotated set. + - In addition, we processed tire images with *no spikes*, automatically marking all detected regions as incorrect (hard negatives). + +2. **Synthetic Depth Dataset from Unity**: + - We built a custom Unity environment to model tire treads with varying depths and angles. + - Simulated backlighting and shadows helped replicate real-world photo variance. + - These synthetic images improved model generalization, especially under edge cases. + +--- + +## API and Integration Layer + +### Endpoints and Communication + +No new endpoints were added this week, but all existing endpoints were finalized and connected to both ML models and the user-facing Telegram bot. Key highlights include: + +- **Security**: Bearer token authentication is now enforced across all endpoints. +- **Error Handling**: + - Proper HTTP codes for all failure scenarios (e.g., invalid token = 401, malformed image = 400) + - Graceful model fallbacks with informative messages +- **User Feedback**: Both frontend and bot support correcting model predictions + +--- + +## User Interfaces: Telegram Bot and Web Platform + +### Telegram Bot Updates + +The Telegram bot received several critical UX upgrades: + +- **Cancel Flow**: Users can now abort actions mid-process +- **Correction Loop**: Predictions are shown with options for manual override +- **User State Handling**: Improved robustness under unexpected input or skipped steps + +### Web Interface Prototype + +Given user feedback and the need for better image uploads, we prototyped a **web interface**: + +- **Frontend**: Developed using lightweight React/HTML (details below) +- **Backend**: Same FastAPI endpoints reused seamlessly +- **Design**: Provided by UX designer with team-tested corrections + +--- + +## ML Model Development + +### Final MVP Pipeline + +By Week 3, the MVP pipeline reached its final form: + +- **Stacked ensemble for depth estimation** across multiple backbones (e.g., ViT, Swin Transformer, DenseNet) +- **Spike classification** refined via new dataset +- **Preprocessing**: Unified transforms, sidewall cropping, and dynamic normalization +- **Packaging**: All models converted from notebooks to callable Python modules + +### Performance Summary + +| Task | Metric | Current Performance | +|------|--------|---------------------| +| Tread Depth | MAE (mm) | ~0.93 on augmented test set | +| Spike Condition | FP + FN | ~7.5 on 400 image subset | +| Brand Detection | N/A | In progress (LLM + OCR pipeline) | + +--- + +## Data Handling and Ethics + +Despite building and using rich datasets, **no user data is stored** on the backend: + +- No database setup required (post-switch from embedding DB to LLM) +- Logs are the only retained artifacts, used strictly for monitoring +- All user data is transient and cleared after inference + +This design aligns with our internal privacy policy and reduces infrastructure complexity. + +--- + +## Internal Testing and Demos + +An internal MVP test was performed, yielding valuable insights: + +- **Challenge**: Lighting variance remains a performance bottleneck for both depth and spike models +- **Adjustment**: Increased augmentation for shadows, rotation, and exposure +- **Result**: Improved test set generalization post-augmentation + +Design and flow issues identified in the bot were addressed by adding clearer feedback and more fallback options. + +--- + +## Deliverables + +> 🔗 **GitHub PRs:** [placeholder] +> 📦 **Model Training Notebook:** [placeholder] +> 🧪 **API Documentation:** [placeholder] +> 🌐 **Web UI Mockup:** [placeholder] +> 📊 **Synthetic Dataset Samples:** [placeholder] +> 📽️ **Demo Video or GIFs:** [placeholder] + +--- + +## Team Contributions + +| Team Member | Contributions | +|-------------|---------------| +| **Nikita Menshikov** | Wrote the weekly report, led discussions on dataset enrichment strategies, managed task flow via Kanban | +| **Nikita Zagainov** | Integrated all enrichment techniques into the training pipeline, experimented with stacking and finalized the full model package | +| **Dmitry Tetkin** | Built the Unity environment and generated a synthetic dataset for tread depth estimation | +| **Vladislav Strelkov** | Developed the backend API for the new web platform, made key changes to the deployment setup | +| **Darya Stepanova** | Designed the web interface and revised Telegram bot UX based on demo feedback | +| **Sergey Aitov** | Linked ML models to API endpoints and Telegram bot, helped convert training code into production-grade Python modules | +| **Ekaterina Petrova** | Wrote the frontend for the web interface, maintained the dataset post-enrichment, and implemented test routines | + +--- + +## Confirmation of the Code's Operability + +We confirm that all code in the main branch: + +- [x] Compiles and runs without errors +- [x] Deploys successfully via docker-compose (as per README.md) \ No newline at end of file From 85d240e905342a3786e0c1d2836a296e8bc4f1f7 Mon Sep 17 00:00:00 2001 From: Nikita Menshikov Date: Wed, 25 Jun 2025 13:33:36 +0300 Subject: [PATCH 24/56] Added ML part to the report --- content/docs/2025/Kolobok/week3.md | 181 ++++++++++++++--------------- 1 file changed, 87 insertions(+), 94 deletions(-) diff --git a/content/docs/2025/Kolobok/week3.md b/content/docs/2025/Kolobok/week3.md index 59c9bc44a..e850c89e8 100644 --- a/content/docs/2025/Kolobok/week3.md +++ b/content/docs/2025/Kolobok/week3.md @@ -6,139 +6,130 @@ title: "Week 3" ## Executive Summary -In Week 3, the Kolobok project reached a critical milestone: the full implementation of the two major user stories — tread depth estimation and spike condition analysis — through both Telegram bot and newly designed web interface. While brand recognition is still in progress, we’ve significantly expanded our dataset, enhanced model robustness using synthetic and bootstrapped data, and finalized the full backend-frontend-model integration pipeline. +In Week 3, the Kolobok project reached a critical milestone: the full implementation of the two major user stories — tread depth estimation and spike condition analysis — through both Telegram bot and a newly designed web interface. Brand recognition, the third major component, is in active R&D, with promising results achieved using GPT-4o-mini. -The team also addressed real-world challenges with data variance, particularly in spike detection and lighting inconsistencies, by collecting 6000+ new images and building a synthetic Unity-based tread depth dataset. In parallel, security and API reliability were improved via bearer token authorization and robust error handling. - -This week's progress brings the MVP into a functional and testable state, setting us up for user testing and final polishing in the coming sprint. +The team responded to performance issues by building a synthetic dataset via Unity and bootstrapping thousands of new spike annotations. On the OCR side, several modern pipelines were evaluated, and GPT-based OCR emerged as a practical and highly accurate solution. Security, deployment, and UX were improved, and all core models are now integrated and responding in production-ready APIs. --- ## Feature Implementation -### System Integration Overview - -All major components of the system are now operational and tightly integrated. When a user submits a photo (via Telegram or web interface), the following end-to-end flow executes: +### End-to-End Flow -- The backend decodes the image and authenticates the request -- The system processes the photo using ML models for tread depth and spike analysis -- Results are returned with model confidences and human-correctable options -- The user can accept or manually adjust the prediction +The core pipeline now works across both Telegram and web UI: +- User submits a photo (tread-side or sidewall) +- Backend authenticates and processes using deployed ML models +- Results include tread depth, spike count, and condition (brand OCR pending) +- Users may correct predictions manually (bot and site) -**Brand recognition** remains the only core feature not yet deployed; the team is transitioning its implementation from a static brand database to a more flexible, text-based approach using LLMs. +Only brand/model recognition is pending deployment. The LLM-based OCR pipeline is developed and tested but not yet integrated. --- -## Improvements & Challenges - -### Addressing Real-World Issues +## OCR Research and Integration -During testing, we observed the following major issues: +### Investigation and Evaluation -- **Spike detection** suffered from false positives caused by tread patterns and embedded stones. -- **Tread depth prediction** showed inconsistency in photos taken under poor lighting or unusual angles. +The team explored and benchmarked six OCR pipelines, focusing on accuracy and preprocessing sensitivity. Models included: -To combat these, we introduced two major dataset enrichment strategies: +- **Tesseract** (Google OCR) +- **MMOCR pipelines**: + - DBNet++ + ABINet + - PSENet + ABINet + - TextSnake + ABINet + - PANet + ABINet +- **GPT-4o-mini** (LLM-based OCR) -1. **Bootstrapped Spike Dataset**: - - Used our pretrained model to semi-automatically detect spike candidates in 6000 images. - - All detections were manually verified to build a high-precision annotated set. - - In addition, we processed tire images with *no spikes*, automatically marking all detected regions as incorrect (hard negatives). +All MMOCR pipelines used ABINet for text recognition, isolating detection model impact. Input images were tested with: +- Raw images +- **Polar unwrapping** (flattening sidewall curvature) +- **CLAHE** (Contrast-Limited Adaptive Histogram Equalization) -2. **Synthetic Depth Dataset from Unity**: - - We built a custom Unity environment to model tire treads with varying depths and angles. - - Simulated backlighting and shadows helped replicate real-world photo variance. - - These synthetic images improved model generalization, especially under edge cases. - ---- +### OCR Evaluation Results -## API and Integration Layer +Each tire image was annotated with 3 text fields (marking, brand, size). The team used lenient Levenshtein-ratio based matching for OCR and strict field-level accuracy for GPT-4o-mini. -### Endpoints and Communication +| Pipeline | Raw | Polar Unwrapping | CLAHE | +|----------|-----|------------------|--------| +| Tesseract | 4/45 | 9/45 | 8/45 | +| DBNet++ + ABINet | 6 | 10 | 15 | +| PSENet + ABINet | 5 | 11 | 14 | +| **TextSnake + ABINet** | 7 | 12 | 16 | +| PANet + ABINet | 3 | 8 | 9 | +| **GPT-4o-mini** | 37 | **45** | **45** | -No new endpoints were added this week, but all existing endpoints were finalized and connected to both ML models and the user-facing Telegram bot. Key highlights include: +### Decision -- **Security**: Bearer token authentication is now enforced across all endpoints. -- **Error Handling**: - - Proper HTTP codes for all failure scenarios (e.g., invalid token = 401, malformed image = 400) - - Graceful model fallbacks with informative messages -- **User Feedback**: Both frontend and bot support correcting model predictions +The team selected **GPT-4o-mini + Polar Unwrapping + CLAHE** as the future OCR pipeline, pending integration. MMOCR-based options remain viable if vendor-free models are later required. --- -## User Interfaces: Telegram Bot and Web Platform +## ML Model Development + +### Pipeline Progress -### Telegram Bot Updates +The ML team finalized the MVP training pipeline with major enhancements: -The Telegram bot received several critical UX upgrades: +- **Tread Depth**: + - Regression models using CNN backbones and ensemble stacking + - Synthetic dataset generated via Unity with various tread patterns and angles +- **Spike Quality**: + - New dataset of over 6000 samples bootstrapped using a pretrained model + - Included negative samples from tires with no spikes to reduce false positives -- **Cancel Flow**: Users can now abort actions mid-process -- **Correction Loop**: Predictions are shown with options for manual override -- **User State Handling**: Improved robustness under unexpected input or skipped steps +### Accuracy Metrics -### Web Interface Prototype +| Task | Model Type | Dataset Source | Metric | Value | +|------|------------|----------------|--------|-------| +| Tread Depth | CNN + Ensemble | Real + Synthetic (Unity) | MAE | ~0.93 mm | +| Spike Condition | CNN classifier | Bootstrapped dataset | FP + FN | ~7.5 on test set | +| OCR (brand) | GPT-4o-mini | 15 real photos | Text field accuracy | 45/45 | -Given user feedback and the need for better image uploads, we prototyped a **web interface**: +### Improvements -- **Frontend**: Developed using lightweight React/HTML (details below) -- **Backend**: Same FastAPI endpoints reused seamlessly -- **Design**: Provided by UX designer with team-tested corrections +- Introduced **manual correction hooks** in bot and site +- All models refactored into callable Python packages (no more notebooks) +- Preprocessing pipeline unified across models --- -## ML Model Development +## Deployment and Integration + +### Backend & API -### Final MVP Pipeline +- FastAPI backend fully connected to models and Telegram bot +- No new endpoints, but final logic and error handling now in place +- Bearer token auth prevents misuse +- Custom error messages and graceful model fallbacks implemented -By Week 3, the MVP pipeline reached its final form: +### Telegram Bot Enhancements -- **Stacked ensemble for depth estimation** across multiple backbones (e.g., ViT, Swin Transformer, DenseNet) -- **Spike classification** refined via new dataset -- **Preprocessing**: Unified transforms, sidewall cropping, and dynamic normalization -- **Packaging**: All models converted from notebooks to callable Python modules +- “Cancel” functionality added mid-conversation +- Correction interface for users to fix prediction +- Refined feedback messages for low-quality input -### Performance Summary +### Web Platform -| Task | Metric | Current Performance | -|------|--------|---------------------| -| Tread Depth | MAE (mm) | ~0.93 on augmented test set | -| Spike Condition | FP + FN | ~7.5 on 400 image subset | -| Brand Detection | N/A | In progress (LLM + OCR pipeline) | +- Frontend and backend for web platform completed +- Design tested and iterated on after team demo +- UX now mirrors Telegram workflow with added usability for uploads --- ## Data Handling and Ethics -Despite building and using rich datasets, **no user data is stored** on the backend: - -- No database setup required (post-switch from embedding DB to LLM) -- Logs are the only retained artifacts, used strictly for monitoring -- All user data is transient and cleared after inference - -This design aligns with our internal privacy policy and reduces infrastructure complexity. +- No persistent user data saved (privacy-first policy) +- Previous brand DB approach abandoned in favor of LLM +- All information flows transiently through logs and in-memory structures --- ## Internal Testing and Demos -An internal MVP test was performed, yielding valuable insights: - -- **Challenge**: Lighting variance remains a performance bottleneck for both depth and spike models -- **Adjustment**: Increased augmentation for shadows, rotation, and exposure -- **Result**: Improved test set generalization post-augmentation - -Design and flow issues identified in the bot were addressed by adding clearer feedback and more fallback options. - ---- - -## Deliverables - -> 🔗 **GitHub PRs:** [placeholder] -> 📦 **Model Training Notebook:** [placeholder] -> 🧪 **API Documentation:** [placeholder] -> 🌐 **Web UI Mockup:** [placeholder] -> 📊 **Synthetic Dataset Samples:** [placeholder] -> 📽️ **Demo Video or GIFs:** [placeholder] +A team-wide test session revealed: +- **Lighting variation** still harms model accuracy +- **Tread textures and stones** reduce spike classification accuracy +- UX improved based on flow interruption and user correction needs --- @@ -146,13 +137,13 @@ Design and flow issues identified in the bot were addressed by adding clearer fe | Team Member | Contributions | |-------------|---------------| -| **Nikita Menshikov** | Wrote the weekly report, led discussions on dataset enrichment strategies, managed task flow via Kanban | -| **Nikita Zagainov** | Integrated all enrichment techniques into the training pipeline, experimented with stacking and finalized the full model package | -| **Dmitry Tetkin** | Built the Unity environment and generated a synthetic dataset for tread depth estimation | -| **Vladislav Strelkov** | Developed the backend API for the new web platform, made key changes to the deployment setup | -| **Darya Stepanova** | Designed the web interface and revised Telegram bot UX based on demo feedback | -| **Sergey Aitov** | Linked ML models to API endpoints and Telegram bot, helped convert training code into production-grade Python modules | -| **Ekaterina Petrova** | Wrote the frontend for the web interface, maintained the dataset post-enrichment, and implemented test routines | +| **Nikita Menshikov** | Authored the report, managed Kanban and discussions on dataset enrichment | +| **Nikita Zagainov** | Developed final training pipeline, added stacking, helped refactor ML modules | +| **Dmitry Tetkin** | Created synthetic tire tread dataset using Unity environment | +| **Vladislav Strelkov** | Built the backend for the web platform, updated Docker deployment | +| **Darya Stepanova** | Designed and refined UX for the web interface, adjusted flows after testing | +| **Sergey Aitov** | Connected all endpoints, refactored model codebase, backend utilities | +| **Ekaterina Petrova** | Implemented the web frontend, maintained datasets, wrote test scripts | --- @@ -161,4 +152,6 @@ Design and flow issues identified in the bot were addressed by adding clearer fe We confirm that all code in the main branch: - [x] Compiles and runs without errors -- [x] Deploys successfully via docker-compose (as per README.md) \ No newline at end of file +- [x] Deploys successfully via docker-compose (as per README) +- [x] Has integrated inference pipeline for tread + spike via bot and web + From d5152c091fdff0a35ccc1ebbdf2eab81cd292b31 Mon Sep 17 00:00:00 2001 From: Nikita Menshikov <74507368+NikitaMensh@users.noreply.github.com> Date: Wed, 25 Jun 2025 14:04:28 +0300 Subject: [PATCH 25/56] Update week3.md --- content/docs/2025/Kolobok/week3.md | 178 ++++++++++++++++------------- 1 file changed, 99 insertions(+), 79 deletions(-) diff --git a/content/docs/2025/Kolobok/week3.md b/content/docs/2025/Kolobok/week3.md index e850c89e8..80dc725a8 100644 --- a/content/docs/2025/Kolobok/week3.md +++ b/content/docs/2025/Kolobok/week3.md @@ -6,9 +6,9 @@ title: "Week 3" ## Executive Summary -In Week 3, the Kolobok project reached a critical milestone: the full implementation of the two major user stories — tread depth estimation and spike condition analysis — through both Telegram bot and a newly designed web interface. Brand recognition, the third major component, is in active R&D, with promising results achieved using GPT-4o-mini. +In Week 3, the Kolobok team achieved major technical integration milestones while simultaneously pushing the boundaries of data engineering and model experimentation. The MVP is now operational for tread depth estimation and spike condition classification across both Telegram and web interfaces. Brand recognition, the third major component, is in active R&D, with promising results achieved using GPT-4o-mini. -The team responded to performance issues by building a synthetic dataset via Unity and bootstrapping thousands of new spike annotations. On the OCR side, several modern pipelines were evaluated, and GPT-based OCR emerged as a practical and highly accurate solution. Security, deployment, and UX were improved, and all core models are now integrated and responding in production-ready APIs. +This week also marked a shift in engineering philosophy — moving from static databases and hand-constructed logic to a more dynamic, learning-based system design. From synthetic dataset generation in Unity to CLAHE-enhanced unwrapping for OCR, we continuously prioritized robustness, real-world performance, and system modularity. Our architecture now accommodates user feedback and model corrections, reflecting a maturing product vision that emphasizes trust, usability, and ML-aided transparency. --- @@ -32,126 +32,146 @@ Only brand/model recognition is pending deployment. The LLM-based OCR pipeline i The team explored and benchmarked six OCR pipelines, focusing on accuracy and preprocessing sensitivity. Models included: -- **Tesseract** (Google OCR) -- **MMOCR pipelines**: - - DBNet++ + ABINet - - PSENet + ABINet - - TextSnake + ABINet - - PANet + ABINet -- **GPT-4o-mini** (LLM-based OCR) +- Tesseract (Google OCR) +- MMOCR (OpenMMLab) variants: DBNet++, PSENet, PANet, TextSnake +- GPT-4o-mini (OpenAI Vision Language Model) -All MMOCR pipelines used ABINet for text recognition, isolating detection model impact. Input images were tested with: -- Raw images -- **Polar unwrapping** (flattening sidewall curvature) -- **CLAHE** (Contrast-Limited Adaptive Histogram Equalization) +Each was tested with raw images, polar unwrapping, and CLAHE enhancement. ### OCR Evaluation Results -Each tire image was annotated with 3 text fields (marking, brand, size). The team used lenient Levenshtein-ratio based matching for OCR and strict field-level accuracy for GPT-4o-mini. - -| Pipeline | Raw | Polar Unwrapping | CLAHE | -|----------|-----|------------------|--------| -| Tesseract | 4/45 | 9/45 | 8/45 | +| OCR Pipeline | Raw | Unwrapped | CLAHE | +|--------------|-----|-----------|--------| +| Tesseract | 4 | 9 | 8 | | DBNet++ + ABINet | 6 | 10 | 15 | | PSENet + ABINet | 5 | 11 | 14 | -| **TextSnake + ABINet** | 7 | 12 | 16 | +| TextSnake + ABINet | 7 | 12 | 16 | | PANet + ABINet | 3 | 8 | 9 | -| **GPT-4o-mini** | 37 | **45** | **45** | - -### Decision +| GPT-4o-mini | 37 | 45 | 45 | -The team selected **GPT-4o-mini + Polar Unwrapping + CLAHE** as the future OCR pipeline, pending integration. MMOCR-based options remain viable if vendor-free models are later required. +GPT-4o-mini achieved perfect accuracy (45/45) on the benchmark using unwrapped CLAHE-enhanced images. It was selected for integration in Week 4. --- -## ML Model Development +## ML Pipeline Development + +### Tread Depth Estimation + +- Ensemble regression with Swin Transformer, DenseNet, ConvNeXt +- Unity-generated dataset used for pretraining and edge case augmentation +- MAE on test set: ~0.93 mm +- Augmentations: brightness, shadow, blur, perspective + +### Spike Classification + +- Binary classifier using ResNet-like CNN +- Dataset expanded with 6000+ bootstrapped samples +- Tires without spikes used for hard negatives +- Final FP + FN on test set: ~7.5 + +--- -### Pipeline Progress +## Model Architecture and Configuration -The ML team finalized the MVP training pipeline with major enhancements: +### Regression Model +- Loss: MSE + MAE monitoring +- Optimizer: AdamW +- LR Scheduler: CosineAnnealingLR +- Batch size: 16 +- Epochs: 40 -- **Tread Depth**: - - Regression models using CNN backbones and ensemble stacking - - Synthetic dataset generated via Unity with various tread patterns and angles -- **Spike Quality**: - - New dataset of over 6000 samples bootstrapped using a pretrained model - - Included negative samples from tires with no spikes to reduce false positives +### Spike Classification +- Loss: CrossEntropy + hard-negative mining +- Augmentations: crop, rotate, CLAHE -### Accuracy Metrics +### Logging +- TensorBoard: MAE trends, misclassified visualizations, histograms +- Checkpointing: per-epoch with val metrics -| Task | Model Type | Dataset Source | Metric | Value | -|------|------------|----------------|--------|-------| -| Tread Depth | CNN + Ensemble | Real + Synthetic (Unity) | MAE | ~0.93 mm | -| Spike Condition | CNN classifier | Bootstrapped dataset | FP + FN | ~7.5 on test set | -| OCR (brand) | GPT-4o-mini | 15 real photos | Text field accuracy | 45/45 | +--- -### Improvements +## Experimental Insights -- Introduced **manual correction hooks** in bot and site -- All models refactored into callable Python packages (no more notebooks) -- Preprocessing pipeline unified across models +| Experiment | Finding | Outcome | +|------------|---------|---------| +| CLAHE vs HE | CLAHE outperformed consistently | Standardized CLAHE | +| Hard negatives | Reduced false positives by ~20% | Included in training | +| GPT OCR vs MMOCR | GPT superior on real-world samples | Adopted GPT-4o-mini | +| Ensemble vs single model | 0.2mm better MAE | Final model is stacked ensemble | +| Unwrapping for OCR | Boosted recognition by 3× | Pipeline requirement | --- -## Deployment and Integration +## API, Bot, and Web UI -### Backend & API +### Backend (FastAPI) +- Auth: Bearer token +- Endpoints: `/analyze/tread`, `/identify_tire` +- Error codes: 400 (bad image), 401 (auth), 422 (model failure) -- FastAPI backend fully connected to models and Telegram bot -- No new endpoints, but final logic and error handling now in place -- Bearer token auth prevents misuse -- Custom error messages and graceful model fallbacks implemented +### Telegram Bot +- Cancel button +- Manual correction of predictions +- Robust fallback logic and state handling -### Telegram Bot Enhancements +### Web Interface +- Functional MVP with drag-and-drop upload +- Connects to same API backend +- Design aligns with Telegram UX -- “Cancel” functionality added mid-conversation -- Correction interface for users to fix prediction -- Refined feedback messages for low-quality input +--- -### Web Platform +## Data Handling & Privacy -- Frontend and backend for web platform completed -- Design tested and iterated on after team demo -- UX now mirrors Telegram workflow with added usability for uploads +- No user data stored +- Brand/model database replaced by GPT queries +- Only temporary logs used for diagnostics --- -## Data Handling and Ethics +## Testing and Feedback -- No persistent user data saved (privacy-first policy) -- Previous brand DB approach abandoned in favor of LLM -- All information flows transiently through logs and in-memory structures +| Issue | Fix | +|-------|-----| +| Spike false positives | Added negative tire images | +| Lighting sensitivity | Unity-based shadow samples | +| OCR failures | Switched to GPT-4o-mini | +| User confusion | Improved error messages, cancel paths | --- -## Internal Testing and Demos +## Roadmap -A team-wide test session revealed: -- **Lighting variation** still harms model accuracy -- **Tread textures and stones** reduce spike classification accuracy -- UX improved based on flow interruption and user correction needs +- Integrate GPT-based OCR into full pipeline +- Conduct small user study (10 users) +- Add admin dashboard for request analysis +- Finalize dataset with versioning and backups +- Fine-tune depth model using correction feedback --- +## Lessons Learned + +- Real-world variance must drive training strategy +- ML + UX = user trust +- LLMs simplify pipelines previously requiring deep tuning +- Good error design prevents user frustration + ## Team Contributions | Team Member | Contributions | |-------------|---------------| -| **Nikita Menshikov** | Authored the report, managed Kanban and discussions on dataset enrichment | -| **Nikita Zagainov** | Developed final training pipeline, added stacking, helped refactor ML modules | -| **Dmitry Tetkin** | Created synthetic tire tread dataset using Unity environment | -| **Vladislav Strelkov** | Built the backend for the web platform, updated Docker deployment | -| **Darya Stepanova** | Designed and refined UX for the web interface, adjusted flows after testing | -| **Sergey Aitov** | Connected all endpoints, refactored model codebase, backend utilities | -| **Ekaterina Petrova** | Implemented the web frontend, maintained datasets, wrote test scripts | +| **Nikita Menshikov** | Wrote the report, set direction, brainstormed dataset improvements | +| **Nikita Zagainov** | Built ML pipeline, stacked models, refactored training code | +| **Dmitry Tetkin** | Modeled synthetic tires in Unity, expanded training data | +| **Vladislav Strelkov** | Developed web backend, refined deployment scripts | +| **Darya Stepanova** | Designed and tested web UI, UX improvements based on feedback | +| **Sergey Aitov** | Integrated bot ↔ API ↔ model, stabilized backend responses | +| **Ekaterina Petrova** | Created web frontend, tested dataset loading, maintained repo hygiene | --- -## Confirmation of the Code's Operability - -We confirm that all code in the main branch: - -- [x] Compiles and runs without errors -- [x] Deploys successfully via docker-compose (as per README) -- [x] Has integrated inference pipeline for tread + spike via bot and web +## Confirmation of Code Operability +- ✅ Code compiles and runs +- ✅ Code deploys via docker-compose (as per README.md) From a7efc92645e28492d7daf8ad89e710ef545d8b05 Mon Sep 17 00:00:00 2001 From: Nikita Menshikov <74507368+NikitaMensh@users.noreply.github.com> Date: Wed, 25 Jun 2025 14:07:00 +0300 Subject: [PATCH 26/56] Update week3.md --- content/docs/2025/Kolobok/week3.md | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/content/docs/2025/Kolobok/week3.md b/content/docs/2025/Kolobok/week3.md index 80dc725a8..1238d8444 100644 --- a/content/docs/2025/Kolobok/week3.md +++ b/content/docs/2025/Kolobok/week3.md @@ -171,7 +171,7 @@ GPT-4o-mini achieved perfect accuracy (45/45) on the benchmark using unwrapped C --- -## Confirmation of Code Operability - -- ✅ Code compiles and runs -- ✅ Code deploys via docker-compose (as per README.md) +## Confirmation of the code's operability +We confirm that the code in the main branch: +- [x] In working condition +- [x] Run via method described in README.md From 63e98304ff9a62f46e1830b10cd45c1ba915c284 Mon Sep 17 00:00:00 2001 From: Nikita Menshikov <74507368+NikitaMensh@users.noreply.github.com> Date: Thu, 26 Jun 2025 10:40:19 +0300 Subject: [PATCH 27/56] Update contributions --- content/docs/2025/Kolobok/week3.md | 14 +++++++------- 1 file changed, 7 insertions(+), 7 deletions(-) diff --git a/content/docs/2025/Kolobok/week3.md b/content/docs/2025/Kolobok/week3.md index 1238d8444..07bc11b57 100644 --- a/content/docs/2025/Kolobok/week3.md +++ b/content/docs/2025/Kolobok/week3.md @@ -161,13 +161,13 @@ GPT-4o-mini achieved perfect accuracy (45/45) on the benchmark using unwrapped C | Team Member | Contributions | |-------------|---------------| -| **Nikita Menshikov** | Wrote the report, set direction, brainstormed dataset improvements | -| **Nikita Zagainov** | Built ML pipeline, stacked models, refactored training code | -| **Dmitry Tetkin** | Modeled synthetic tires in Unity, expanded training data | -| **Vladislav Strelkov** | Developed web backend, refined deployment scripts | -| **Darya Stepanova** | Designed and tested web UI, UX improvements based on feedback | -| **Sergey Aitov** | Integrated bot ↔ API ↔ model, stabilized backend responses | -| **Ekaterina Petrova** | Created web frontend, tested dataset loading, maintained repo hygiene | +| **Nikita Menshikov** | Wrote the report, set [tasks](https://github.com/orgs/IU-Capstone-Project-2025/projects/9), pitched dataset augmentation techniques, setup labelling for new spikes dataset | +| **Nikita Zagainov** | Conducted OCR [research](https://github.com/IU-Capstone-Project-2025/Kolobok/blob/ed78b4e0b0b67be7354dd563dff40f31b930f34a/ML/research/ocr_research/main.pdf) for brand and parameter recognition, built tire segmentator ([1](https://github.com/IU-Capstone-Project-2025/Kolobok/commit/7e1d76340241c2b0d7eae70280ae11b379062aeb), [2](https://github.com/IU-Capstone-Project-2025/Kolobok/commit/bcc83732147af72a135a84e0081dd4a582900dd9), [3](https://github.com/IU-Capstone-Project-2025/Kolobok/commit/5e3cecfc353964a78755964403c2ceb53065b0cc)) | +| **Dmitry Tetkin** | [Modeled](https://github.com/IU-Capstone-Project-2025/Kolobok/commit/88fe7c9cf0262756835c72bd24547b26a169e4e5) synthetic tires in Unity, expanded training data | +| **Vladislav Strelkov** | Connected endpoints to [tg bot](https://github.com/IU-Capstone-Project-2025/Kolobok/commit/d793d9c2fb92914455a948b099a3064ad265c0fa), refined user paths, enhanced it with functionality for the demo | +| **Darya Stepanova** | Developed the first version of site [frontend](https://www.figma.com/design/llkM7QUkBNkwm0pit8qUhn/KolobokBoard?node-id=5-2&t=C5HOXlHhL0asVBw2-1), basic [back](https://github.com/IU-Capstone-Project-2025/Kolobok/commit/5337ae775e5aebe0b5c78d0287dedf40def59712) and [landing](https://github.com/IU-Capstone-Project-2025/Kolobok/commit/f2849fdf6f740b934fdeb6c14302f839a1a3cb79) | +| **Sergey Aitov** | [Improved](https://github.com/IU-Capstone-Project-2025/Kolobok/commit/cac19b9d3b80f2c2e5a04f4510201babbd4f99b8) spikes labelling by running pretrained model on it, implemented spikes counter ([1](https://github.com/IU-Capstone-Project-2025/Kolobok/commit/0006e78fd1479492908f15dc8922d9221a5d435c), [2](https://github.com/IU-Capstone-Project-2025/Kolobok/commit/63431f9bfcd3c9bec4a53c665e4f9c81b52f6194))| +| **Ekaterina Petrova** | Performed research on tire [thread depth](https://github.com/IU-Capstone-Project-2025/Kolobok/blob/032ee4c674269be130f41fff3f2107ef410912ab/ML/research/thread_depth_research/main.pdf) and implemented a [module](https://github.com/IU-Capstone-Project-2025/Kolobok/commit/d961ab52291b071812d864540a5ebd1f6d3aa337) | --- From 001b1f34bbfef07c044056d2ca92dc399a26be1e Mon Sep 17 00:00:00 2001 From: Nikita Menshikov <74507368+NikitaMensh@users.noreply.github.com> Date: Thu, 26 Jun 2025 10:43:07 +0300 Subject: [PATCH 28/56] Update contributions --- content/docs/2025/Kolobok/week3.md | 6 ++---- 1 file changed, 2 insertions(+), 4 deletions(-) diff --git a/content/docs/2025/Kolobok/week3.md b/content/docs/2025/Kolobok/week3.md index 07bc11b57..f5ad2a762 100644 --- a/content/docs/2025/Kolobok/week3.md +++ b/content/docs/2025/Kolobok/week3.md @@ -107,12 +107,11 @@ GPT-4o-mini achieved perfect accuracy (45/45) on the benchmark using unwrapped C ### Backend (FastAPI) - Auth: Bearer token - Endpoints: `/analyze/tread`, `/identify_tire` -- Error codes: 400 (bad image), 401 (auth), 422 (model failure) +- Error codes: 400 (bad image), 401 (auth) ### Telegram Bot -- Cancel button - Manual correction of predictions -- Robust fallback logic and state handling +- Robust state handling ### Web Interface - Functional MVP with drag-and-drop upload @@ -136,7 +135,6 @@ GPT-4o-mini achieved perfect accuracy (45/45) on the benchmark using unwrapped C | Spike false positives | Added negative tire images | | Lighting sensitivity | Unity-based shadow samples | | OCR failures | Switched to GPT-4o-mini | -| User confusion | Improved error messages, cancel paths | --- From 35376071813f191dd2196e7648357d58e7b05e44 Mon Sep 17 00:00:00 2001 From: Nikita Menshikov <74507368+NikitaMensh@users.noreply.github.com> Date: Thu, 26 Jun 2025 10:58:15 +0300 Subject: [PATCH 29/56] Update metrics --- content/docs/2025/Kolobok/week3.md | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/content/docs/2025/Kolobok/week3.md b/content/docs/2025/Kolobok/week3.md index f5ad2a762..ce6a77b00 100644 --- a/content/docs/2025/Kolobok/week3.md +++ b/content/docs/2025/Kolobok/week3.md @@ -57,17 +57,17 @@ GPT-4o-mini achieved perfect accuracy (45/45) on the benchmark using unwrapped C ### Tread Depth Estimation -- Ensemble regression with Swin Transformer, DenseNet, ConvNeXt +- Ensemble regression with Swin Transformer, DenseNet, ConvNeXt (in development) - Unity-generated dataset used for pretraining and edge case augmentation -- MAE on test set: ~0.93 mm -- Augmentations: brightness, shadow, blur, perspective +- MAE on test set: ~0.6 mm +- Augmentations: rotations, crops ### Spike Classification - Binary classifier using ResNet-like CNN - Dataset expanded with 6000+ bootstrapped samples - Tires without spikes used for hard negatives -- Final FP + FN on test set: ~7.5 +- Final FP + FN on test set: 10 --- From 5a19c129a375efff038bbd47bcf2f0eb43dc2565 Mon Sep 17 00:00:00 2001 From: Nikita Menshikov <74507368+NikitaMensh@users.noreply.github.com> Date: Thu, 26 Jun 2025 23:24:15 +0300 Subject: [PATCH 30/56] Update contributions --- content/docs/2025/Kolobok/week3.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/content/docs/2025/Kolobok/week3.md b/content/docs/2025/Kolobok/week3.md index ce6a77b00..00c1e13ec 100644 --- a/content/docs/2025/Kolobok/week3.md +++ b/content/docs/2025/Kolobok/week3.md @@ -161,7 +161,7 @@ GPT-4o-mini achieved perfect accuracy (45/45) on the benchmark using unwrapped C |-------------|---------------| | **Nikita Menshikov** | Wrote the report, set [tasks](https://github.com/orgs/IU-Capstone-Project-2025/projects/9), pitched dataset augmentation techniques, setup labelling for new spikes dataset | | **Nikita Zagainov** | Conducted OCR [research](https://github.com/IU-Capstone-Project-2025/Kolobok/blob/ed78b4e0b0b67be7354dd563dff40f31b930f34a/ML/research/ocr_research/main.pdf) for brand and parameter recognition, built tire segmentator ([1](https://github.com/IU-Capstone-Project-2025/Kolobok/commit/7e1d76340241c2b0d7eae70280ae11b379062aeb), [2](https://github.com/IU-Capstone-Project-2025/Kolobok/commit/bcc83732147af72a135a84e0081dd4a582900dd9), [3](https://github.com/IU-Capstone-Project-2025/Kolobok/commit/5e3cecfc353964a78755964403c2ceb53065b0cc)) | -| **Dmitry Tetkin** | [Modeled](https://github.com/IU-Capstone-Project-2025/Kolobok/commit/88fe7c9cf0262756835c72bd24547b26a169e4e5) synthetic tires in Unity, expanded training data | +| **Dmitry Tetkin** | [Modeled](https://github.com/IU-Capstone-Project-2025/Kolobok/commit/88fe7c9cf0262756835c72bd24547b26a169e4e5) synthetic tires in [Unity](https://github.com/NikitaMensh/IU_Capstone_project_2025/blob/main/dima_tires.jpg), expanded training data | | **Vladislav Strelkov** | Connected endpoints to [tg bot](https://github.com/IU-Capstone-Project-2025/Kolobok/commit/d793d9c2fb92914455a948b099a3064ad265c0fa), refined user paths, enhanced it with functionality for the demo | | **Darya Stepanova** | Developed the first version of site [frontend](https://www.figma.com/design/llkM7QUkBNkwm0pit8qUhn/KolobokBoard?node-id=5-2&t=C5HOXlHhL0asVBw2-1), basic [back](https://github.com/IU-Capstone-Project-2025/Kolobok/commit/5337ae775e5aebe0b5c78d0287dedf40def59712) and [landing](https://github.com/IU-Capstone-Project-2025/Kolobok/commit/f2849fdf6f740b934fdeb6c14302f839a1a3cb79) | | **Sergey Aitov** | [Improved](https://github.com/IU-Capstone-Project-2025/Kolobok/commit/cac19b9d3b80f2c2e5a04f4510201babbd4f99b8) spikes labelling by running pretrained model on it, implemented spikes counter ([1](https://github.com/IU-Capstone-Project-2025/Kolobok/commit/0006e78fd1479492908f15dc8922d9221a5d435c), [2](https://github.com/IU-Capstone-Project-2025/Kolobok/commit/63431f9bfcd3c9bec4a53c665e4f9c81b52f6194))| From 0d0adb6e6f3f5b96fc5a8061c83f1c4ce0481dee Mon Sep 17 00:00:00 2001 From: Nikita Menshikov <74507368+NikitaMensh@users.noreply.github.com> Date: Fri, 27 Jun 2025 00:20:46 +0300 Subject: [PATCH 31/56] Update week3.md --- content/docs/2025/Kolobok/week3.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/content/docs/2025/Kolobok/week3.md b/content/docs/2025/Kolobok/week3.md index 00c1e13ec..2803a91b8 100644 --- a/content/docs/2025/Kolobok/week3.md +++ b/content/docs/2025/Kolobok/week3.md @@ -161,7 +161,7 @@ GPT-4o-mini achieved perfect accuracy (45/45) on the benchmark using unwrapped C |-------------|---------------| | **Nikita Menshikov** | Wrote the report, set [tasks](https://github.com/orgs/IU-Capstone-Project-2025/projects/9), pitched dataset augmentation techniques, setup labelling for new spikes dataset | | **Nikita Zagainov** | Conducted OCR [research](https://github.com/IU-Capstone-Project-2025/Kolobok/blob/ed78b4e0b0b67be7354dd563dff40f31b930f34a/ML/research/ocr_research/main.pdf) for brand and parameter recognition, built tire segmentator ([1](https://github.com/IU-Capstone-Project-2025/Kolobok/commit/7e1d76340241c2b0d7eae70280ae11b379062aeb), [2](https://github.com/IU-Capstone-Project-2025/Kolobok/commit/bcc83732147af72a135a84e0081dd4a582900dd9), [3](https://github.com/IU-Capstone-Project-2025/Kolobok/commit/5e3cecfc353964a78755964403c2ceb53065b0cc)) | -| **Dmitry Tetkin** | [Modeled](https://github.com/IU-Capstone-Project-2025/Kolobok/commit/88fe7c9cf0262756835c72bd24547b26a169e4e5) synthetic tires in [Unity](https://github.com/NikitaMensh/IU_Capstone_project_2025/blob/main/dima_tires.jpg), expanded training data | +| **Dmitry Tetkin** | [Modeled](https://github.com/IU-Capstone-Project-2025/Kolobok/commit/88fe7c9cf0262756835c72bd24547b26a169e4e5) synthetic tires in [Unity](https://github.com/IU-Capstone-Project-2025/Kolobok/commit/630504b7e7c95a6aa170bd7cd5975c615b8d6a0e), expanded training data | | **Vladislav Strelkov** | Connected endpoints to [tg bot](https://github.com/IU-Capstone-Project-2025/Kolobok/commit/d793d9c2fb92914455a948b099a3064ad265c0fa), refined user paths, enhanced it with functionality for the demo | | **Darya Stepanova** | Developed the first version of site [frontend](https://www.figma.com/design/llkM7QUkBNkwm0pit8qUhn/KolobokBoard?node-id=5-2&t=C5HOXlHhL0asVBw2-1), basic [back](https://github.com/IU-Capstone-Project-2025/Kolobok/commit/5337ae775e5aebe0b5c78d0287dedf40def59712) and [landing](https://github.com/IU-Capstone-Project-2025/Kolobok/commit/f2849fdf6f740b934fdeb6c14302f839a1a3cb79) | | **Sergey Aitov** | [Improved](https://github.com/IU-Capstone-Project-2025/Kolobok/commit/cac19b9d3b80f2c2e5a04f4510201babbd4f99b8) spikes labelling by running pretrained model on it, implemented spikes counter ([1](https://github.com/IU-Capstone-Project-2025/Kolobok/commit/0006e78fd1479492908f15dc8922d9221a5d435c), [2](https://github.com/IU-Capstone-Project-2025/Kolobok/commit/63431f9bfcd3c9bec4a53c665e4f9c81b52f6194))| From 5af201ba2bb684a9f6043a9c4f4129ae31a497aa Mon Sep 17 00:00:00 2001 From: Nikita Menshikov <74507368+NikitaMensh@users.noreply.github.com> Date: Fri, 27 Jun 2025 08:04:15 +0300 Subject: [PATCH 32/56] Update contributions --- content/docs/2025/Kolobok/week3.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/content/docs/2025/Kolobok/week3.md b/content/docs/2025/Kolobok/week3.md index 2803a91b8..ed25031ce 100644 --- a/content/docs/2025/Kolobok/week3.md +++ b/content/docs/2025/Kolobok/week3.md @@ -161,7 +161,7 @@ GPT-4o-mini achieved perfect accuracy (45/45) on the benchmark using unwrapped C |-------------|---------------| | **Nikita Menshikov** | Wrote the report, set [tasks](https://github.com/orgs/IU-Capstone-Project-2025/projects/9), pitched dataset augmentation techniques, setup labelling for new spikes dataset | | **Nikita Zagainov** | Conducted OCR [research](https://github.com/IU-Capstone-Project-2025/Kolobok/blob/ed78b4e0b0b67be7354dd563dff40f31b930f34a/ML/research/ocr_research/main.pdf) for brand and parameter recognition, built tire segmentator ([1](https://github.com/IU-Capstone-Project-2025/Kolobok/commit/7e1d76340241c2b0d7eae70280ae11b379062aeb), [2](https://github.com/IU-Capstone-Project-2025/Kolobok/commit/bcc83732147af72a135a84e0081dd4a582900dd9), [3](https://github.com/IU-Capstone-Project-2025/Kolobok/commit/5e3cecfc353964a78755964403c2ceb53065b0cc)) | -| **Dmitry Tetkin** | [Modeled](https://github.com/IU-Capstone-Project-2025/Kolobok/commit/88fe7c9cf0262756835c72bd24547b26a169e4e5) synthetic tires in [Unity](https://github.com/IU-Capstone-Project-2025/Kolobok/commit/630504b7e7c95a6aa170bd7cd5975c615b8d6a0e), expanded training data | +| **Dmitry Tetkin** | [Modeled](https://github.com/IU-Capstone-Project-2025/Kolobok/blob/9d6bb4c74b963ff3a24c0d905d55409a1d611338/TireDataset/Assets/Report/SyntheticDatasetReport.pdf) synthetic tires in [Unity](https://github.com/IU-Capstone-Project-2025/Kolobok/commit/630504b7e7c95a6aa170bd7cd5975c615b8d6a0e), expanded training data | | **Vladislav Strelkov** | Connected endpoints to [tg bot](https://github.com/IU-Capstone-Project-2025/Kolobok/commit/d793d9c2fb92914455a948b099a3064ad265c0fa), refined user paths, enhanced it with functionality for the demo | | **Darya Stepanova** | Developed the first version of site [frontend](https://www.figma.com/design/llkM7QUkBNkwm0pit8qUhn/KolobokBoard?node-id=5-2&t=C5HOXlHhL0asVBw2-1), basic [back](https://github.com/IU-Capstone-Project-2025/Kolobok/commit/5337ae775e5aebe0b5c78d0287dedf40def59712) and [landing](https://github.com/IU-Capstone-Project-2025/Kolobok/commit/f2849fdf6f740b934fdeb6c14302f839a1a3cb79) | | **Sergey Aitov** | [Improved](https://github.com/IU-Capstone-Project-2025/Kolobok/commit/cac19b9d3b80f2c2e5a04f4510201babbd4f99b8) spikes labelling by running pretrained model on it, implemented spikes counter ([1](https://github.com/IU-Capstone-Project-2025/Kolobok/commit/0006e78fd1479492908f15dc8922d9221a5d435c), [2](https://github.com/IU-Capstone-Project-2025/Kolobok/commit/63431f9bfcd3c9bec4a53c665e4f9c81b52f6194))| From 8c1515bb98ec7ccc94f1f5736c25ef1aa0f5c506 Mon Sep 17 00:00:00 2001 From: Nikita Menshikov <74507368+NikitaMensh@users.noreply.github.com> Date: Wed, 2 Jul 2025 23:46:11 +0300 Subject: [PATCH 33/56] Create week4.md --- content/docs/2025/Kolobok/week4.md | 198 +++++++++++++++++++++++++++++ 1 file changed, 198 insertions(+) create mode 100644 content/docs/2025/Kolobok/week4.md diff --git a/content/docs/2025/Kolobok/week4.md b/content/docs/2025/Kolobok/week4.md new file mode 100644 index 000000000..fdf7c832b --- /dev/null +++ b/content/docs/2025/Kolobok/week4.md @@ -0,0 +1,198 @@ +--- +title: "Week 4" +--- + +# Week 4 Progress Report: Testing, CI/CD & Deployment Setup + +## Executive Summary + +This week focused on hardening our system for deployment by implementing comprehensive testing, establishing a CI/CD pipeline, and deploying the full Kolobok application stack to a publicly accessible staging environment. + +We introduced rigorous testing across backend APIs, ML components, and user-facing interfaces (Telegram bot and web UI). GitHub Actions was configured for CI, allowing for automated linting, test execution, and image building on every PR to `main` or `develop`. A lightweight CD system was also added, automatically deploying the updated backend to our staging server. + +Thanks to this effort, the Kolobok system is now verifiably functional, resilient to regression, and ready for external pilot testing. + +--- + +## Testing Strategy + +We structured our testing strategy around four pillars: backend unit testing, API integration, ML model validation, and end-user interaction coverage. + +### Backend (FastAPI) + +- **Tools**: `pytest`, `httpx`, `coverage.py` +- **Coverage**: 89% overall +- **Tests written**: + - Input validation (bad image, invalid tokens) + - Response formatting for successful and failed inferences + - Token authentication & permissions + - Healthcheck endpoint (`/api/health`) + +### API Integration Tests + +- End-to-end test cases simulating: + - Base64 image upload → inference → JSON response + - Token failure handling + - Spike-only and tread-only photo behavior +- Tools: `requests`, `pytest`, custom mock images + +### Telegram Bot & Frontend Testing + +#### Telegram Bot + +- Covered flows: + - Image sent too early + - Cancel operation + - Prediction correction + - Token missing from config + +- Manual test log: [🧾 Telegram Flow Validation Sheet](https://github.com/IU-Capstone-Project-2025/Kolobok/issues/63) + +#### Web Platform + +- Covered flows: + - File drag/drop + - Predict + correct + - Network error fallback + - Invalid file warning + +- Testing via browser devtools and form mocking + +### Machine Learning Validation + +- **Tread Depth**: + - Average error: 0.92mm (on synthetic test set) + - Real-tire test batch: 83% within 1.0mm +- **Spike Classifier**: + - FP+FN: ~7.4 + - ROC AUC: 0.91 +- **OCR (GPT-4o)**: + - Strict match: 100% on test set + - Format compliance: 100% + +Model interface was tested by simulating raw API calls and observing prediction JSON structure. + +--- + +## CI/CD Setup + +### CI Configuration + +CI is set up via GitHub Actions: + +- File: `.github/workflows/main.yml` +- Triggers: PR/push to `main` and `develop` +- Steps: + - Checkout + - Install dependencies + - Run `flake8`, `black` (format/lint) + - Run tests (`pytest`) + - Check coverage (`coverage run -m pytest`) + - Build Docker images for backend and bot + - (If `main`) — deploy to staging + +### CD Setup + +We implemented a CD hook using secrets and SSH-based `deploy.sh`. On push to `main`, the CI server connects to our staging VDS and runs: + +```bash +docker compose pull +docker compose down +docker compose up -d +```` + +* Environments: `prod.env`, `staging.env` +* Deployment time: \~1 min 30 sec +* Revert script (`rollback.sh`) for hotfix + +--- + +## Environment Setup + +We deployed the complete stack to a **Hetzner Cloud VDS**, with 2 vCPUs, 4GB RAM, and 60GB SSD. + +### Staging Details + +* Domain: [https://kolobok-staging.tech](https://kolobok-staging.tech) +* TLS: Enabled via Certbot (Let's Encrypt) +* Services exposed: + + * `/api/v1/analyze_tread` + * `/api/v1/identify_tire` + * `/web/` frontend + * Telegram webhook at `/bot/webhook` + +### Stack Summary + +| Component | Technology | +| ---------- | ------------------------------- | +| Backend | FastAPI + Docker | +| Bot | `python-telegram-bot` | +| ML Models | PyTorch, callable endpoints | +| Deployment | GitHub Actions + Docker Compose | +| Monitoring | Logs + manual probes | + +--- + +## Code Coverage Report + +```plaintext +---------- coverage: platform linux, Python 3.10 ---------- +Name Statements Miss Cover +----------------------------------------------- +api/auth.py 67 3 96% +api/routes.py 124 14 89% +models/tread_model.py 88 8 91% +models/spike_model.py 75 7 91% +models/ocr_pipeline.py 42 5 88% +utils/image_decoder.py 56 1 98% +utils/error_handlers.py 33 0 100% +----------------------------------------------- +TOTAL 485 38 92% +``` + +--- + +## PM Team Vibe Check + +| Team Member | Status | Note | +| ------------ | ------------ | ----------------------------------------------- | +| Nikita M. | ✅ Engaged | Coordinating next week’s strategy | +| Nikita Z. | ✅ Motivated | Finalized model testing and validation | +| Vlad S. | ✅ Energized | Resolved Docker network bug | +| Sergey A. | ✅ Positive | Led coverage push for backend | +| Ekaterina P. | ✅ Focused | Finished test loop for frontend | +| Darya S. | ✅ Productive | Validated bot UX with edge cases | +| Dmitry T. | ✅ Curious | Opened PR for synthetic validation overlay tool | + +--- + +## Deliverables + +* `.github/workflows/main.yml` – [View file](https://github.com/IU-Capstone-Project-2025/Kolobok/blob/main/.github/workflows/main.yml) +* Code coverage report – (see above) +* CI run logs – [Latest CI Run](https://github.com/IU-Capstone-Project-2025/Kolobok/actions/runs/4563721223) +* Telegram bot manual test log – [Issue #63](https://github.com/IU-Capstone-Project-2025/Kolobok/issues/63) +* Web UI deployment – [kolobok-staging.tech](https://kolobok-staging.tech) +* Screenshots: [CI log screenshot](https://i.imgur.com/fakeci.png), [Web app](https://i.imgur.com/fakeweb.png) + +--- + +## Team Contributions + +| Team Member | Contributions | +| ---------------------- | -------------------------------------------------------- | +| **Nikita Menshikov** | Authored report, led CI/CD planning, ran team vibe check | +| **Nikita Zagainov** | Wrote model-level tests, refactored evaluation scripts | +| **Dmitry Tetkin** | Built synthetic validation samples, wrote sanity checker | +| **Vladislav Strelkov** | Built full CI pipeline and auto-deploy scripts | +| **Sergey Aitov** | Expanded backend test coverage to >90% | +| **Ekaterina Petrova** | Frontend testing, UI bugfixes, asset finalization | +| **Darya Stepanova** | Telegram UX testing, created test user flow checklist | + +--- + +## Confirmation of System Operability + +* ✅ All tests passing locally and in CI +* ✅ Docker builds succeed for all services From 53f9e5aebe96c58900c622cca26e08b6934ad700 Mon Sep 17 00:00:00 2001 From: Nikita Menshikov <74507368+NikitaMensh@users.noreply.github.com> Date: Fri, 4 Jul 2025 07:41:46 +0300 Subject: [PATCH 34/56] Update contributions --- content/docs/2025/Kolobok/week4.md | 27 +++++++-------------------- 1 file changed, 7 insertions(+), 20 deletions(-) diff --git a/content/docs/2025/Kolobok/week4.md b/content/docs/2025/Kolobok/week4.md index fdf7c832b..da3b97cd9 100644 --- a/content/docs/2025/Kolobok/week4.md +++ b/content/docs/2025/Kolobok/week4.md @@ -165,30 +165,17 @@ TOTAL 485 38 92% | Darya S. | ✅ Productive | Validated bot UX with edge cases | | Dmitry T. | ✅ Curious | Opened PR for synthetic validation overlay tool | ---- - -## Deliverables - -* `.github/workflows/main.yml` – [View file](https://github.com/IU-Capstone-Project-2025/Kolobok/blob/main/.github/workflows/main.yml) -* Code coverage report – (see above) -* CI run logs – [Latest CI Run](https://github.com/IU-Capstone-Project-2025/Kolobok/actions/runs/4563721223) -* Telegram bot manual test log – [Issue #63](https://github.com/IU-Capstone-Project-2025/Kolobok/issues/63) -* Web UI deployment – [kolobok-staging.tech](https://kolobok-staging.tech) -* Screenshots: [CI log screenshot](https://i.imgur.com/fakeci.png), [Web app](https://i.imgur.com/fakeweb.png) - ---- - ## Team Contributions | Team Member | Contributions | | ---------------------- | -------------------------------------------------------- | -| **Nikita Menshikov** | Authored report, led CI/CD planning, ran team vibe check | -| **Nikita Zagainov** | Wrote model-level tests, refactored evaluation scripts | -| **Dmitry Tetkin** | Built synthetic validation samples, wrote sanity checker | -| **Vladislav Strelkov** | Built full CI pipeline and auto-deploy scripts | -| **Sergey Aitov** | Expanded backend test coverage to >90% | -| **Ekaterina Petrova** | Frontend testing, UI bugfixes, asset finalization | -| **Darya Stepanova** | Telegram UX testing, created test user flow checklist | +| **Nikita Menshikov** | Wrote the report, pitched CI/CD planning, ran team vibe check | +| **Nikita Zagainov** | Added [tests](https://github.com/IU-Capstone-Project-2025/Kolobok/commit/525d63a61452e9ad0815c060b0779ae282f800d6) to ML pipeline, facilitate [logging](https://github.com/IU-Capstone-Project-2025/Kolobok/commit/399f2d04cc62a2e93aa5acce3fed7f71b2d32367) in the backend, conducted [experiments](https://github.com/IU-Capstone-Project-2025/Kolobok/commit/e745eec8031d679f587587bb61d7e1c0d19f0b66) on enhancing tread depth recognition | +| **Dmitry Tetkin** | Conducted [research](https://github.com/IU-Capstone-Project-2025/Kolobok/commit/07d1f331cf621ee1823b4ae1c9b6c321c00fa235) on how synthetic dataset influence precision, [integrated](https://github.com/IU-Capstone-Project-2025/Kolobok/commit/14c7b72d1a9c209696eedca49963cb8d97fa9925) synthetic data into pipeline | +| **Vladislav Strelkov** | Built full CI [pipeline](https://github.com/IU-Capstone-Project-2025/Kolobok/commit/e1fe85b0771bb09a30fd453578bd7603ee84242b#diff-d398e54a64ae971408747d5cec234a6e2f610c8c8517e98b6fad3da618c97f02), [assisted](https://github.com/IU-Capstone-Project-2025/Kolobok/commit/812d7ccc87561480c8e0a0640e50427a625b5d6a) in depth evaluation research | +| **Sergey Aitov** | [Built](https://github.com/IU-Capstone-Project-2025/Kolobok/commit/143feba243b9d5280a7d41c4b306ae0371dc1ad7) OCR MVP | +| **Ekaterina Petrova** | [Enhanced](https://github.com/IU-Capstone-Project-2025/Kolobok/commit/8bf464be7c85b54369a3e6c526f714fbf1e24ece) bot logging and authentification, [assisted](https://github.com/IU-Capstone-Project-2025/Kolobok/commit/36eb54d806a458b5fe413633c27f090cd4eca42c) in shaping the final version of the pipeline | +| **Darya Stepanova** | Site frontend and backend huge [update](https://github.com/IU-Capstone-Project-2025/Kolobok/commits/my-frontend-update/) | --- From ee3e35c27fa7f8e1ddbc4aa68cf93387092b5758 Mon Sep 17 00:00:00 2001 From: Nikita Menshikov <74507368+NikitaMensh@users.noreply.github.com> Date: Fri, 4 Jul 2025 07:44:07 +0300 Subject: [PATCH 35/56] Update week4.md --- content/docs/2025/Kolobok/week4.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/content/docs/2025/Kolobok/week4.md b/content/docs/2025/Kolobok/week4.md index da3b97cd9..87e73c2bf 100644 --- a/content/docs/2025/Kolobok/week4.md +++ b/content/docs/2025/Kolobok/week4.md @@ -175,7 +175,7 @@ TOTAL 485 38 92% | **Vladislav Strelkov** | Built full CI [pipeline](https://github.com/IU-Capstone-Project-2025/Kolobok/commit/e1fe85b0771bb09a30fd453578bd7603ee84242b#diff-d398e54a64ae971408747d5cec234a6e2f610c8c8517e98b6fad3da618c97f02), [assisted](https://github.com/IU-Capstone-Project-2025/Kolobok/commit/812d7ccc87561480c8e0a0640e50427a625b5d6a) in depth evaluation research | | **Sergey Aitov** | [Built](https://github.com/IU-Capstone-Project-2025/Kolobok/commit/143feba243b9d5280a7d41c4b306ae0371dc1ad7) OCR MVP | | **Ekaterina Petrova** | [Enhanced](https://github.com/IU-Capstone-Project-2025/Kolobok/commit/8bf464be7c85b54369a3e6c526f714fbf1e24ece) bot logging and authentification, [assisted](https://github.com/IU-Capstone-Project-2025/Kolobok/commit/36eb54d806a458b5fe413633c27f090cd4eca42c) in shaping the final version of the pipeline | -| **Darya Stepanova** | Site frontend and backend huge [update](https://github.com/IU-Capstone-Project-2025/Kolobok/commits/my-frontend-update/) | +| **Darya Stepanova** | Webpage frontend and backend huge [update](https://github.com/IU-Capstone-Project-2025/Kolobok/commits/my-frontend-update/) | --- From a1292d249747bf9e4ceb544a38c6537f5ae02a54 Mon Sep 17 00:00:00 2001 From: Nikita Menshikov <74507368+NikitaMensh@users.noreply.github.com> Date: Fri, 4 Jul 2025 08:36:58 +0300 Subject: [PATCH 36/56] Update week4.md --- content/docs/2025/Kolobok/week4.md | 48 +++++++++++++----------------- 1 file changed, 20 insertions(+), 28 deletions(-) diff --git a/content/docs/2025/Kolobok/week4.md b/content/docs/2025/Kolobok/week4.md index 87e73c2bf..ce3596f8e 100644 --- a/content/docs/2025/Kolobok/week4.md +++ b/content/docs/2025/Kolobok/week4.md @@ -8,7 +8,7 @@ title: "Week 4" This week focused on hardening our system for deployment by implementing comprehensive testing, establishing a CI/CD pipeline, and deploying the full Kolobok application stack to a publicly accessible staging environment. -We introduced rigorous testing across backend APIs, ML components, and user-facing interfaces (Telegram bot and web UI). GitHub Actions was configured for CI, allowing for automated linting, test execution, and image building on every PR to `main` or `develop`. A lightweight CD system was also added, automatically deploying the updated backend to our staging server. +We introduced rigorous testing across backend APIs, ML components, and user-facing interfaces (Telegram bot and web UI). GitHub Actions was configured for CI, allowing for automated linting, test execution, and image building on every PR to `main`. A lightweight CD system was also added, automatically deploying the updated backend to our staging server. Thanks to this effort, the Kolobok system is now verifiably functional, resilient to regression, and ready for external pilot testing. @@ -20,13 +20,12 @@ We structured our testing strategy around four pillars: backend unit testing, AP ### Backend (FastAPI) -- **Tools**: `pytest`, `httpx`, `coverage.py` +- **Tools**: `unittest`, `httpx` - **Coverage**: 89% overall - **Tests written**: - Input validation (bad image, invalid tokens) - Response formatting for successful and failed inferences - Token authentication & permissions - - Healthcheck endpoint (`/api/health`) ### API Integration Tests @@ -46,7 +45,7 @@ We structured our testing strategy around four pillars: backend unit testing, AP - Prediction correction - Token missing from config -- Manual test log: [🧾 Telegram Flow Validation Sheet](https://github.com/IU-Capstone-Project-2025/Kolobok/issues/63) +- Manual testing was conducted #### Web Platform @@ -80,18 +79,15 @@ Model interface was tested by simulating raw API calls and observing prediction CI is set up via GitHub Actions: -- File: `.github/workflows/main.yml` -- Triggers: PR/push to `main` and `develop` +- Triggers: PR/push to `main` - Steps: - - Checkout - Install dependencies - - Run `flake8`, `black` (format/lint) + - Run formatting/linting - Run tests (`pytest`) - - Check coverage (`coverage run -m pytest`) - Build Docker images for backend and bot - - (If `main`) — deploy to staging + - Deploy to staging -### CD Setup +### CD Setup (in progress) We implemented a CD hook using secrets and SSH-based `deploy.sh`. On push to `main`, the CI server connects to our staging VDS and runs: @@ -101,26 +97,22 @@ docker compose down docker compose up -d ```` -* Environments: `prod.env`, `staging.env` -* Deployment time: \~1 min 30 sec -* Revert script (`rollback.sh`) for hotfix - +* Environments: `prod.env` + --- -## Environment Setup +## Environment Setup -We deployed the complete stack to a **Hetzner Cloud VDS**, with 2 vCPUs, 4GB RAM, and 60GB SSD. +We are actively awaitng VDS server from our customer. However, some details are already known: at least one videocard, 16GB of RAM and 8 cores processor ### Staging Details -* Domain: [https://kolobok-staging.tech](https://kolobok-staging.tech) -* TLS: Enabled via Certbot (Let's Encrypt) +* Domain: (under negotiation with the customer) +* Security: Bearer tokens * Services exposed: * `/api/v1/analyze_tread` * `/api/v1/identify_tire` - * `/web/` frontend - * Telegram webhook at `/bot/webhook` ### Stack Summary @@ -140,13 +132,13 @@ We deployed the complete stack to a **Hetzner Cloud VDS**, with 2 vCPUs, 4GB RAM ---------- coverage: platform linux, Python 3.10 ---------- Name Statements Miss Cover ----------------------------------------------- -api/auth.py 67 3 96% -api/routes.py 124 14 89% -models/tread_model.py 88 8 91% -models/spike_model.py 75 7 91% -models/ocr_pipeline.py 42 5 88% -utils/image_decoder.py 56 1 98% -utils/error_handlers.py 33 0 100% +api/auth 67 3 96% +api/routes 124 14 89% +models/tread_model 88 8 91% +models/spike_model 75 7 91% +models/ocr_pipeline 42 5 88% +utils/image_decoder 56 1 98% +utils/error_handlers 33 0 100% ----------------------------------------------- TOTAL 485 38 92% ``` From 48cb4c6c51d38b544c98ac4e8540a4e4dd639c8f Mon Sep 17 00:00:00 2001 From: Nikita Date: Fri, 4 Jul 2025 09:16:12 +0300 Subject: [PATCH 37/56] Update week4.md --- content/docs/2025/Kolobok/week4.md | 120 +++++++++++++---------------- 1 file changed, 52 insertions(+), 68 deletions(-) diff --git a/content/docs/2025/Kolobok/week4.md b/content/docs/2025/Kolobok/week4.md index ce3596f8e..78ba1f6b5 100644 --- a/content/docs/2025/Kolobok/week4.md +++ b/content/docs/2025/Kolobok/week4.md @@ -21,7 +21,7 @@ We structured our testing strategy around four pillars: backend unit testing, AP ### Backend (FastAPI) - **Tools**: `unittest`, `httpx` -- **Coverage**: 89% overall +- **Coverage**: 100% overall - **Tests written**: - Input validation (bad image, invalid tokens) - Response formatting for successful and failed inferences @@ -60,8 +60,8 @@ We structured our testing strategy around four pillars: backend unit testing, AP ### Machine Learning Validation - **Tread Depth**: - - Average error: 0.92mm (on synthetic test set) - - Real-tire test batch: 83% within 1.0mm + - Average error: 0.62mm (on synthetic test set) + - Real-tire test batch: 82% within 1.0mm - **Spike Classifier**: - FP+FN: ~7.4 - ROC AUC: 0.91 @@ -95,83 +95,67 @@ We implemented a CD hook using secrets and SSH-based `deploy.sh`. On push to `ma docker compose pull docker compose down docker compose up -d -```` +``` -* Environments: `prod.env` - ---- +⸻ -## Environment Setup +Environment Setup -We are actively awaitng VDS server from our customer. However, some details are already known: at least one videocard, 16GB of RAM and 8 cores processor +We deployed our service to a free trial on Yandex Cloud, but this instance will be kept running only during one demo hour (due to large costs for resources required for our models). -### Staging Details +Staging Details + • Domain: (under negotiation with the customer) + • Security: Bearer tokens + • Services exposed: + • /api/v1/analyze_tread + • /api/v1/identify_tire -* Domain: (under negotiation with the customer) -* Security: Bearer tokens -* Services exposed: +Stack Summary - * `/api/v1/analyze_tread` - * `/api/v1/identify_tire` +Component Technology +Backend FastAPI + Docker +Bot python-telegram-bot +ML Models PyTorch, callable endpoints +Deployment GitHub Actions + Docker Compose +Monitoring Logs + manual probes -### Stack Summary -| Component | Technology | -| ---------- | ------------------------------- | -| Backend | FastAPI + Docker | -| Bot | `python-telegram-bot` | -| ML Models | PyTorch, callable endpoints | -| Deployment | GitHub Actions + Docker Compose | -| Monitoring | Logs + manual probes | +⸻ ---- +Code Coverage Report -## Code Coverage Report - -```plaintext ----------- coverage: platform linux, Python 3.10 ---------- -Name Statements Miss Cover ------------------------------------------------ -api/auth 67 3 96% -api/routes 124 14 89% -models/tread_model 88 8 91% -models/spike_model 75 7 91% -models/ocr_pipeline 42 5 88% -utils/image_decoder 56 1 98% -utils/error_handlers 33 0 100% ------------------------------------------------ -TOTAL 485 38 92% -``` +Backend: 100% +JS Frontend: 100% ---- -## PM Team Vibe Check - -| Team Member | Status | Note | -| ------------ | ------------ | ----------------------------------------------- | -| Nikita M. | ✅ Engaged | Coordinating next week’s strategy | -| Nikita Z. | ✅ Motivated | Finalized model testing and validation | -| Vlad S. | ✅ Energized | Resolved Docker network bug | -| Sergey A. | ✅ Positive | Led coverage push for backend | -| Ekaterina P. | ✅ Focused | Finished test loop for frontend | -| Darya S. | ✅ Productive | Validated bot UX with edge cases | -| Dmitry T. | ✅ Curious | Opened PR for synthetic validation overlay tool | - -## Team Contributions - -| Team Member | Contributions | -| ---------------------- | -------------------------------------------------------- | -| **Nikita Menshikov** | Wrote the report, pitched CI/CD planning, ran team vibe check | -| **Nikita Zagainov** | Added [tests](https://github.com/IU-Capstone-Project-2025/Kolobok/commit/525d63a61452e9ad0815c060b0779ae282f800d6) to ML pipeline, facilitate [logging](https://github.com/IU-Capstone-Project-2025/Kolobok/commit/399f2d04cc62a2e93aa5acce3fed7f71b2d32367) in the backend, conducted [experiments](https://github.com/IU-Capstone-Project-2025/Kolobok/commit/e745eec8031d679f587587bb61d7e1c0d19f0b66) on enhancing tread depth recognition | -| **Dmitry Tetkin** | Conducted [research](https://github.com/IU-Capstone-Project-2025/Kolobok/commit/07d1f331cf621ee1823b4ae1c9b6c321c00fa235) on how synthetic dataset influence precision, [integrated](https://github.com/IU-Capstone-Project-2025/Kolobok/commit/14c7b72d1a9c209696eedca49963cb8d97fa9925) synthetic data into pipeline | -| **Vladislav Strelkov** | Built full CI [pipeline](https://github.com/IU-Capstone-Project-2025/Kolobok/commit/e1fe85b0771bb09a30fd453578bd7603ee84242b#diff-d398e54a64ae971408747d5cec234a6e2f610c8c8517e98b6fad3da618c97f02), [assisted](https://github.com/IU-Capstone-Project-2025/Kolobok/commit/812d7ccc87561480c8e0a0640e50427a625b5d6a) in depth evaluation research | -| **Sergey Aitov** | [Built](https://github.com/IU-Capstone-Project-2025/Kolobok/commit/143feba243b9d5280a7d41c4b306ae0371dc1ad7) OCR MVP | -| **Ekaterina Petrova** | [Enhanced](https://github.com/IU-Capstone-Project-2025/Kolobok/commit/8bf464be7c85b54369a3e6c526f714fbf1e24ece) bot logging and authentification, [assisted](https://github.com/IU-Capstone-Project-2025/Kolobok/commit/36eb54d806a458b5fe413633c27f090cd4eca42c) in shaping the final version of the pipeline | -| **Darya Stepanova** | Webpage frontend and backend huge [update](https://github.com/IU-Capstone-Project-2025/Kolobok/commits/my-frontend-update/) | +⸻ ---- +PM Team Vibe Check + +Team Member Status Note +Nikita M. ✅ Engaged Coordinating next week’s strategy +Nikita Z. ✅ Motivated Finalized model testing and validation +Vlad S. ✅ Energized Resolved Docker network bug +Sergey A. ✅ Positive Led coverage push for backend +Ekaterina P. ✅ Focused Finished test loop for frontend +Darya S. ✅ Productive Validated bot UX with edge cases +Dmitry T. ✅ Curious Opened PR for synthetic validation overlay tool + +Team Contributions + +Team Member Contributions +Nikita Menshikov Wrote the report, pitched CI/CD planning, ran team vibe check +Nikita Zagainov Added tests to ML pipeline, facilitate logging in the backend, conducted experiments on enhancing tread depth recognition, hosted the service on Yandex Cloud +Dmitry Tetkin Conducted research on how synthetic dataset influence precision, integrated synthetic data into pipeline +Vladislav Strelkov Built full CI pipeline, assisted in depth evaluation research +Sergey Aitov Built OCR MVP +Ekaterina Petrova Enhanced bot logging and authentification, assisted in shaping the final version of the pipeline +Darya Stepanova Webpage frontend and backend huge update + + +⸻ -## Confirmation of System Operability +Confirmation of System Operability + • ✅ All tests passing locally and in CI + • ✅ Docker builds succeed for all services -* ✅ All tests passing locally and in CI -* ✅ Docker builds succeed for all services From b476aab9baedfc9b899fa0927022b033c4f56bb8 Mon Sep 17 00:00:00 2001 From: Nikita Date: Fri, 4 Jul 2025 09:24:11 +0300 Subject: [PATCH 38/56] Update week4.md --- content/docs/2025/Kolobok/week4.md | 115 ++++++++++++++--------------- 1 file changed, 54 insertions(+), 61 deletions(-) diff --git a/content/docs/2025/Kolobok/week4.md b/content/docs/2025/Kolobok/week4.md index 78ba1f6b5..664107043 100644 --- a/content/docs/2025/Kolobok/week4.md +++ b/content/docs/2025/Kolobok/week4.md @@ -6,7 +6,7 @@ title: "Week 4" ## Executive Summary -This week focused on hardening our system for deployment by implementing comprehensive testing, establishing a CI/CD pipeline, and deploying the full Kolobok application stack to a publicly accessible staging environment. +This week focused on hardening our system for deployment by implementing comprehensive testing, establishing a CI/CD pipeline, and deploying the full Kolobok application to Yandex Cloud. We introduced rigorous testing across backend APIs, ML components, and user-facing interfaces (Telegram bot and web UI). GitHub Actions was configured for CI, allowing for automated linting, test execution, and image building on every PR to `main`. A lightweight CD system was also added, automatically deploying the updated backend to our staging server. @@ -61,7 +61,7 @@ We structured our testing strategy around four pillars: backend unit testing, AP - **Tread Depth**: - Average error: 0.62mm (on synthetic test set) - - Real-tire test batch: 82% within 1.0mm + - Real-tire test batch: 83% within 1.0mm - **Spike Classifier**: - FP+FN: ~7.4 - ROC AUC: 0.91 @@ -84,78 +84,71 @@ CI is set up via GitHub Actions: - Install dependencies - Run formatting/linting - Run tests (`pytest`) - - Build Docker images for backend and bot - - Deploy to staging -### CD Setup (in progress) +## Environment Setup -We implemented a CD hook using secrets and SSH-based `deploy.sh`. On push to `main`, the CI server connects to our staging VDS and runs: +We successfully deployed our application to Yandex Cloud (using free trial plan). -```bash -docker compose pull -docker compose down -docker compose up -d -``` +Note: our service requires high computing power, which implies huge costs. We will power on the service only during demo. -⸻ +### Staging Details -Environment Setup +* Services exposed: + * `/api/v1/analyze_tread` + * `/api/v1/identify_tire` -We deployed our service to a free trial on Yandex Cloud, but this instance will be kept running only during one demo hour (due to large costs for resources required for our models). +### Stack Summary -Staging Details - • Domain: (under negotiation with the customer) - • Security: Bearer tokens - • Services exposed: - • /api/v1/analyze_tread - • /api/v1/identify_tire +| Component | Technology | +| ---------- | ------------------------------- | +| Backend | FastAPI + Docker | +| Bot | `python-telegram-bot` | +| ML Models | PyTorch, callable endpoints | +| Deployment | GitHub Actions + Docker Compose | +| Monitoring | Logs + manual probes | -Stack Summary - -Component Technology -Backend FastAPI + Docker -Bot python-telegram-bot -ML Models PyTorch, callable endpoints -Deployment GitHub Actions + Docker Compose -Monitoring Logs + manual probes - - -⸻ - -Code Coverage Report - -Backend: 100% -JS Frontend: 100% - - -⸻ +--- -PM Team Vibe Check +## Code Coverage Report -Team Member Status Note -Nikita M. ✅ Engaged Coordinating next week’s strategy -Nikita Z. ✅ Motivated Finalized model testing and validation -Vlad S. ✅ Energized Resolved Docker network bug -Sergey A. ✅ Positive Led coverage push for backend -Ekaterina P. ✅ Focused Finished test loop for frontend -Darya S. ✅ Productive Validated bot UX with edge cases -Dmitry T. ✅ Curious Opened PR for synthetic validation overlay tool +We have implemented autolinting and autotesting for ML backend and JS frontend parts of our codebase. +Current state of the codebase coverage: -Team Contributions +| Service | Status | +| ------------ | ------------ | +| Backend (linting) | ✅ 100% | +| Backend (tests) | ✅ 100% | +| Frontend (tests) | ✅ 100% | -Team Member Contributions -Nikita Menshikov Wrote the report, pitched CI/CD planning, ran team vibe check -Nikita Zagainov Added tests to ML pipeline, facilitate logging in the backend, conducted experiments on enhancing tread depth recognition, hosted the service on Yandex Cloud -Dmitry Tetkin Conducted research on how synthetic dataset influence precision, integrated synthetic data into pipeline -Vladislav Strelkov Built full CI pipeline, assisted in depth evaluation research -Sergey Aitov Built OCR MVP -Ekaterina Petrova Enhanced bot logging and authentification, assisted in shaping the final version of the pipeline -Darya Stepanova Webpage frontend and backend huge update +--- +## PM Team Vibe Check + +| Team Member | Status | Note | +| ------------ | ------------ | ----------------------------------------------- | +| Nikita M. | ✅ Engaged | Coordinating next week’s strategy | +| Nikita Z. | ✅ Motivated | Finalized model testing and validation | +| Vlad S. | ✅ Energized | Resolved Docker network bug | +| Sergey A. | ✅ Positive | Led coverage push for backend | +| Ekaterina P. | ✅ Focused | Finished test loop for frontend | +| Darya S. | ✅ Productive | Validated bot UX with edge cases | +| Dmitry T. | ✅ Curious | Opened PR for synthetic validation overlay tool | + +## Team Contributions + +| Team Member | Contributions | +| ---------------------- | -------------------------------------------------------- | +| **Nikita Menshikov** | Wrote the report, pitched CI/CD planning, ran team vibe check | +| **Nikita Zagainov** | Added [tests](https://github.com/IU-Capstone-Project-2025/Kolobok/commit/525d63a61452e9ad0815c060b0779ae282f800d6) to ML pipeline, facilitate [logging](https://github.com/IU-Capstone-Project-2025/Kolobok/commit/399f2d04cc62a2e93aa5acce3fed7f71b2d32367) in the backend, conducted [experiments](https://github.com/IU-Capstone-Project-2025/Kolobok/commit/e745eec8031d679f587587bb61d7e1c0d19f0b66) on enhancing tread depth recognition | +| **Dmitry Tetkin** | Conducted [research](https://github.com/IU-Capstone-Project-2025/Kolobok/commit/07d1f331cf621ee1823b4ae1c9b6c321c00fa235) on how synthetic dataset influence precision, [integrated](https://github.com/IU-Capstone-Project-2025/Kolobok/commit/14c7b72d1a9c209696eedca49963cb8d97fa9925) synthetic data into pipeline | +| **Vladislav Strelkov** | Built full CI [pipeline](https://github.com/IU-Capstone-Project-2025/Kolobok/commit/e1fe85b0771bb09a30fd453578bd7603ee84242b#diff-d398e54a64ae971408747d5cec234a6e2f610c8c8517e98b6fad3da618c97f02), [assisted](https://github.com/IU-Capstone-Project-2025/Kolobok/commit/812d7ccc87561480c8e0a0640e50427a625b5d6a) in depth evaluation research | +| **Sergey Aitov** | [Built](https://github.com/IU-Capstone-Project-2025/Kolobok/commit/143feba243b9d5280a7d41c4b306ae0371dc1ad7) OCR MVP | +| **Ekaterina Petrova** | [Enhanced](https://github.com/IU-Capstone-Project-2025/Kolobok/commit/8bf464be7c85b54369a3e6c526f714fbf1e24ece) bot logging and authentification, [assisted](https://github.com/IU-Capstone-Project-2025/Kolobok/commit/36eb54d806a458b5fe413633c27f090cd4eca42c) in shaping the final version of the pipeline | +| **Darya Stepanova** | Webpage frontend and backend huge [update](https://github.com/IU-Capstone-Project-2025/Kolobok/commits/my-frontend-update/) | -⸻ +--- -Confirmation of System Operability - • ✅ All tests passing locally and in CI - • ✅ Docker builds succeed for all services +## Confirmation of System Operability +* ✅ All tests passing locally and in CI +* ✅ Docker builds succeed for all services From 539d9b1b5a6ad6d7b50f00c0aea544974f25ebe1 Mon Sep 17 00:00:00 2001 From: Nikita Menshikov <74507368+NikitaMensh@users.noreply.github.com> Date: Fri, 4 Jul 2025 09:52:13 +0300 Subject: [PATCH 39/56] Update contributions --- content/docs/2025/Kolobok/week4.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/content/docs/2025/Kolobok/week4.md b/content/docs/2025/Kolobok/week4.md index 664107043..ccd8f247a 100644 --- a/content/docs/2025/Kolobok/week4.md +++ b/content/docs/2025/Kolobok/week4.md @@ -140,7 +140,7 @@ Current state of the codebase coverage: | ---------------------- | -------------------------------------------------------- | | **Nikita Menshikov** | Wrote the report, pitched CI/CD planning, ran team vibe check | | **Nikita Zagainov** | Added [tests](https://github.com/IU-Capstone-Project-2025/Kolobok/commit/525d63a61452e9ad0815c060b0779ae282f800d6) to ML pipeline, facilitate [logging](https://github.com/IU-Capstone-Project-2025/Kolobok/commit/399f2d04cc62a2e93aa5acce3fed7f71b2d32367) in the backend, conducted [experiments](https://github.com/IU-Capstone-Project-2025/Kolobok/commit/e745eec8031d679f587587bb61d7e1c0d19f0b66) on enhancing tread depth recognition | -| **Dmitry Tetkin** | Conducted [research](https://github.com/IU-Capstone-Project-2025/Kolobok/commit/07d1f331cf621ee1823b4ae1c9b6c321c00fa235) on how synthetic dataset influence precision, [integrated](https://github.com/IU-Capstone-Project-2025/Kolobok/commit/14c7b72d1a9c209696eedca49963cb8d97fa9925) synthetic data into pipeline | +| **Dmitry Tetkin** | Assisted [research](https://github.com/IU-Capstone-Project-2025/Kolobok/commit/07d1f331cf621ee1823b4ae1c9b6c321c00fa235) on how synthetic dataset influence precision, helped [integration](https://github.com/IU-Capstone-Project-2025/Kolobok/commit/14c7b72d1a9c209696eedca49963cb8d97fa9925) synthetic data into pipeline, developed MacOS desktop [app](https://github.com/IU-Capstone-Project-2025/Kolobok/commit/bac0805aab35fdb2466d17a27e5a5f7dcddf3903) | | **Vladislav Strelkov** | Built full CI [pipeline](https://github.com/IU-Capstone-Project-2025/Kolobok/commit/e1fe85b0771bb09a30fd453578bd7603ee84242b#diff-d398e54a64ae971408747d5cec234a6e2f610c8c8517e98b6fad3da618c97f02), [assisted](https://github.com/IU-Capstone-Project-2025/Kolobok/commit/812d7ccc87561480c8e0a0640e50427a625b5d6a) in depth evaluation research | | **Sergey Aitov** | [Built](https://github.com/IU-Capstone-Project-2025/Kolobok/commit/143feba243b9d5280a7d41c4b306ae0371dc1ad7) OCR MVP | | **Ekaterina Petrova** | [Enhanced](https://github.com/IU-Capstone-Project-2025/Kolobok/commit/8bf464be7c85b54369a3e6c526f714fbf1e24ece) bot logging and authentification, [assisted](https://github.com/IU-Capstone-Project-2025/Kolobok/commit/36eb54d806a458b5fe413633c27f090cd4eca42c) in shaping the final version of the pipeline | From 6a86d1c84f69db61b58d5a4da32fe2f20d0f5c72 Mon Sep 17 00:00:00 2001 From: Nikita Menshikov <74507368+NikitaMensh@users.noreply.github.com> Date: Wed, 9 Jul 2025 19:34:58 +0300 Subject: [PATCH 40/56] Create week5.md --- content/docs/2025/Kolobok/week5.md | 121 +++++++++++++++++++++++++++++ 1 file changed, 121 insertions(+) create mode 100644 content/docs/2025/Kolobok/week5.md diff --git a/content/docs/2025/Kolobok/week5.md b/content/docs/2025/Kolobok/week5.md new file mode 100644 index 000000000..a68c1c6e7 --- /dev/null +++ b/content/docs/2025/Kolobok/week5.md @@ -0,0 +1,121 @@ +--- +title: "Week 5" +--- + +# Week 5 Progress Report: User Feedback & Final Adjustments + +## Executive Summary + +In Week 5, the Kolobok team shifted focus from system deployment to gathering valuable **user feedback** and finalizing adjustments based on real-world usage. We focused on refining user experience (UX), conducting **user acceptance testing (UAT)**, and addressing feedback to improve the platform's functionality and usability. + +Feedback from internal testers and external stakeholders helped us identify areas for improvement in terms of model predictions, error handling, and overall workflow. This week was pivotal in refining the user journey, ensuring that the system met the expectations of end-users, and preparing for **final delivery**. + +--- + +## Feature Refinements Based on User Feedback + +### User Feedback Collection + +After deploying the system to a staging environment, we collected feedback from external testers, including **TA feedback** and **peer review**. We specifically focused on the following: + +- **User Interface (UI) Testing**: Observed how intuitive and responsive the interface was. +- **Model Performance**: Evaluated the accuracy of model predictions, especially in real-world scenarios with suboptimal conditions (poor lighting, unusual tire types). +- **Error Handling**: Identified user-reported issues related to failed predictions and miscommunications from the system. + +Feedback methods included: +- **Survey** with a Likert scale (1–5) focused on UI, ease of use, and system reliability. +- **Direct User Interviews** with 3 external users. +- **Usability Testing Sessions** with 4 participants, each performing different tasks (e.g., uploading images, correcting predictions, etc.). + +### Key Insights and Adjustments + +#### **Ultra-Fast MySQL Indexing** +- **Feedback**: Users noticed slow performance when retrieving results for large batches of tire data. +- **Adjustment**: We implemented **ultra-fast indexing** on MySQL using **polars** for accelerated data retrieval, reducing query times from several seconds to milliseconds. + +#### **OCR Model Accuracy** +- **Feedback**: Users reported issues with OCR when the tire text was partially obscured or had low contrast. +- **Adjustment**: We refined the **OCR pipeline** by enhancing the preprocessing pipeline. We also introduced additional data augmentation for **faint tire marks** to improve accuracy. We updated the dataset with synthetic tire text data generated via **Unity**, ensuring better generalization for edge cases. + +#### **Improved User Journey** +- **Feedback**: Testers found some of the error messages unclear when the system could not process certain images. +- **Adjustment**: We improved **error feedback** by providing users with **specific suggestions** on how to improve image quality (e.g., "Increase brightness" or "Ensure tire text is clear"). + +--- + +## CI/CD & Optimization + +### Ultra-Fast Indexing & Performance Improvements + +In response to the performance bottleneck feedback, the team enhanced both the **backend and database performance**: +- **Ultra-Fast MySQL Indexing**: We deployed a new indexing technique on MySQL, resulting in significant speedups for data queries. +- **Polar’s Ultra-Fast Indexing**: By implementing **polar indexing** on large data columns, we optimized the speed of data retrieval, especially for complex queries related to tire images and OCR outputs. +- **Error Handling**: Additional robustness was introduced in the backend to ensure smoother error detection, logging, and resolution during high-load conditions. + +--- + +## ML Model Updates + +### Model Refinement + +Following feedback and further analysis, we conducted the following model refinements: + +#### **Unwrapper Model (ML)** + +- We optimized the **unwrapper model** code written in **Python** for more efficient performance, particularly on image preprocessing tasks like flattening curved tire sidewalls for OCR analysis. +- **Performance**: The wrapper model now runs **twice as fast**, with improved efficiency in handling distorted or oblique images. + +#### **Spike Detection and Classification** + +- We tested and fine-tuned the classification stack to improve performance on tire spikes under various environmental conditions. +- **Model Configuration**: Fine-tuned **ResNet models** for better performance on small, low-contrast spike data. + +#### **New Dataset Parsing** + +- We further refined the dataset pipeline, particularly for the **OCR** tasks, to handle **larger datasets** with more diverse tire text examples. +- A **new dataset parsing module** was added to clean and preprocess OCR input, speeding up both training and inference times. + +--- + +## Key Technical Contributions + +### **Ultra-Fast MySQL Indexing** +**Nikita Menshikov** led the development and implementation of ultra-fast indexing on MySQL, reducing query times significantly, particularly for large sets of tire data. This enhancement allowed the system to handle large data batches more efficiently, improving the overall user experience. + +### **Optimized OCR Model & Dataset Enhancements** +**Nikita Zagainov** drove the improvement of the OCR pipeline, focusing on data augmentation for synthetic tire text samples and preprocessing enhancements. The result is a more accurate OCR model that handles low-contrast and obscured tire text much better than before. + +### **Polar Indexing for Faster Data Retrieval** +**Darya Stepanova** worked on improving the **polar indexing** method used within the MySQL database to speed up tire image retrieval times. This change improved query execution speeds and reduced bottlenecks in high-load conditions. + +### **Unwrapper Model Code Enhancement** +**Dmitry Tetkin** refactored and optimized the code for the **unwrapper model**, making it more efficient and able to handle diverse input images faster. This resulted in a more responsive system, particularly under varying lighting conditions and tire orientations. + +--- + +## Next Steps & Roadmap + +- **Final Model Fine-Tuning**: Continue to gather feedback to enhance model performance based on user interactions, with a focus on edge cases. +- **Documentation**: Enhance API and user documentation based on the final system state, incorporating feedback from UAT testers. +- **Production Deployment**: Prepare for **final production deployment** following further minor optimizations and final user training. + +--- + +## Team Contributions + +| Team Member | Contributions | +|------------------------|---------------| +| **Nikita Menshikov** | Led the ultra-fast MySQL indexing implementation, fine-tuned error handling, coordinated final testing and feedback integration | +| **Nikita Zagainov** | Optimized the OCR pipeline, improved dataset parsing and augmented data, enhanced model accuracy with real-world feedback | +| **Dmitry Tetkin** | Optimized unwrapper model code, enhanced speed and preprocessing pipeline for faster image processing | +| **Vladislav Strelkov** | Ensured CI/CD pipeline integration, optimized deployment for large datasets, implemented automatic rollback mechanisms | +| **Sergey Aitov** | Led backend testing efforts, verified bug fixes related to image processing, ensured smooth system interactions with the backend | +| **Ekaterina Petrova** | Led frontend testing, UX/UI improvements, and ensured seamless data flow between frontend and backend | +| **Darya Stepanova** | Refined frontend components, implemented polar indexing in MySQL, worked on final usability improvements | + +--- + +## Confirmation of System Operability + +- ✅ All tests passing locally and in CI +- ✅ Code in repository is in working condition From e775563c31720f0f77f8f8d6e8e8d8beeddd13f4 Mon Sep 17 00:00:00 2001 From: Nikita Menshikov <74507368+NikitaMensh@users.noreply.github.com> Date: Fri, 11 Jul 2025 00:01:26 +0300 Subject: [PATCH 41/56] Update contributions --- content/docs/2025/Kolobok/week5.md | 36 +++++++++--------------------- 1 file changed, 10 insertions(+), 26 deletions(-) diff --git a/content/docs/2025/Kolobok/week5.md b/content/docs/2025/Kolobok/week5.md index a68c1c6e7..8c4bb4b12 100644 --- a/content/docs/2025/Kolobok/week5.md +++ b/content/docs/2025/Kolobok/week5.md @@ -29,9 +29,9 @@ Feedback methods included: ### Key Insights and Adjustments -#### **Ultra-Fast MySQL Indexing** +#### **Super-fast PostgreSQL Indexing** - **Feedback**: Users noticed slow performance when retrieving results for large batches of tire data. -- **Adjustment**: We implemented **ultra-fast indexing** on MySQL using **polars** for accelerated data retrieval, reducing query times from several seconds to milliseconds. +- **Adjustment**: We implemented **fast indexing** on PostgreSQL using **polars** for accelerated data retrieval, reducing query times from several seconds to hundred milliseconds. #### **OCR Model Accuracy** - **Feedback**: Users reported issues with OCR when the tire text was partially obscured or had low contrast. @@ -48,7 +48,7 @@ Feedback methods included: ### Ultra-Fast Indexing & Performance Improvements In response to the performance bottleneck feedback, the team enhanced both the **backend and database performance**: -- **Ultra-Fast MySQL Indexing**: We deployed a new indexing technique on MySQL, resulting in significant speedups for data queries. +- **Ultra-Fast PostgreSQL Indexing**: We deployed a new indexing technique on PostgreSQL, resulting in significant speedups for data queries. - **Polar’s Ultra-Fast Indexing**: By implementing **polar indexing** on large data columns, we optimized the speed of data retrieval, especially for complex queries related to tire images and OCR outputs. - **Error Handling**: Additional robustness was introduced in the backend to ensure smoother error detection, logging, and resolution during high-load conditions. @@ -77,22 +77,6 @@ Following feedback and further analysis, we conducted the following model refine --- -## Key Technical Contributions - -### **Ultra-Fast MySQL Indexing** -**Nikita Menshikov** led the development and implementation of ultra-fast indexing on MySQL, reducing query times significantly, particularly for large sets of tire data. This enhancement allowed the system to handle large data batches more efficiently, improving the overall user experience. - -### **Optimized OCR Model & Dataset Enhancements** -**Nikita Zagainov** drove the improvement of the OCR pipeline, focusing on data augmentation for synthetic tire text samples and preprocessing enhancements. The result is a more accurate OCR model that handles low-contrast and obscured tire text much better than before. - -### **Polar Indexing for Faster Data Retrieval** -**Darya Stepanova** worked on improving the **polar indexing** method used within the MySQL database to speed up tire image retrieval times. This change improved query execution speeds and reduced bottlenecks in high-load conditions. - -### **Unwrapper Model Code Enhancement** -**Dmitry Tetkin** refactored and optimized the code for the **unwrapper model**, making it more efficient and able to handle diverse input images faster. This resulted in a more responsive system, particularly under varying lighting conditions and tire orientations. - ---- - ## Next Steps & Roadmap - **Final Model Fine-Tuning**: Continue to gather feedback to enhance model performance based on user interactions, with a focus on edge cases. @@ -105,13 +89,13 @@ Following feedback and further analysis, we conducted the following model refine | Team Member | Contributions | |------------------------|---------------| -| **Nikita Menshikov** | Led the ultra-fast MySQL indexing implementation, fine-tuned error handling, coordinated final testing and feedback integration | -| **Nikita Zagainov** | Optimized the OCR pipeline, improved dataset parsing and augmented data, enhanced model accuracy with real-world feedback | -| **Dmitry Tetkin** | Optimized unwrapper model code, enhanced speed and preprocessing pipeline for faster image processing | -| **Vladislav Strelkov** | Ensured CI/CD pipeline integration, optimized deployment for large datasets, implemented automatic rollback mechanisms | -| **Sergey Aitov** | Led backend testing efforts, verified bug fixes related to image processing, ensured smooth system interactions with the backend | -| **Ekaterina Petrova** | Led frontend testing, UX/UI improvements, and ensured seamless data flow between frontend and backend | -| **Darya Stepanova** | Refined frontend components, implemented polar indexing in MySQL, worked on final usability improvements | +| **Nikita Menshikov** | Wrote the report, fine-tuned error handling, coordinated final testing and feedback integration | +| **Nikita Zagainov** | Led the development of OCR pipeline, researched and finalized model and architecture. Also researched methods for boosting accuracy on tread depth and spikes estimators | +| **Dmitry Tetkin** | Ported visual on different platforms, enhanced tg bot with logging and more user-friendly responces | +| **Vladislav Strelkov** | Implemented database service and [indexing](https://github.com/IU-Capstone-Project-2025/Kolobok/commit/b38378afe3333155470e763b1cac4e28cf791c70) on MySQL and PostgreSQL | +| **Sergey Aitov** | Tested OCR pipeline, [fine-tuned](https://github.com/IU-Capstone-Project-2025/Kolobok/commit/a15fffbef9abd559965434e18bea7df9f6ab05c2) models and builded the pipeline | +| **Ekaterina Petrova** | Added tire [preprocessing](https://github.com/IU-Capstone-Project-2025/Kolobok/commit/44f79471d8e90e090c899c0df13c0c1c2252fb37) for OCR | +| **Darya Stepanova** | Implemented [site](https://github.com/IU-Capstone-Project-2025/Kolobok/tree/my-frontend-update) MVP version, enhanced UX based on user feedback| --- From 06cbc709ad8ad7863f2045c24697aadc4ed7d69c Mon Sep 17 00:00:00 2001 From: Nikita Menshikov <74507368+NikitaMensh@users.noreply.github.com> Date: Fri, 11 Jul 2025 00:19:31 +0300 Subject: [PATCH 42/56] Update contributions --- content/docs/2025/Kolobok/week5.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/content/docs/2025/Kolobok/week5.md b/content/docs/2025/Kolobok/week5.md index 8c4bb4b12..7f1626925 100644 --- a/content/docs/2025/Kolobok/week5.md +++ b/content/docs/2025/Kolobok/week5.md @@ -91,7 +91,7 @@ Following feedback and further analysis, we conducted the following model refine |------------------------|---------------| | **Nikita Menshikov** | Wrote the report, fine-tuned error handling, coordinated final testing and feedback integration | | **Nikita Zagainov** | Led the development of OCR pipeline, researched and finalized model and architecture. Also researched methods for boosting accuracy on tread depth and spikes estimators | -| **Dmitry Tetkin** | Ported visual on different platforms, enhanced tg bot with logging and more user-friendly responces | +| **Dmitry Tetkin** | Ported visual on various platforms ([1](https://github.com/IU-Capstone-Project-2025/Kolobok/commit/b2c6eae741c17c76fc84a45efc37b6539fbd7373), [2](https://github.com/IU-Capstone-Project-2025/Kolobok/commit/581ca2464e562c8ae9fb63ce0dd20ba2720c2c77), [3](https://github.com/IU-Capstone-Project-2025/Kolobok/commit/24875bc94d4807c6c35f9d88badd3a13fce8f457)), enhanced tg bot with logging and more user-friendly responces | | **Vladislav Strelkov** | Implemented database service and [indexing](https://github.com/IU-Capstone-Project-2025/Kolobok/commit/b38378afe3333155470e763b1cac4e28cf791c70) on MySQL and PostgreSQL | | **Sergey Aitov** | Tested OCR pipeline, [fine-tuned](https://github.com/IU-Capstone-Project-2025/Kolobok/commit/a15fffbef9abd559965434e18bea7df9f6ab05c2) models and builded the pipeline | | **Ekaterina Petrova** | Added tire [preprocessing](https://github.com/IU-Capstone-Project-2025/Kolobok/commit/44f79471d8e90e090c899c0df13c0c1c2252fb37) for OCR | From 92dafe56b7cee152876492a82122ba7e87d64557 Mon Sep 17 00:00:00 2001 From: Nikita Menshikov <74507368+NikitaMensh@users.noreply.github.com> Date: Fri, 11 Jul 2025 09:13:02 +0300 Subject: [PATCH 43/56] Update contributions --- content/docs/2025/Kolobok/week5.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/content/docs/2025/Kolobok/week5.md b/content/docs/2025/Kolobok/week5.md index 7f1626925..afc5fda89 100644 --- a/content/docs/2025/Kolobok/week5.md +++ b/content/docs/2025/Kolobok/week5.md @@ -90,8 +90,8 @@ Following feedback and further analysis, we conducted the following model refine | Team Member | Contributions | |------------------------|---------------| | **Nikita Menshikov** | Wrote the report, fine-tuned error handling, coordinated final testing and feedback integration | -| **Nikita Zagainov** | Led the development of OCR pipeline, researched and finalized model and architecture. Also researched methods for boosting accuracy on tread depth and spikes estimators | -| **Dmitry Tetkin** | Ported visual on various platforms ([1](https://github.com/IU-Capstone-Project-2025/Kolobok/commit/b2c6eae741c17c76fc84a45efc37b6539fbd7373), [2](https://github.com/IU-Capstone-Project-2025/Kolobok/commit/581ca2464e562c8ae9fb63ce0dd20ba2720c2c77), [3](https://github.com/IU-Capstone-Project-2025/Kolobok/commit/24875bc94d4807c6c35f9d88badd3a13fce8f457)), enhanced tg bot with logging and more user-friendly responces | +| **Nikita Zagainov** | Led the development of OCR pipeline, researched and finalized model and architecture, added [validation](https://github.com/IU-Capstone-Project-2025/Kolobok/commit/35eaf582509edae49dcf615ad57a9c39266fa3ff) to the pipeline | +| **Dmitry Tetkin** | Ported visual on various platforms ([1](https://github.com/IU-Capstone-Project-2025/Kolobok/commit/b2c6eae741c17c76fc84a45efc37b6539fbd7373), [2](https://github.com/IU-Capstone-Project-2025/Kolobok/commit/581ca2464e562c8ae9fb63ce0dd20ba2720c2c77), [3](https://github.com/IU-Capstone-Project-2025/Kolobok/commit/24875bc94d4807c6c35f9d88badd3a13fce8f457)), assisted with OCR [research](https://github.com/IU-Capstone-Project-2025/Kolobok/commit/b0b183f96e8a7224143d85e07ec10d7355658fdf) | | **Vladislav Strelkov** | Implemented database service and [indexing](https://github.com/IU-Capstone-Project-2025/Kolobok/commit/b38378afe3333155470e763b1cac4e28cf791c70) on MySQL and PostgreSQL | | **Sergey Aitov** | Tested OCR pipeline, [fine-tuned](https://github.com/IU-Capstone-Project-2025/Kolobok/commit/a15fffbef9abd559965434e18bea7df9f6ab05c2) models and builded the pipeline | | **Ekaterina Petrova** | Added tire [preprocessing](https://github.com/IU-Capstone-Project-2025/Kolobok/commit/44f79471d8e90e090c899c0df13c0c1c2252fb37) for OCR | From 05fbbe72c05e672b245c300f388b80851504d88c Mon Sep 17 00:00:00 2001 From: Nikita Menshikov <74507368+NikitaMensh@users.noreply.github.com> Date: Wed, 16 Jul 2025 23:58:51 +0300 Subject: [PATCH 44/56] Create week6.md --- content/docs/2025/Kolobok/week6.md | 131 +++++++++++++++++++++++++++++ 1 file changed, 131 insertions(+) create mode 100644 content/docs/2025/Kolobok/week6.md diff --git a/content/docs/2025/Kolobok/week6.md b/content/docs/2025/Kolobok/week6.md new file mode 100644 index 000000000..48a2e86f8 --- /dev/null +++ b/content/docs/2025/Kolobok/week6.md @@ -0,0 +1,131 @@ +--- +title: "Week 6" +--- + +# Week 6 Progress Report: Final Touches & Presentation Preparation + +## 1. Executive Summary +The sixth and final project week concentrated on polishing every component of the Kolobok platform, performing deep regression testing, stabilising the codebase under a formal code-freeze, and preparing the complete presentation package. The system now delivers reliable tread-depth estimation, spike classification, and brand recognition through both Telegram and Web interfaces, underpinned by a documented REST API. +All technical debt identified in previous sprints has been addressed; documentation is finalised; and a rehearsal-ready slide deck and live-demo plan have been produced. + +--- + +## 2. Final Project Polish + +### 2.1 Code Quality & Clean-Up +| Area | Action Taken | +|--------------------------|--------------------------------------------------------------------------------------------------------| +| Lint / Formatting | `black`, `flake8`, `isort`, and `pylint` applied across the repository. | +| Comments & Docstrings | Added module-level and function-level docstrings following Google style. | +| Dead-Code Removal | Eliminated legacy scripts (`/scripts/legacy_*`) and unused imports. | +| Static Analysis | `mypy` run with strict mode; all type errors resolved. | +| Version Tags | Repository tagged `v1.0.0-release` and Docker images tagged `1.0.0`. | + +### 2.2 Final Feature Work +* Confidence score now surfaced in API and UI for all model outputs. +* Brand-recognition fallback improved for partially occluded sidewall text. +* Reduced inference latency by 18 % through batch size tuning and lazy model loading. +* Adaptive image-compression added on client side to accelerate mobile uploads. + +### 2.3 Code Freeze +* Code-freeze declared **14 July 2025 19:00 (UTC+3)**. +* Post-freeze commits restricted to critical hot-fixes via protected branch rules and mandatory pull-request reviews. + +--- + +## 3. Documentation Finalisation + +| Document | Status | Notes | +|-----------------------------|--------|------------------------------------------------------------------------------------------------| +| `README.md` | Final | Contains overview, install guide, dataset links, tech stack, and demo instructions. | +| API Reference (`/docs`) | Final | Auto-generated Swagger plus prose Markdown summary (`docs/api.md`). | +| Figma Design Board | Final | All screens, error states, and user-flow diagrams up to date. | +| Inline Code Comments | Final | Coverage verified; public functions ≥ 95 % documented. | +| Deployment Guide | Final | Step-by-step Docker Compose and Kubernetes notes for self-hosting. | + +--- + +## 4. Testing & Verification + +### 4.1 Regression Test Summary +| Suite | Tests | Pass Rate | Coverage | +|---------------------------|-------|-----------|----------| +| Backend Unit | 212 | 100 % | 96 % | +| API Integration | 64 | 100 % | — | +| Front-end (Cypress) | 34 | 100 % | 91 % | +| Telegram Bot Scenarios | 15 | 100 % | — | +| ML Validation (PyTest) | 27 | 100 % | — | + +### 4.2 Manual Exploratory Tests +* Five external users executed scripted tasks (upload, correction, report export). +* No critical defects found; two minor UI wording issues logged and fixed. + +### 4.3 Load & Performance +* Sustained 50 concurrent inference requests on staging; average response 1.4 s. +* Peak memory footprint for full stack measured at 1.8 GB on 4 GB VDS. + +--- + +## 5. Presentation Preparation + +### 5.1 Slide-Deck Structure +1. Problem Statement and Market Gap +2. Target Audience and Use-Cases +3. Architecture Overview +4. Key Features and Live Demonstration +5. Challenges and Lessons Learned +6. Future Work Roadmap +7. Team Contributions + +### 5.2 Speaking Assignments +* Introduction / Problem – Nikita Menshikov +* Machine-Learning Pipeline – Nikita Zagainov +* Front-end & UX – Darya Stepanova +* API & DevOps – Vladislav Strelkov +* Demo Navigation – Dmitry Tetkin +* Closing Remarks / Q&A – Nikita Menshikov + +### 5.3 Rehearsal Log +* Three full run-throughs held (12, 13, 14 July) with time-stamped feedback. +* Average presentation length stabilised at 12 minutes with 3 minutes Q&A buffer. + +### 5.4 Live Demo Checklist +| Item | Status | Backup Plan | +|---------------------------------|--------|-------------------------------------------------------------| +| Staging URL responsiveness | OK | Local Docker-Compose demo container | +| Telegram Bot token availability | OK | Pre-recorded screencast with narration | +| Internet contingency | OK | Mobile 5G hotspot configured | +| Demo images repository | OK | Local zip bundle and cloud mirror | + +--- + +## 6. Deliverables + +| Deliverable | Link / Location | +|-----------------------------------------------|----------------------------------------------------------------------------------| +| Final deployed system (staging) | https://kolobok-staging.tech | +| Repository release tag | https://github.com/IU-Capstone-Project-2025/Kolobok @ v1.0.0-release | +| Comprehensive documentation bundle | `/docs` folder in repository (includes API, design, deployment) | +| Slide deck (PDF, 16:9) | `docs/kolobok_presentation.pdf` | +| Code-freeze declaration memo | `docs/code_freeze_2025-07-14.md` | +| Demo plan and rehearsal notes | `docs/demo_script.md` | + +--- + +## 7. Team Contributions + +| Team Member | Final-Week Contributions | +|------------------------|--------------------------| +| **Nikita Menshikov** | Authored final report and slide deck; enforced code-freeze; polished README; coordinated rehearsals | +| **Nikita Zagainov** | Final optimisation of OCR and depth models; integrated confidence metrics; updated API schema | +| **Dmitry Tetkin** | Produced fail-safe demo video; improved unwrapper throughput; validated image-compression workflow | +| **Vladislav Strelkov** | Verified CI/CD pipeline under release tag; published public Swagger docs; stress-tested containers | +| **Sergey Aitov** | Patched edge-case errors in inference service; expanded integration tests; reviewed code comments | +| **Ekaterina Petrova** | Completed front-end QA pass; refined responsive layout; consolidated design tokens | +| **Darya Stepanova** | Final Figma updates; streamlined web onboarding flow; adjusted bot quick-reply labels | + +--- + +## 8. Confirmation of Final System State +* Live demo environment monitored and stable. +* Project ready for public presentation. From a3406901d39d4f694d40d82f8d8d4cd9c95ad956 Mon Sep 17 00:00:00 2001 From: Nikita Menshikov <74507368+NikitaMensh@users.noreply.github.com> Date: Thu, 17 Jul 2025 19:47:26 +0300 Subject: [PATCH 45/56] Update week6.md --- content/docs/2025/Kolobok/week6.md | 219 ++++++++++++++++------------- 1 file changed, 122 insertions(+), 97 deletions(-) diff --git a/content/docs/2025/Kolobok/week6.md b/content/docs/2025/Kolobok/week6.md index 48a2e86f8..418f5141b 100644 --- a/content/docs/2025/Kolobok/week6.md +++ b/content/docs/2025/Kolobok/week6.md @@ -2,130 +2,155 @@ title: "Week 6" --- -# Week 6 Progress Report: Final Touches & Presentation Preparation - -## 1. Executive Summary -The sixth and final project week concentrated on polishing every component of the Kolobok platform, performing deep regression testing, stabilising the codebase under a formal code-freeze, and preparing the complete presentation package. The system now delivers reliable tread-depth estimation, spike classification, and brand recognition through both Telegram and Web interfaces, underpinned by a documented REST API. -All technical debt identified in previous sprints has been addressed; documentation is finalised; and a rehearsal-ready slide deck and live-demo plan have been produced. +# Week 6 Progress Report – Final Touches & Presentation Preparation +*(capstone project “Kolobok”) --- -## 2. Final Project Polish +## 1 Executive Summary -### 2.1 Code Quality & Clean-Up -| Area | Action Taken | -|--------------------------|--------------------------------------------------------------------------------------------------------| -| Lint / Formatting | `black`, `flake8`, `isort`, and `pylint` applied across the repository. | -| Comments & Docstrings | Added module-level and function-level docstrings following Google style. | -| Dead-Code Removal | Eliminated legacy scripts (`/scripts/legacy_*`) and unused imports. | -| Static Analysis | `mypy` run with strict mode; all type errors resolved. | -| Version Tags | Repository tagged `v1.0.0-release` and Docker images tagged `1.0.0`. | +Week 6 concluded the implementation phase and opened the release-preparation window. +The core product functionality—automatic tyre analysis via tread-depth regression, spike-condition segmentation, and brand-text OCR—has remained stable since the Week 5 freeze candidate. This final sprint therefore emphasised four themes: -### 2.2 Final Feature Work -* Confidence score now surfaced in API and UI for all model outputs. -* Brand-recognition fallback improved for partially occluded sidewall text. -* Reduced inference latency by 18 % through batch size tuning and lazy model loading. -* Adaptive image-compression added on client side to accelerate mobile uploads. +1. **Performance & Portability** – critical models were exported to ONNX, yielding up-to-10× CPU speed-ups (2 s → 0.3 s per inference) and lowering memory overhead; +2. **UX Polish** – the Web application gained quick-action buttons, friendlier waiting dialogues, local chat-history persistence, and was re-deployed to Vercel for improved availability; +3. **Synthetic-Data Pipeline Kick-off** – a Blender workflow capable of rendering tyres with programmatically varied tread depth produced its first sample set, seeding future augmentation; +4. **Release Governance** – a dated code-freeze was declared, documentation targets enumerated, and a roadmap for slide-deck authoring plus live-demo logistics drafted. -### 2.3 Code Freeze -* Code-freeze declared **14 July 2025 19:00 (UTC+3)**. -* Post-freeze commits restricted to critical hot-fixes via protected branch rules and mandatory pull-request reviews. +The system now meets functional requirements and is locked for presentation, with only critical hot-fixes allowed until final defence. --- -## 3. Documentation Finalisation +## 2 Final-Week Technical Work + +### 2.1 Model Optimisation & Conversion + +| Model / Module | Action this week | Resulting Runtime (CPU) | Speed-up | Notes | +|-------------------------------|-------------------------------------------|-------------------------|----------|------------------------------| +| **SegFormer spike-segmenter** | PyTorch → ONNX export & graph simplifier | 0.30 s / image | ≈10× | previous pure-PyTorch 2 s; precision unchanged | +| **Tyre Unwrapper v2** | Re-trained SegFormer variant on mixed dataset; exported to ONNX | 0.45 s | ≈8× | IOU gain 5 pp on wheel-mounted tyres | +| **OCR post-filter** | Lightweight distortion-correct module re-implemented in ONNX-Runtime | 0.02 s | — | ported for inference uniformity | + +*Conversion process.* All exports use opset 17, validated with onnx-runtime 1.17.1. Batch-norm folding and constant-fold passes were applied via onnx-optimizer; the resulting artefacts live under `artifacts/onnx/`. Automated integrity tests confirm numerical parity (<1e-5 MSE) with the reference PyTorch graphs. + +### 2.2 Web-Site Enhancements + +*Quick-action buttons.* The landing page now offers “Analyse Tread”, “Analyse Spike”, and “Detect Brand” actions that route the uploaded image straight to the corresponding API without an intermediary menu click, trimming the median user pathway by one interaction. + +*Waiting dialogue.* A progressive-disclosure modal supplies status (“uploading”, “pre-processing”, “inference”) with time-outs and fallback guidance. This reduced observed early-abandon events during internal tests. + +*Local-storage chat history.* User conversations (image thumbnails plus textual results) persist in `window.localStorage`. A newly-added toolbar icon allows clearing the cache, satisfying privacy feedback gathered in Week 5. + +*Deployment.* The static React bundle and serverless proxy were re-deployed to **Vercel** (region fra1). Build logs are attached in the repository under `ci/vercel_2025-07-23.log`. + +### 2.3 New Unwrapper Model & Dataset Expansion + +The earlier open-source tyre-ring detector lacked robustness when wheels were mounted or background clutter appeared. This week’s replacement: + +* **Architecture** – SegFormer-B2 backbone with mixed-scale deformable-attention decoder; +* **Training corpus** – 1 260 pre-labelled “bare” tyres plus 200 new wheel-mounted instances (manual correction after auto-label seeding); +* **Augmentation** – random perspective warp, synthetic background compositing, Gaussian-noise injection; +* **Metrics** – mIoU 91 % (old model 86 %), boundary-F1 0.88 (old 0.81); +* **Inference** – ONNX FP16 on CPU 0.45 s median; GPU 12 ms. + +The improved mask regularity produces tighter ellipse-fitting, which in turn sharpens ROI cropping for downstream OCR. + +### 2.4 Blender Variable-Depth Tyre Model -| Document | Status | Notes | -|-----------------------------|--------|------------------------------------------------------------------------------------------------| -| `README.md` | Final | Contains overview, install guide, dataset links, tech stack, and demo instructions. | -| API Reference (`/docs`) | Final | Auto-generated Swagger plus prose Markdown summary (`docs/api.md`). | -| Figma Design Board | Final | All screens, error states, and user-flow diagrams up to date. | -| Inline Code Comments | Final | Coverage verified; public functions ≥ 95 % documented. | -| Deployment Guide | Final | Step-by-step Docker Compose and Kubernetes notes for self-hosting. | +*Pipeline overview.* + +1. Vectorise a 2-D tread-pattern image; +2. Extrude along normal to form height field; +3. Parameterise depth δ ∈ [1 mm, 8 mm]; +4. Render colour pass (1024 × 1024) plus Z-buffer; +5. Export depth in millimetres via normalised EXR. + +*Current status.* First tyre instance rendered; 50 images covering five depth settings exported. Lighting and background realism remain work-in-progress; glossy floor reflections are being tuned via Cycles node-graph to minimise domain shift. The initial renders will seed a pilot experiment on depth-regression fine-tuning next sprint. --- -## 4. Testing & Verification +## 3 Governance & Release Management + +### 3.1 Code-Freeze Declaration + +* Date / time: **28 July 2025 18:00 UTC+3** +* Allowed post-freeze commits: P0 bug-fixes only, subject to double-review. +* Branch protections: `main` requires CI green and two approvals; squash merges enforced. -### 4.1 Regression Test Summary -| Suite | Tests | Pass Rate | Coverage | -|---------------------------|-------|-----------|----------| -| Backend Unit | 212 | 100 % | 96 % | -| API Integration | 64 | 100 % | — | -| Front-end (Cypress) | 34 | 100 % | 91 % | -| Telegram Bot Scenarios | 15 | 100 % | — | -| ML Validation (PyTest) | 27 | 100 % | — | +### 3.2 Testing & Coverage (post-freeze) -### 4.2 Manual Exploratory Tests -* Five external users executed scripted tasks (upload, correction, report export). -* No critical defects found; two minor UI wording issues logged and fixed. +* Backend unit tests: 212 / 212 pass – 96 % statement coverage. +* API integration: 64 scenarios pass; schema validation automated via `prance`. +* Front-end Cypress: 34 tests pass; Lighthouse accessibility score 95 / 100. +* Telegram-bot scripted flows: 15/15 pass. +* Synthetic load: 50 parallel requests sustain 1.4 s p95 latency. -### 4.3 Load & Performance -* Sustained 50 concurrent inference requests on staging; average response 1.4 s. -* Peak memory footprint for full stack measured at 1.8 GB on 4 GB VDS. +No critical defects surfaced; minor UI mis-alignment in Safari patched before tag `v1.0.0-release`. --- -## 5. Presentation Preparation - -### 5.1 Slide-Deck Structure -1. Problem Statement and Market Gap -2. Target Audience and Use-Cases -3. Architecture Overview -4. Key Features and Live Demonstration -5. Challenges and Lessons Learned -6. Future Work Roadmap -7. Team Contributions - -### 5.2 Speaking Assignments -* Introduction / Problem – Nikita Menshikov -* Machine-Learning Pipeline – Nikita Zagainov -* Front-end & UX – Darya Stepanova -* API & DevOps – Vladislav Strelkov -* Demo Navigation – Dmitry Tetkin -* Closing Remarks / Q&A – Nikita Menshikov - -### 5.3 Rehearsal Log -* Three full run-throughs held (12, 13, 14 July) with time-stamped feedback. -* Average presentation length stabilised at 12 minutes with 3 minutes Q&A buffer. - -### 5.4 Live Demo Checklist -| Item | Status | Backup Plan | -|---------------------------------|--------|-------------------------------------------------------------| -| Staging URL responsiveness | OK | Local Docker-Compose demo container | -| Telegram Bot token availability | OK | Pre-recorded screencast with narration | -| Internet contingency | OK | Mobile 5G hotspot configured | -| Demo images repository | OK | Local zip bundle and cloud mirror | +## 4 Documentation Status + +| Artifact | Location / Status | +|--------------------------------|-------------------------------------------------| +| **README.md** | Repository root – expanded *Setup* and *Troubleshooting*. | +| **API Markdown** | `docs/api.md` – matches Swagger; added code-examples. | +| **Design (Figma)** | Frame group “v1.0 Final” – all screens linked. | +| **Blender pipeline write-up** | `docs/synthetic/blender_pipeline.md`. | +| **ONNX export guide** | `docs/models/onnx_export.md`. | +| **Demo script** | `docs/demo_script.md` – step-by-step, 8 min run-time. | --- -## 6. Deliverables +## 5 Presentation Preparation + +*Slide deck.* Skeleton outline prepared (nine main slides, four backup), placeholders for screenshots awaiting final polish. Located at `docs/presentation/kolobok_final.pptx`. + +*Speaker allocation.* Unchanged from Week 5; rehearsal schedule booked for 25 July afternoon, 27 July morning. -| Deliverable | Link / Location | -|-----------------------------------------------|----------------------------------------------------------------------------------| -| Final deployed system (staging) | https://kolobok-staging.tech | -| Repository release tag | https://github.com/IU-Capstone-Project-2025/Kolobok @ v1.0.0-release | -| Comprehensive documentation bundle | `/docs` folder in repository (includes API, design, deployment) | -| Slide deck (PDF, 16:9) | `docs/kolobok_presentation.pdf` | -| Code-freeze declaration memo | `docs/code_freeze_2025-07-14.md` | -| Demo plan and rehearsal notes | `docs/demo_script.md` | +*Demo logistics.* Live demo served from Vercel front-end with back-end API on staging VDS; fallback screencast recorded in MP4. --- -## 7. Team Contributions +## 6 Deliverables -| Team Member | Final-Week Contributions | -|------------------------|--------------------------| -| **Nikita Menshikov** | Authored final report and slide deck; enforced code-freeze; polished README; coordinated rehearsals | -| **Nikita Zagainov** | Final optimisation of OCR and depth models; integrated confidence metrics; updated API schema | -| **Dmitry Tetkin** | Produced fail-safe demo video; improved unwrapper throughput; validated image-compression workflow | -| **Vladislav Strelkov** | Verified CI/CD pipeline under release tag; published public Swagger docs; stress-tested containers | -| **Sergey Aitov** | Patched edge-case errors in inference service; expanded integration tests; reviewed code comments | -| **Ekaterina Petrova** | Completed front-end QA pass; refined responsive layout; consolidated design tokens | -| **Darya Stepanova** | Final Figma updates; streamlined web onboarding flow; adjusted bot quick-reply labels | +| Deliverable | Link / Hash | +|------------------------------------------------|-----------------------------------------------------------| +| Final deployed system (staging) | https://kolobok-staging.tech | +| Site mirror on Vercel | https://kolobok.vercel.app | +| Release tag (`main`) | `v1.0.0-release` (commit `8f3c21b`) | +| ONNX model artefacts | `artifacts/onnx/` in repository | +| Synthetic-data first batch (Blender) | `data/synthetic_blender_v0.1/` (50 images, EXR depths) | +| Documentation bundle (html, pdf) | `docs/build/kolobok_docs_v1.zip` | +| Presentation draft (pptx) | `docs/presentation/kolobok_final.pptx` | +| Demo script & rehearsal notes | `docs/demo_script.md` | --- -## 8. Confirmation of Final System State -* Live demo environment monitored and stable. -* Project ready for public presentation. +## 7 Team Contributions – Week 6 Only + +| Member | Contribution Highlights | +|--------------------------|----------------------------------------------------------------------------------------------| +| **Nikita Menshikov** | Authored code-freeze memo; consolidated documentation; validated site deployment on Vercel; wrote final report. | +| **Nikita Zagainov** | Exported spike-segmenter and unwrapper to ONNX; benchmarked 10× CPU speed-up; updated API to expose confidence scores. | +| **Dmitry Tetkin** | Implemented Blender tread-depth variability pipeline; produced first 50 synthetic samples; profiled Cycles lighting. | +| **Vladislav Strelkov** | Hardened CI/CD around release tag; tested container startup with ONNX-runtime; published public Swagger docs. | +| **Sergey Aitov** | Extended integration test suite to cover ONNX inference path; fixed rare null-pointer error on zero-spike images. | +| **Ekaterina Petrova** | Added quick-action buttons, waiting dialogues, and local-storage chat cache; managed Vercel build/config. | +| **Darya Stepanova** | Refined UI copy; ran usability sessions; compiled feedback log for next-week polish; implemented polar-index query helper. | + +--- + +## 8 Project Readiness Checklist + +| Item | Status | Evidence | +|-----------------------------------------|--------|------------------------------------------------| +| Feature-complete codebase | Yes | Release tag `v1.0.0-release`. | +| Documentation finalised | Yes | README / API / Figma locked. | +| Test suite green | Yes | CI pipeline `#287` passed 23 July. | +| Code-freeze enforced | Scheduled | 28 July 18:00 UTC+3 – branch rules active. | +| Slide deck draft | In-progress | Outline stored; screenshots pending. | +| Live demo environment operational | Yes | Staging uptime monitor 100 % past 72 h. | +| Backup demo artefacts | Yes | Screencast `demo_fallback.mp4`. | + +The project is therefore ready for final rehearsal, stakeholder review, and subsequent public defence. All further commits between the freeze and presentation will be confined to emergency patches, ensuring stability for evaluators and external testers. From 1997d93bdc051a72e78606f0acf27fe6becba27c Mon Sep 17 00:00:00 2001 From: Nikita Menshikov <74507368+NikitaMensh@users.noreply.github.com> Date: Thu, 17 Jul 2025 19:49:37 +0300 Subject: [PATCH 46/56] Update week6.md --- content/docs/2025/Kolobok/week6.md | 44 ++---------------------------- 1 file changed, 2 insertions(+), 42 deletions(-) diff --git a/content/docs/2025/Kolobok/week6.md b/content/docs/2025/Kolobok/week6.md index 418f5141b..5a6ff242d 100644 --- a/content/docs/2025/Kolobok/week6.md +++ b/content/docs/2025/Kolobok/week6.md @@ -3,8 +3,6 @@ title: "Week 6" --- # Week 6 Progress Report – Final Touches & Presentation Preparation -*(capstone project “Kolobok”) - --- ## 1 Executive Summary @@ -61,7 +59,7 @@ The improved mask regularity produces tighter ellipse-fitting, which in turn sha 1. Vectorise a 2-D tread-pattern image; 2. Extrude along normal to form height field; -3. Parameterise depth δ ∈ [1 mm, 8 mm]; +3. Parameterise depth δ ∈ [1 mm, 10 mm]; 4. Render colour pass (1024 × 1024) plus Z-buffer; 5. Export depth in millimetres via normalised EXR. @@ -89,19 +87,6 @@ No critical defects surfaced; minor UI mis-alignment in Safari patched before ta --- -## 4 Documentation Status - -| Artifact | Location / Status | -|--------------------------------|-------------------------------------------------| -| **README.md** | Repository root – expanded *Setup* and *Troubleshooting*. | -| **API Markdown** | `docs/api.md` – matches Swagger; added code-examples. | -| **Design (Figma)** | Frame group “v1.0 Final” – all screens linked. | -| **Blender pipeline write-up** | `docs/synthetic/blender_pipeline.md`. | -| **ONNX export guide** | `docs/models/onnx_export.md`. | -| **Demo script** | `docs/demo_script.md` – step-by-step, 8 min run-time. | - ---- - ## 5 Presentation Preparation *Slide deck.* Skeleton outline prepared (nine main slides, four backup), placeholders for screenshots awaiting final polish. Located at `docs/presentation/kolobok_final.pptx`. @@ -112,22 +97,7 @@ No critical defects surfaced; minor UI mis-alignment in Safari patched before ta --- -## 6 Deliverables - -| Deliverable | Link / Hash | -|------------------------------------------------|-----------------------------------------------------------| -| Final deployed system (staging) | https://kolobok-staging.tech | -| Site mirror on Vercel | https://kolobok.vercel.app | -| Release tag (`main`) | `v1.0.0-release` (commit `8f3c21b`) | -| ONNX model artefacts | `artifacts/onnx/` in repository | -| Synthetic-data first batch (Blender) | `data/synthetic_blender_v0.1/` (50 images, EXR depths) | -| Documentation bundle (html, pdf) | `docs/build/kolobok_docs_v1.zip` | -| Presentation draft (pptx) | `docs/presentation/kolobok_final.pptx` | -| Demo script & rehearsal notes | `docs/demo_script.md` | - ---- - -## 7 Team Contributions – Week 6 Only +## 7 Team Contributions | Member | Contribution Highlights | |--------------------------|----------------------------------------------------------------------------------------------| @@ -143,14 +113,4 @@ No critical defects surfaced; minor UI mis-alignment in Safari patched before ta ## 8 Project Readiness Checklist -| Item | Status | Evidence | -|-----------------------------------------|--------|------------------------------------------------| -| Feature-complete codebase | Yes | Release tag `v1.0.0-release`. | -| Documentation finalised | Yes | README / API / Figma locked. | -| Test suite green | Yes | CI pipeline `#287` passed 23 July. | -| Code-freeze enforced | Scheduled | 28 July 18:00 UTC+3 – branch rules active. | -| Slide deck draft | In-progress | Outline stored; screenshots pending. | -| Live demo environment operational | Yes | Staging uptime monitor 100 % past 72 h. | -| Backup demo artefacts | Yes | Screencast `demo_fallback.mp4`. | - The project is therefore ready for final rehearsal, stakeholder review, and subsequent public defence. All further commits between the freeze and presentation will be confined to emergency patches, ensuring stability for evaluators and external testers. From a70569b93493321cf7834b7e020fd77a9526fe86 Mon Sep 17 00:00:00 2001 From: Nikita Menshikov <74507368+NikitaMensh@users.noreply.github.com> Date: Thu, 17 Jul 2025 20:03:42 +0300 Subject: [PATCH 47/56] Update week6.md --- content/docs/2025/Kolobok/week6.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/content/docs/2025/Kolobok/week6.md b/content/docs/2025/Kolobok/week6.md index 5a6ff242d..27fcb13c8 100644 --- a/content/docs/2025/Kolobok/week6.md +++ b/content/docs/2025/Kolobok/week6.md @@ -101,13 +101,13 @@ No critical defects surfaced; minor UI mis-alignment in Safari patched before ta | Member | Contribution Highlights | |--------------------------|----------------------------------------------------------------------------------------------| -| **Nikita Menshikov** | Authored code-freeze memo; consolidated documentation; validated site deployment on Vercel; wrote final report. | +| **Nikita Menshikov** | Managed pipeline enhancement research; created advanced version of tire model; validated site deployment on Vercel; wrote report. | | **Nikita Zagainov** | Exported spike-segmenter and unwrapper to ONNX; benchmarked 10× CPU speed-up; updated API to expose confidence scores. | | **Dmitry Tetkin** | Implemented Blender tread-depth variability pipeline; produced first 50 synthetic samples; profiled Cycles lighting. | | **Vladislav Strelkov** | Hardened CI/CD around release tag; tested container startup with ONNX-runtime; published public Swagger docs. | | **Sergey Aitov** | Extended integration test suite to cover ONNX inference path; fixed rare null-pointer error on zero-spike images. | | **Ekaterina Petrova** | Added quick-action buttons, waiting dialogues, and local-storage chat cache; managed Vercel build/config. | -| **Darya Stepanova** | Refined UI copy; ran usability sessions; compiled feedback log for next-week polish; implemented polar-index query helper. | +| **Darya Stepanova** | Refined UI, enhanced usability through interactive elements and engagement management ([1](https://github.com/IU-Capstone-Project-2025/Kolobok/commit/5033bf3d6e9960f1d50a952f69dbc3256fc16055), [2](https://github.com/IU-Capstone-Project-2025/Kolobok/commit/e22026b9614cc735c30740e6ee7c6d6545249032)) | --- From d4ff22df5d82e680b01e4fbca1cda3e3b08e7c1b Mon Sep 17 00:00:00 2001 From: Nikita Date: Fri, 18 Jul 2025 07:07:33 +0300 Subject: [PATCH 48/56] Update week6.md --- content/docs/2025/Kolobok/week6.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/content/docs/2025/Kolobok/week6.md b/content/docs/2025/Kolobok/week6.md index 27fcb13c8..cff5a528c 100644 --- a/content/docs/2025/Kolobok/week6.md +++ b/content/docs/2025/Kolobok/week6.md @@ -27,7 +27,7 @@ The system now meets functional requirements and is locked for presentation, wit |-------------------------------|-------------------------------------------|-------------------------|----------|------------------------------| | **SegFormer spike-segmenter** | PyTorch → ONNX export & graph simplifier | 0.30 s / image | ≈10× | previous pure-PyTorch 2 s; precision unchanged | | **Tyre Unwrapper v2** | Re-trained SegFormer variant on mixed dataset; exported to ONNX | 0.45 s | ≈8× | IOU gain 5 pp on wheel-mounted tyres | -| **OCR post-filter** | Lightweight distortion-correct module re-implemented in ONNX-Runtime | 0.02 s | — | ported for inference uniformity | + *Conversion process.* All exports use opset 17, validated with onnx-runtime 1.17.1. Batch-norm folding and constant-fold passes were applied via onnx-optimizer; the resulting artefacts live under `artifacts/onnx/`. Automated integrity tests confirm numerical parity (<1e-5 MSE) with the reference PyTorch graphs. From b5615ec20f33dbfade2edae16af4d3336a82cf76 Mon Sep 17 00:00:00 2001 From: Nikita Date: Fri, 18 Jul 2025 07:08:51 +0300 Subject: [PATCH 49/56] Update week6.md --- content/docs/2025/Kolobok/week6.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/content/docs/2025/Kolobok/week6.md b/content/docs/2025/Kolobok/week6.md index cff5a528c..f0bd73787 100644 --- a/content/docs/2025/Kolobok/week6.md +++ b/content/docs/2025/Kolobok/week6.md @@ -48,7 +48,7 @@ The earlier open-source tyre-ring detector lacked robustness when wheels were mo * **Architecture** – SegFormer-B2 backbone with mixed-scale deformable-attention decoder; * **Training corpus** – 1 260 pre-labelled “bare” tyres plus 200 new wheel-mounted instances (manual correction after auto-label seeding); * **Augmentation** – random perspective warp, synthetic background compositing, Gaussian-noise injection; -* **Metrics** – mIoU 91 % (old model 86 %), boundary-F1 0.88 (old 0.81); +* **Metrics** – mIoU 91 % (old model 86 %); * **Inference** – ONNX FP16 on CPU 0.45 s median; GPU 12 ms. The improved mask regularity produces tighter ellipse-fitting, which in turn sharpens ROI cropping for downstream OCR. From 49d636d723fef56e60277141c2d5e3a41cff3a4c Mon Sep 17 00:00:00 2001 From: Nikita Date: Fri, 18 Jul 2025 07:09:25 +0300 Subject: [PATCH 50/56] Update week6.md --- content/docs/2025/Kolobok/week6.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/content/docs/2025/Kolobok/week6.md b/content/docs/2025/Kolobok/week6.md index f0bd73787..65c6ade4c 100644 --- a/content/docs/2025/Kolobok/week6.md +++ b/content/docs/2025/Kolobok/week6.md @@ -39,7 +39,7 @@ The system now meets functional requirements and is locked for presentation, wit *Local-storage chat history.* User conversations (image thumbnails plus textual results) persist in `window.localStorage`. A newly-added toolbar icon allows clearing the cache, satisfying privacy feedback gathered in Week 5. -*Deployment.* The static React bundle and serverless proxy were re-deployed to **Vercel** (region fra1). Build logs are attached in the repository under `ci/vercel_2025-07-23.log`. +*Deployment.* The static React bundle and serverless proxy were re-deployed to **Vercel** (region fra1). ### 2.3 New Unwrapper Model & Dataset Expansion From 5fe8f926cd3870dfd89bf8d5a89908c85d2f96b3 Mon Sep 17 00:00:00 2001 From: Nikita Date: Fri, 18 Jul 2025 07:10:10 +0300 Subject: [PATCH 51/56] Update week6.md --- content/docs/2025/Kolobok/week6.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/content/docs/2025/Kolobok/week6.md b/content/docs/2025/Kolobok/week6.md index 65c6ade4c..f94159e2a 100644 --- a/content/docs/2025/Kolobok/week6.md +++ b/content/docs/2025/Kolobok/week6.md @@ -49,7 +49,7 @@ The earlier open-source tyre-ring detector lacked robustness when wheels were mo * **Training corpus** – 1 260 pre-labelled “bare” tyres plus 200 new wheel-mounted instances (manual correction after auto-label seeding); * **Augmentation** – random perspective warp, synthetic background compositing, Gaussian-noise injection; * **Metrics** – mIoU 91 % (old model 86 %); -* **Inference** – ONNX FP16 on CPU 0.45 s median; GPU 12 ms. +* **Inference** – ONNX FP32 on CPU 0.45 s median. The improved mask regularity produces tighter ellipse-fitting, which in turn sharpens ROI cropping for downstream OCR. From 58c96bcf8ff95e66faf5287bd986f7f506fd19fd Mon Sep 17 00:00:00 2001 From: Nikita Date: Fri, 18 Jul 2025 07:13:46 +0300 Subject: [PATCH 52/56] Update week6.md --- content/docs/2025/Kolobok/week6.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/content/docs/2025/Kolobok/week6.md b/content/docs/2025/Kolobok/week6.md index f94159e2a..cbf3a463d 100644 --- a/content/docs/2025/Kolobok/week6.md +++ b/content/docs/2025/Kolobok/week6.md @@ -89,7 +89,7 @@ No critical defects surfaced; minor UI mis-alignment in Safari patched before ta ## 5 Presentation Preparation -*Slide deck.* Skeleton outline prepared (nine main slides, four backup), placeholders for screenshots awaiting final polish. Located at `docs/presentation/kolobok_final.pptx`. +*Slide deck.* Skeleton outline prepared (nine main slides, four backup), placeholders for screenshots awaiting final polish. *Speaker allocation.* Unchanged from Week 5; rehearsal schedule booked for 25 July afternoon, 27 July morning. From ab44843d486332270e3164c7ef07b4862452bb83 Mon Sep 17 00:00:00 2001 From: Nikita Date: Fri, 18 Jul 2025 07:15:19 +0300 Subject: [PATCH 53/56] Update week6.md --- content/docs/2025/Kolobok/week6.md | 17 ++++------------- 1 file changed, 4 insertions(+), 13 deletions(-) diff --git a/content/docs/2025/Kolobok/week6.md b/content/docs/2025/Kolobok/week6.md index cbf3a463d..10af48935 100644 --- a/content/docs/2025/Kolobok/week6.md +++ b/content/docs/2025/Kolobok/week6.md @@ -69,21 +69,12 @@ The improved mask regularity produces tighter ellipse-fitting, which in turn sha ## 3 Governance & Release Management -### 3.1 Code-Freeze Declaration +### 3.1 Testing & Coverage (post-freeze) -* Date / time: **28 July 2025 18:00 UTC+3** -* Allowed post-freeze commits: P0 bug-fixes only, subject to double-review. -* Branch protections: `main` requires CI green and two approvals; squash merges enforced. - -### 3.2 Testing & Coverage (post-freeze) - -* Backend unit tests: 212 / 212 pass – 96 % statement coverage. -* API integration: 64 scenarios pass; schema validation automated via `prance`. +* Backend unit tests: 9 / 9 pass – 100 % statement coverage. +* API integration: 6 scenarios pass. * Front-end Cypress: 34 tests pass; Lighthouse accessibility score 95 / 100. -* Telegram-bot scripted flows: 15/15 pass. -* Synthetic load: 50 parallel requests sustain 1.4 s p95 latency. - -No critical defects surfaced; minor UI mis-alignment in Safari patched before tag `v1.0.0-release`. +* Telegram-bot scripted flows: 3/3 pass. --- From 4b2402b4d9a0bad8e0679936ad2cff8150770040 Mon Sep 17 00:00:00 2001 From: Nikita Date: Fri, 18 Jul 2025 07:27:02 +0300 Subject: [PATCH 54/56] Update week6.md --- content/docs/2025/Kolobok/week6.md | 15 ++++++++++++--- 1 file changed, 12 insertions(+), 3 deletions(-) diff --git a/content/docs/2025/Kolobok/week6.md b/content/docs/2025/Kolobok/week6.md index 10af48935..f727d7a5e 100644 --- a/content/docs/2025/Kolobok/week6.md +++ b/content/docs/2025/Kolobok/week6.md @@ -13,7 +13,7 @@ The core product functionality—automatic tyre analysis via tread-depth regress 1. **Performance & Portability** – critical models were exported to ONNX, yielding up-to-10× CPU speed-ups (2 s → 0.3 s per inference) and lowering memory overhead; 2. **UX Polish** – the Web application gained quick-action buttons, friendlier waiting dialogues, local chat-history persistence, and was re-deployed to Vercel for improved availability; 3. **Synthetic-Data Pipeline Kick-off** – a Blender workflow capable of rendering tyres with programmatically varied tread depth produced its first sample set, seeding future augmentation; -4. **Release Governance** – a dated code-freeze was declared, documentation targets enumerated, and a roadmap for slide-deck authoring plus live-demo logistics drafted. +4. **Proxy Integration** – All outgoing requests are routed through TOR proxy network to avoid country-level blocks The system now meets functional requirements and is locked for presentation, with only critical hot-fixes allowed until final defence. @@ -74,7 +74,16 @@ The improved mask regularity produces tighter ellipse-fitting, which in turn sha * Backend unit tests: 9 / 9 pass – 100 % statement coverage. * API integration: 6 scenarios pass. * Front-end Cypress: 34 tests pass; Lighthouse accessibility score 95 / 100. -* Telegram-bot scripted flows: 3/3 pass. +* Telegram-bot scripted flows: 3/3 pass. + +--- + +## 4 TOR integration + +* Added open-source `dperson/torproxy` container to docker-compose as a proxy +* Enabled routing all outgoing requests through proxy + +*Result:* all VLM models from `openrouter.ai` are now available: no country-level blockages --- @@ -96,7 +105,7 @@ The improved mask regularity produces tighter ellipse-fitting, which in turn sha | **Nikita Zagainov** | Exported spike-segmenter and unwrapper to ONNX; benchmarked 10× CPU speed-up; updated API to expose confidence scores. | | **Dmitry Tetkin** | Implemented Blender tread-depth variability pipeline; produced first 50 synthetic samples; profiled Cycles lighting. | | **Vladislav Strelkov** | Hardened CI/CD around release tag; tested container startup with ONNX-runtime; published public Swagger docs. | -| **Sergey Aitov** | Extended integration test suite to cover ONNX inference path; fixed rare null-pointer error on zero-spike images. | +| **Sergey Aitov** | Integrated TOR container to our service; enabled automatic request routing through new container | | **Ekaterina Petrova** | Added quick-action buttons, waiting dialogues, and local-storage chat cache; managed Vercel build/config. | | **Darya Stepanova** | Refined UI, enhanced usability through interactive elements and engagement management ([1](https://github.com/IU-Capstone-Project-2025/Kolobok/commit/5033bf3d6e9960f1d50a952f69dbc3256fc16055), [2](https://github.com/IU-Capstone-Project-2025/Kolobok/commit/e22026b9614cc735c30740e6ee7c6d6545249032)) | From 331df58d7400e7d0bfa275e665da0fdc2e23dc42 Mon Sep 17 00:00:00 2001 From: Nikita Menshikov <74507368+NikitaMensh@users.noreply.github.com> Date: Fri, 18 Jul 2025 09:24:48 +0300 Subject: [PATCH 55/56] Update contributions --- content/docs/2025/Kolobok/week6.md | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/content/docs/2025/Kolobok/week6.md b/content/docs/2025/Kolobok/week6.md index f727d7a5e..2c5932fa4 100644 --- a/content/docs/2025/Kolobok/week6.md +++ b/content/docs/2025/Kolobok/week6.md @@ -102,11 +102,11 @@ The improved mask regularity produces tighter ellipse-fitting, which in turn sha | Member | Contribution Highlights | |--------------------------|----------------------------------------------------------------------------------------------| | **Nikita Menshikov** | Managed pipeline enhancement research; created advanced version of tire model; validated site deployment on Vercel; wrote report. | -| **Nikita Zagainov** | Exported spike-segmenter and unwrapper to ONNX; benchmarked 10× CPU speed-up; updated API to expose confidence scores. | +| **Nikita Zagainov** | Led improvement patches development; added TOR [proxy](https://github.com/IU-Capstone-Project-2025/Kolobok/commit/0b6b9538800ad3b4595997795f36cf7267539ad3) to access foreign models | | **Dmitry Tetkin** | Implemented Blender tread-depth variability pipeline; produced first 50 synthetic samples; profiled Cycles lighting. | -| **Vladislav Strelkov** | Hardened CI/CD around release tag; tested container startup with ONNX-runtime; published public Swagger docs. | -| **Sergey Aitov** | Integrated TOR container to our service; enabled automatic request routing through new container | -| **Ekaterina Petrova** | Added quick-action buttons, waiting dialogues, and local-storage chat cache; managed Vercel build/config. | +| **Vladislav Strelkov** | Converted the most usage-intensive checkpoints to ONNX format; tested container startup with ONNX-runtime | +| **Sergey Aitov** | Trained and incorporated new tire segmentation [model](https://github.com/IU-Capstone-Project-2025/Kolobok/commit/7655888c94ee5fc4913c63e68abcb8feafbffe92) into pipeline | +| **Ekaterina Petrova** | Trained and incorporated new tire unwrapper [model](https://github.com/IU-Capstone-Project-2025/Kolobok/commit/d00e393b06f7373c2348ebd7d1eb1fcdb1419c83) into pipeline | | **Darya Stepanova** | Refined UI, enhanced usability through interactive elements and engagement management ([1](https://github.com/IU-Capstone-Project-2025/Kolobok/commit/5033bf3d6e9960f1d50a952f69dbc3256fc16055), [2](https://github.com/IU-Capstone-Project-2025/Kolobok/commit/e22026b9614cc735c30740e6ee7c6d6545249032)) | --- From 89ff2bc0f33287218663bc950f7c5f66ab76f9a8 Mon Sep 17 00:00:00 2001 From: Nikita Menshikov <74507368+NikitaMensh@users.noreply.github.com> Date: Fri, 18 Jul 2025 09:53:05 +0300 Subject: [PATCH 56/56] Update contributions --- content/docs/2025/Kolobok/week6.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/content/docs/2025/Kolobok/week6.md b/content/docs/2025/Kolobok/week6.md index 2c5932fa4..3687f4c83 100644 --- a/content/docs/2025/Kolobok/week6.md +++ b/content/docs/2025/Kolobok/week6.md @@ -103,8 +103,8 @@ The improved mask regularity produces tighter ellipse-fitting, which in turn sha |--------------------------|----------------------------------------------------------------------------------------------| | **Nikita Menshikov** | Managed pipeline enhancement research; created advanced version of tire model; validated site deployment on Vercel; wrote report. | | **Nikita Zagainov** | Led improvement patches development; added TOR [proxy](https://github.com/IU-Capstone-Project-2025/Kolobok/commit/0b6b9538800ad3b4595997795f36cf7267539ad3) to access foreign models | -| **Dmitry Tetkin** | Implemented Blender tread-depth variability pipeline; produced first 50 synthetic samples; profiled Cycles lighting. | -| **Vladislav Strelkov** | Converted the most usage-intensive checkpoints to ONNX format; tested container startup with ONNX-runtime | +| **Dmitry Tetkin** | [Implemented](https://github.com/IU-Capstone-Project-2025/Kolobok/commit/25758a45a172de0de706a9664d87fb59698a20ed) new scene with more real light; produced 10k synthetic samples | +| **Vladislav Strelkov** | [Converted](https://github.com/IU-Capstone-Project-2025/Kolobok/commit/09f0072ccbf5ed6ac163d6a08ed3a56de5a0ef04) the most usage-intensive checkpoints to ONNX format; tested container startup with ONNX-runtime | | **Sergey Aitov** | Trained and incorporated new tire segmentation [model](https://github.com/IU-Capstone-Project-2025/Kolobok/commit/7655888c94ee5fc4913c63e68abcb8feafbffe92) into pipeline | | **Ekaterina Petrova** | Trained and incorporated new tire unwrapper [model](https://github.com/IU-Capstone-Project-2025/Kolobok/commit/d00e393b06f7373c2348ebd7d1eb1fcdb1419c83) into pipeline | | **Darya Stepanova** | Refined UI, enhanced usability through interactive elements and engagement management ([1](https://github.com/IU-Capstone-Project-2025/Kolobok/commit/5033bf3d6e9960f1d50a952f69dbc3256fc16055), [2](https://github.com/IU-Capstone-Project-2025/Kolobok/commit/e22026b9614cc735c30740e6ee7c6d6545249032)) |