Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Tiling and TilesPatcher #89

Merged
merged 5 commits into from
Oct 2, 2024
Merged

Tiling and TilesPatcher #89

merged 5 commits into from
Oct 2, 2024

Conversation

spbui00
Copy link
Contributor

@spbui00 spbui00 commented Sep 30, 2024

High level structure

Tiling node's key responsibilities

  • Divide the input frame into tiles based on the grid size and overlap.
  • Resize each tile to match the neural network’s input size.

Pathcer node's key responsibilities

  • bounding box extraction (decoding of neural network output)
  • scales and map the output bounding boxes to global image coordinates based on tile_index
  • performs Non-Maximum Suppression (NMS) (read more in paper or this Medium post) to eliminate duplicate detections and outputs the final set of bounding boxes
           ┌───────────────┐
           │               │
           │  ReplayVideo  │  # Inputs video frames
           │   or Camera   │
           │               │
           └──────┬────────┘
                  │
                  │ passthroughVideo 
                  │ (full frame)
                  ▼
           ┌───────────────┐
           │               │
           │    Tiling     │  # Splits frame into tiles
           │               │
           └──────┬────────┘
                  │
                  │ outputTiles
                  │ (n individual tiles)
                  ▼
           ┌───────────────┐
           │               │
           │ NeuralNetwork │  # Processes each tile, perform detection
           │               │
           └──────┬────────┘
                  │
                  │ outputDetections 
                  │ (n detections)
                  ▼
           ┌───────────────┐
           │               │
           │    Patcher    │  # Decode nn output and merges tile results
           │               │
           └──────┬────────┘
                  │
                  │ outputBoxes 
                  │ (merged boxes from n tiles)
                  ▼
           ┌───────────────┐
           │               │
           │    Display    │  # Shows final frame with bboxes from detections
           │               │
           └───────────────┘

klemen1999 and others added 2 commits September 4, 2024 13:21
@klemen1999 @kozlov721
@actions-user @kkeroo @jkbmrz
Release depthai-nodes v0.0.1 (#53)
8fc6ba2 
* Added labeler to CI (#43)

* Updated requirements.txt (#44)

* Added coverage badge to README (#45)

* Added package version (#46)

* Update ci.yaml

* [Automated] Updated coverage badge

* Update README.md

* YOLOv8 kpts and seg example. (#48)

* Extend MPPalmDetection parser. (#49)

* Add support for different input sizes.

* Support extended palm detection in examples.

* Pre-commit fix.

* Add normalization.

* feat: add support for metric depth

* fix: broken depth tests

* fix: pre-commit

* fix: change default depth limit to float

* feat: add test for metric depth map

* Developer guide docs added. (#51)

* Support for metrics depth type. (#52)

* Hotfix: remove no-commit from pre-commit (#54)

* Added automatic PyPI publishing (#55)

---------

Co-authored-by: Martin Kozlovský <kozlovsky.m7@gmail.com>
Co-authored-by: GitHub Actions <actions@github.com>
Co-authored-by: Jaša Kerec <61207502+kkeroo@users.noreply.github.com>
Co-authored-by: jkbmrz <jakob.mraz@gmail.com>
Co-authored-by: jkbmrz <74824974+jkbmrz@users.noreply.github.com>
@spbui00
initial commit with two nodes - Tiling and TilesPatcher | TilesPatche…
a26279a 
…r implemented to work with dai.ImgDetections
@github-actions github-actions bot added the release New version release label Sep 30, 2024
@spbui00 spbui00 changed the title initial commit with two nodes - Tiling and TilesPatcher Tiling and TilesPatcher Sep 30, 2024
jkbmrz
jkbmrz reviewed Oct 1, 2024
View reviewed changes
Copy link
Collaborator

@jkbmrz jkbmrz left a comment •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM in general.
Please re-root changes to depthai_nodes/ml/helpers (as discussed on Slack) and change the PR to merge into dev instead of main. Also, add docstrings to the newly added classes.

@spbui00 spbui00 changed the base branch from main to dev October 1, 2024 21:49
@jkbmrz
Copy link
Collaborator

jkbmrz commented Oct 2, 2024

I've approved the request. Just fix the failing tests before you merge!

@spbui00 spbui00 merged commit 2a312f8 into dev Oct 2, 2024
4 of 5 checks passed
@spbui00 spbui00 deleted the tiling branch October 2, 2024 10:33
@kkeroo kkeroo mentioned this pull request Oct 7, 2024
Merged
klemen1999 added a commit that referenced this pull request Oct 7, 2024
@klemen1999 @kozlov721
@actions-user @kkeroo @jkbmrz @aljazkonec1 @HonzaCuhel @spbui00
Release depthai-nodes v0.0.2 (#101)
9458283 
* Added labeler to CI (#43)

* Updated requirements.txt (#44)

* Added coverage badge to README (#45)

* Added package version (#46)

* Update ci.yaml

* [Automated] Updated coverage badge

* Update README.md

* YOLOv8 kpts and seg example. (#48)

* Extend MPPalmDetection parser. (#49)

* Add support for different input sizes.

* Support extended palm detection in examples.

* Pre-commit fix.

* Add normalization.

* feat: add support for metric depth

* fix: broken depth tests

* fix: pre-commit

* fix: change default depth limit to float

* feat: add test for metric depth map

* Developer guide docs added. (#51)

* Support for metrics depth type. (#52)

* Hotfix: remove no-commit from pre-commit (#54)

* Added automatic PyPI publishing (#55)

* feat: add parser for DM-Count crowd density estimation model

* fix: pre-commit

* fix: add missing imports

* feat: add property methods to Map2D message

* fix: density map dimensions

* fix: pre-commit

* feat: unify map parsers

* feat: unify map message creators

* refactor: simplify map message file name

* feat: add map tests

* fix: pre-commit

* fix: map tests

* fix: map scaling test

* fix: pre-commit

* [Automated] Updated coverage badge

* feat: remove thermal image parser and creator

* feat: remove thermal image tests

* fix: remove imports of the deleted objects

* [Automated] Updated coverage badge

* fix: create_map_message output type hint

* fix: pre-commit

* Adds FastSAM to example. (#58)

* Extend Segmentation Parser. (#59)

* Extend input shapes options.

* Small fix.

* Added PaddlePaddle Text detection model parser.

* Added Parser to __init__

* Fixes Shape Error bug

* pre-commit error

* [fix] precommit

* [Automated] Updated coverage badge

* Revision.

* Remove syntax error

Co-authored-by: KlemenSkrlj <47853619+klemen1999@users.noreply.github.com>

* Update depthai_nodes/ml/parsers/utils/ppocr.py

* Renamed some functions.

* Removed wrong  __init__ imports

* PaddleOCR model returns a sequence of classification probabilities for characters.

This PR adds a classification_sequence_message that returns a sequence of charecters corresponding to maximum probability in the sequence.
The message has optional parameters to remove duplicates and ignore background class indexes.

In addition this PR also adds a PaddleOCR Parser, which utilizes the classification_sequence message.

* [Automated] Updated coverage badge

* Lane detection parser. (#61)

* Add LaneDetectionParser.

* Add tests.

* Add LaneDetection parser to parsers.

* Example added.

* typo.

* Precommit fix.

* [Automated] Updated coverage badge

---------

Co-authored-by: GitHub Actions <actions@github.com>

* [Automated] Updated coverage badge

* PaddleOCR Parser and Classification Sequence message

* Added unit tests and modified some assertions

* pre-commit fixs

* Doc string fix

* [Automated] Updated coverage badge

* small fix

* [BUG] small bug fix where x_min was not <= x_max

* Added VehicleAttribte Parser and tests.

* Added docstrings.

* [Automated] Updated coverage badge

* Changed to MultiClassification parser

* [Automated] Updated coverage badge

* Updated docstrings and names

* fixed unit-test name

* Updated variable names.

* Refactor code.

* remove AgeGender Message.

* Refactor examples.

* refactor: separate visualizers and their utils

* refactor: remove mapping.py

* feat: add support for models with 2D map output

* fix: add FPS limit to avoid OAK-D Lite errors

* docs: update README.md

* fix: pre-commit

* [HOT-FIX] Update ppdet to return polygons.

* [Automated] Updated coverage badge

* Update depthai_nodes/ml/parsers/ppdet.py

Co-authored-by: KlemenSkrlj <47853619+klemen1999@users.noreply.github.com>

* dele one line

* Better error in MPPalmDetectionParser. (#72)

* Fix RVC4 order (#76)

* XFeat parser for Mono and Stereo. (#82)

* Tiling and TilesPatcher (#89)

* Release depthai-nodes v0.0.1 (#53)

* Added labeler to CI (#43)

* Updated requirements.txt (#44)

* Added coverage badge to README (#45)

* Added package version (#46)

* Update ci.yaml

* [Automated] Updated coverage badge

* Update README.md

* YOLOv8 kpts and seg example. (#48)

* Extend MPPalmDetection parser. (#49)

* Add support for different input sizes.

* Support extended palm detection in examples.

* Pre-commit fix.

* Add normalization.

* feat: add support for metric depth

* fix: broken depth tests

* fix: pre-commit

* fix: change default depth limit to float

* feat: add test for metric depth map

* Developer guide docs added. (#51)

* Support for metrics depth type. (#52)

* Hotfix: remove no-commit from pre-commit (#54)

* Added automatic PyPI publishing (#55)

---------

Co-authored-by: Martin Kozlovský <kozlovsky.m7@gmail.com>
Co-authored-by: GitHub Actions <actions@github.com>
Co-authored-by: Jaša Kerec <61207502+kkeroo@users.noreply.github.com>
Co-authored-by: jkbmrz <jakob.mraz@gmail.com>
Co-authored-by: jkbmrz <74824974+jkbmrz@users.noreply.github.com>

* initial commit with two nodes - Tiling and TilesPatcher | TilesPatcher implemented to work with dai.ImgDetections

* running pre-commit

* refactored locations of nodes into helpers + docstring added

---------

Co-authored-by: KlemenSkrlj <47853619+klemen1999@users.noreply.github.com>
Co-authored-by: Martin Kozlovský <kozlovsky.m7@gmail.com>
Co-authored-by: GitHub Actions <actions@github.com>
Co-authored-by: Jaša Kerec <61207502+kkeroo@users.noreply.github.com>
Co-authored-by: jkbmrz <jakob.mraz@gmail.com>
Co-authored-by: jkbmrz <74824974+jkbmrz@users.noreply.github.com>

* reformatted docstring in epytext (#92)

* reformatted docstring in epytext

* [Automated] Updated coverage badge

---------

Co-authored-by: GitHub Actions <actions@github.com>

* Minor release preparation. (#93)

* Bigger tolerance for classification scores.

* Improved MultiClassificationParser.

* Warnings in Keypoints and Segmentation parsers.

* Examples for new models.

* Adjusted tests due to tolerance change.

* Add XFeat Mono and Stereo to examples.

* XFeat demo instructions.

* Remove warp drawing in stereo mode.

* Version bump. (#97)

* Release: Fix merge conflicts (#99)

---------

Co-authored-by: Martin Kozlovský <kozlovsky.m7@gmail.com>
Co-authored-by: GitHub Actions <actions@github.com>
Co-authored-by: Jaša Kerec <61207502+kkeroo@users.noreply.github.com>
Co-authored-by: jkbmrz <jakob.mraz@gmail.com>
Co-authored-by: jkbmrz <74824974+jkbmrz@users.noreply.github.com>
Co-authored-by: aljazkonec1 <konec.aljaz1@gmail.com>
Co-authored-by: aljazkonec1 <53098513+aljazkonec1@users.noreply.github.com>
Co-authored-by: Jan Čuhel <79118988+HonzaCuhel@users.noreply.github.com>
Co-authored-by: Han Bui <74987252+spbui00@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@jkbmrz jkbmrz jkbmrz approved these changes

@tersekmatija tersekmatija Awaiting requested review from tersekmatija tersekmatija is a code owner

@kkeroo kkeroo Awaiting requested review from kkeroo kkeroo is a code owner

@klemen1999 klemen1999 Awaiting requested review from klemen1999 klemen1999 is a code owner

Assignees

@spbui00 spbui00

Labels
release New version release
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@spbui00 @jkbmrz @klemen1999