From 2f51030797a9d449e7c80455e68af0e4b3e8e0c8 Mon Sep 17 00:00:00 2001 From: klemen1999 Date: Fri, 25 Oct 2024 14:07:20 +0200 Subject: [PATCH 1/3] Added docs for message types --- depthai_nodes/ml/messages/README.md | 132 +++++++++++++++++++++++++--- 1 file changed, 120 insertions(+), 12 deletions(-) diff --git a/depthai_nodes/ml/messages/README.md b/depthai_nodes/ml/messages/README.md index 1a97da12..5f63b235 100644 --- a/depthai_nodes/ml/messages/README.md +++ b/depthai_nodes/ml/messages/README.md @@ -1,17 +1,125 @@ -# Implement custom messages in depthai3 +# Message Types -Existing depthai messages that inherit the Buffer class can be extended by: +Here are the custom message types that we introduce in this package. They are used as output types of the parsers. -``` -import depthai as dai +**Table of Contents** +- [Classifications](#classifications) +- [Cluster](#cluster) +- [Clusters](#clusters) +- [ImgDetectionExtended](#imgdetectionextended) +- [ImgDetectionsExtended](#imgdetectionsextended) +- [Keypoint](#keypoint) +- [Keypoints](#keypoints) +- [Line](#line) +- [Lines](#lines) +- [Map2d](#map2d) +- [Prediction](#prediction) +- [Predictions](#predictions) +- [SegmentationMask](#segmentationmask) +- [SegmentationMaskSAM](#segmentationmaskssam) -class MyCustomMessage(dai.): - def __init__(self): - dai..__init__(self) - self.myField = 42 +## Classification +Classification class for storing the classes and their respective scores. - def printMe(self): - print("My field is", self.myField) -``` +### Attributes +- **classes** (list[str]): A list of classes. +- **scores** (NDArray[np.float32]): Corresponding probability scores. -Note, this does NOT allow for passing to device and back. + +## Cluster +Cluster class for storing a cluster. + +### Attributes +- **label** (int): Label of the cluster. +- **points** (List[dai.Point2f]): List of points in the cluster. + + +## Clusters +Clusters class for storing clusters. + +### Attributes +- **clusters** (List[[Cluster](#cluster)]): List of clusters. + + +## ImgDetectionExtended +A class for storing image detections in (x_center, y_center, width, height) format with additional angle and keypoints. + +### Attributes +- **x_center** (float): The X coordinate of the center of the bounding box, relative to the input width. +- **y_center** (float): The Y coordinate of the center of the bounding box, relative to the input height. +- **width** (float): The width of the bounding box, relative to the input width. +- **height** (float): The height of the bounding box, relative to the input height. +- **angle** (float): The angle of the bounding box expressed in degrees. +- **confidence** (float): Confidence of the detection. +- **label** (int): Label of the detection. +- **keypoints** (List[[Keypoint](#keypoint)]): Keypoints of the detection. + +## ImgDetectionsExtended +ImgDetectionsExtended class for storing image detections with keypoints. + +### Attributes +- **detections** (List[[ImgDetectionExtended](#imgdetectionextended)]): Image detections with keypoints. +- **masks** (np.ndarray): The segmentation masks of the image. All masks are stored in a single numpy array. + +## Keypoint +Keypoint class for storing a keypoint. + +### Attributes +- **x** (float): X coordinate of the keypoint, relative to the input height. +- **y** (float): Y coordinate of the keypoint, relative to the input width. +- **z** (Optional[float]): Z coordinate of the keypoint. +- **confidence** (Optional[float]): Confidence of the keypoint. + +## Keypoints +Keypoints class for storing keypoints. + +### Attributes +- **keypoints** (List[[Keypoint](#keypoint)]): List of Keypoint objects, each representing a keypoint. + +## Line +Line class for storing a line. + +### Attributes +- **start_point** (dai.Point2f): Start point of the line with x and y coordinates. +- **end_point** (dai.Point2f): End point of the line with x and y coordinates. +- **confidence** (float): Confidence of the line. + +## Lines +Lines class for storing lines. + +### Attributes +- **lines** (List[[Line](#line)]): List of detected lines. + +## Map2D +Map2D class for storing a 2D map of floats. + +### Attributes +- **map** (NDArray[np.float32]): 2D map. +- **width** (int): 2D Map width. +- **height** (int): 2D Map height. + +## Prediction +Prediction class for storing a prediction. + +### Attributes +- **prediction** (float): The predicted value. + +## Predictions +Predictions class for storing predictions. + +### Attributes +- **predictions** (List[[Prediction](#prediction)]): List of predictions. + + +## SegmentationMask +SegmentationMask class for a single- or multi-object segmentation mask. Background is represented with "-1" and foreground classes with non-negative integers. + +### Attributes +- **mask** (NDArray[np.int8]): Segmentation mask. + + +## SegmentationMasksSAM +SegmentationMasksSAM class for storing segmentation masks. + +### Attributes +- **masks** (np.ndarray): Mask coefficients. From 57fe8bc48203dcc469ed0cd369c79ac93d9a835f Mon Sep 17 00:00:00 2001 From: klemen1999 Date: Fri, 25 Oct 2024 14:07:49 +0200 Subject: [PATCH 2/3] Added docs for message types --- depthai_nodes/ml/messages/README.md | 52 +++++++++++++++++++++-------- 1 file changed, 38 insertions(+), 14 deletions(-) diff --git a/depthai_nodes/ml/messages/README.md b/depthai_nodes/ml/messages/README.md index 5f63b235..46f07cc1 100644 --- a/depthai_nodes/ml/messages/README.md +++ b/depthai_nodes/ml/messages/README.md @@ -3,6 +3,7 @@ Here are the custom message types that we introduce in this package. They are used as output types of the parsers. **Table of Contents** + - [Classifications](#classifications) - [Cluster](#cluster) - [Clusters](#clusters) @@ -19,32 +20,37 @@ Here are the custom message types that we introduce in this package. They are us - [SegmentationMaskSAM](#segmentationmaskssam) ## Classification + Classification class for storing the classes and their respective scores. ### Attributes -- **classes** (list[str]): A list of classes. -- **scores** (NDArray[np.float32]): Corresponding probability scores. +- **classes** (list\[str\]): A list of classes. +- **scores** (NDArray\[np.float32\]): Corresponding probability scores. ## Cluster + Cluster class for storing a cluster. ### Attributes -- **label** (int): Label of the cluster. -- **points** (List[dai.Point2f]): List of points in the cluster. +- **label** (int): Label of the cluster. +- **points** (List\[dai.Point2f\]): List of points in the cluster. ## Clusters + Clusters class for storing clusters. ### Attributes -- **clusters** (List[[Cluster](#cluster)]): List of clusters. +- **clusters** (List\[[Cluster](#cluster)\]): List of clusters. ## ImgDetectionExtended + A class for storing image detections in (x_center, y_center, width, height) format with additional angle and keypoints. ### Attributes + - **x_center** (float): The X coordinate of the center of the bounding box, relative to the input width. - **y_center** (float): The Y coordinate of the center of the bounding box, relative to the input height. - **width** (float): The width of the bounding box, relative to the input width. @@ -52,74 +58,92 @@ A class for storing image detections in (x_center, y_center, width, height) form - **angle** (float): The angle of the bounding box expressed in degrees. - **confidence** (float): Confidence of the detection. - **label** (int): Label of the detection. -- **keypoints** (List[[Keypoint](#keypoint)]): Keypoints of the detection. +- **keypoints** (List\[[Keypoint](#keypoint)\]): Keypoints of the detection. ## ImgDetectionsExtended + ImgDetectionsExtended class for storing image detections with keypoints. ### Attributes -- **detections** (List[[ImgDetectionExtended](#imgdetectionextended)]): Image detections with keypoints. + +- **detections** (List\[[ImgDetectionExtended](#imgdetectionextended)\]): Image detections with keypoints. - **masks** (np.ndarray): The segmentation masks of the image. All masks are stored in a single numpy array. ## Keypoint + Keypoint class for storing a keypoint. ### Attributes + - **x** (float): X coordinate of the keypoint, relative to the input height. - **y** (float): Y coordinate of the keypoint, relative to the input width. -- **z** (Optional[float]): Z coordinate of the keypoint. -- **confidence** (Optional[float]): Confidence of the keypoint. +- **z** (Optional\[float\]): Z coordinate of the keypoint. +- **confidence** (Optional\[float\]): Confidence of the keypoint. ## Keypoints + Keypoints class for storing keypoints. ### Attributes -- **keypoints** (List[[Keypoint](#keypoint)]): List of Keypoint objects, each representing a keypoint. + +- **keypoints** (List\[[Keypoint](#keypoint)\]): List of Keypoint objects, each representing a keypoint. ## Line + Line class for storing a line. ### Attributes + - **start_point** (dai.Point2f): Start point of the line with x and y coordinates. - **end_point** (dai.Point2f): End point of the line with x and y coordinates. - **confidence** (float): Confidence of the line. ## Lines + Lines class for storing lines. ### Attributes -- **lines** (List[[Line](#line)]): List of detected lines. + +- **lines** (List\[[Line](#line)\]): List of detected lines. ## Map2D + Map2D class for storing a 2D map of floats. ### Attributes -- **map** (NDArray[np.float32]): 2D map. + +- **map** (NDArray\[np.float32\]): 2D map. - **width** (int): 2D Map width. - **height** (int): 2D Map height. ## Prediction + Prediction class for storing a prediction. ### Attributes + - **prediction** (float): The predicted value. ## Predictions + Predictions class for storing predictions. ### Attributes -- **predictions** (List[[Prediction](#prediction)]): List of predictions. +- **predictions** (List\[[Prediction](#prediction)\]): List of predictions. ## SegmentationMask + SegmentationMask class for a single- or multi-object segmentation mask. Background is represented with "-1" and foreground classes with non-negative integers. ### Attributes -- **mask** (NDArray[np.int8]): Segmentation mask. +- **mask** (NDArray\[np.int8\]): Segmentation mask. ## SegmentationMasksSAM + SegmentationMasksSAM class for storing segmentation masks. ### Attributes + - **masks** (np.ndarray): Mask coefficients. From a8f637447f6830f4de3b851cfc2d45494dcddfe5 Mon Sep 17 00:00:00 2001 From: KlemenSkrlj <47853619+klemen1999@users.noreply.github.com> Date: Fri, 25 Oct 2024 14:30:20 +0200 Subject: [PATCH 3/3] Update depthai_nodes/ml/messages/README.md Co-authored-by: jkbmrz <74824974+jkbmrz@users.noreply.github.com> --- depthai_nodes/ml/messages/README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/depthai_nodes/ml/messages/README.md b/depthai_nodes/ml/messages/README.md index 46f07cc1..7761a50c 100644 --- a/depthai_nodes/ml/messages/README.md +++ b/depthai_nodes/ml/messages/README.md @@ -19,7 +19,7 @@ Here are the custom message types that we introduce in this package. They are us - [SegmentationMask](#segmentationmask) - [SegmentationMaskSAM](#segmentationmaskssam) -## Classification +## Classifications Classification class for storing the classes and their respective scores.