Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Added docs for message types #118

Merged
merged 3 commits into from
Oct 25, 2024
Merged
Changes from 2 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
156 changes: 144 additions & 12 deletions depthai_nodes/ml/messages/README.md
Original file line number Diff line number Diff line change
@@ -1,17 +1,149 @@
# Implement custom messages in depthai3
# Message Types

Existing depthai messages that inherit the Buffer class can be extended by:
Here are the custom message types that we introduce in this package. They are used as output types of the parsers.

```
import depthai as dai
**Table of Contents**

class MyCustomMessage(dai.<MessageOfChoice>):
def __init__(self):
dai.<MessageOfChoice>.__init__(self)
self.myField = 42
- [Classifications](#classifications)
- [Cluster](#cluster)
- [Clusters](#clusters)
- [ImgDetectionExtended](#imgdetectionextended)
- [ImgDetectionsExtended](#imgdetectionsextended)
- [Keypoint](#keypoint)
- [Keypoints](#keypoints)
- [Line](#line)
- [Lines](#lines)
- [Map2d](#map2d)
- [Prediction](#prediction)
- [Predictions](#predictions)
- [SegmentationMask](#segmentationmask)
- [SegmentationMaskSAM](#segmentationmaskssam)

def printMe(self):
print("My field is", self.myField)
```
## Classification

klemen1999 marked this conversation as resolved.
Show resolved Hide resolved
Note, this does NOT allow for passing to device and back.
Classification class for storing the classes and their respective scores.

### Attributes

- **classes** (list\[str\]): A list of classes.
- **scores** (NDArray\[np.float32\]): Corresponding probability scores.

## Cluster

Cluster class for storing a cluster.

### Attributes

- **label** (int): Label of the cluster.
- **points** (List\[dai.Point2f\]): List of points in the cluster.

## Clusters

Clusters class for storing clusters.

### Attributes

- **clusters** (List\[[Cluster](#cluster)\]): List of clusters.

## ImgDetectionExtended

A class for storing image detections in (x_center, y_center, width, height) format with additional angle and keypoints.

### Attributes

- **x_center** (float): The X coordinate of the center of the bounding box, relative to the input width.
- **y_center** (float): The Y coordinate of the center of the bounding box, relative to the input height.
- **width** (float): The width of the bounding box, relative to the input width.
- **height** (float): The height of the bounding box, relative to the input height.
- **angle** (float): The angle of the bounding box expressed in degrees.
- **confidence** (float): Confidence of the detection.
- **label** (int): Label of the detection.
- **keypoints** (List\[[Keypoint](#keypoint)\]): Keypoints of the detection.

## ImgDetectionsExtended

ImgDetectionsExtended class for storing image detections with keypoints.

### Attributes

- **detections** (List\[[ImgDetectionExtended](#imgdetectionextended)\]): Image detections with keypoints.
- **masks** (np.ndarray): The segmentation masks of the image. All masks are stored in a single numpy array.

## Keypoint

Keypoint class for storing a keypoint.

### Attributes

- **x** (float): X coordinate of the keypoint, relative to the input height.
- **y** (float): Y coordinate of the keypoint, relative to the input width.
- **z** (Optional\[float\]): Z coordinate of the keypoint.
- **confidence** (Optional\[float\]): Confidence of the keypoint.

## Keypoints

Keypoints class for storing keypoints.

### Attributes

- **keypoints** (List\[[Keypoint](#keypoint)\]): List of Keypoint objects, each representing a keypoint.

## Line

Line class for storing a line.

### Attributes

- **start_point** (dai.Point2f): Start point of the line with x and y coordinates.
- **end_point** (dai.Point2f): End point of the line with x and y coordinates.
- **confidence** (float): Confidence of the line.

## Lines

Lines class for storing lines.

### Attributes

- **lines** (List\[[Line](#line)\]): List of detected lines.

## Map2D

Map2D class for storing a 2D map of floats.

### Attributes

- **map** (NDArray\[np.float32\]): 2D map.
- **width** (int): 2D Map width.
- **height** (int): 2D Map height.

## Prediction

Prediction class for storing a prediction.

### Attributes

- **prediction** (float): The predicted value.

## Predictions

Predictions class for storing predictions.

### Attributes

- **predictions** (List\[[Prediction](#prediction)\]): List of predictions.

## SegmentationMask

SegmentationMask class for a single- or multi-object segmentation mask. Background is represented with "-1" and foreground classes with non-negative integers.

### Attributes

- **mask** (NDArray\[np.int8\]): Segmentation mask.

## SegmentationMasksSAM

SegmentationMasksSAM class for storing segmentation masks.

### Attributes

- **masks** (np.ndarray): Mask coefficients.
Loading