Add keypoint-detection task to Hub #870
Conversation
merveenoyan
requested review from
osanseviero,
SBrandeis,
gary149,
Wauplin,
julien-c and
pcuenca
as code owners
|
also cc @NielsRogge |
| @@ -131,6 +131,7 @@ export const TASKS_MODEL_LIBRARIES: Record<PipelineType, ModelLibraryKey[]> = { | |||
| "image-to-image": ["diffusers", "transformers", "transformers.js"], | |||
| "image-to-text": ["transformers", "transformers.js"], | |||
| "image-to-video": ["diffusers"], | |||
| "keypoint-detection": ["transformers"], | |||
There was a problem hiding this comment.
What does this mean? Cause there's no keypoint detection pipeline in Transformers yet
There was a problem hiding this comment.
this is not about pipelines, it's to show sort of which libraries support this task, e.g. in tasks
we currently have SuperPoint and will soon have ViTPose in transformers so we do support it
There was a problem hiding this comment.
we currently have SuperPoint and will soon have ViTPose
I think we should tag those models then before the PR is merged
There was a problem hiding this comment.
I have opened PRs to 30-40 models and some are merged already I think, see
also Sapiens model will have a lot of keypoint models soon (it was one unified model repo, I sent authors a script to automate separation)
There was a problem hiding this comment.
Nice (I opened https://github.com/huggingface-internal/moon-landing/issues/11015 internally )
There was a problem hiding this comment.
I have opened PRs to 30-40 models and some are merged
Yes, I see 4 models now 👍
packages/tasks/src/pipelines.ts
Outdated
| "keypoint-detection": { | ||
| name: "Keypoint Detection", | ||
| modality: "cv", | ||
| color: "red", | ||
| hideInDatasets: true, | ||
| }, |
There was a problem hiding this comment.
Should it be a subtask of object-detection instead? There's already a face-detection subtask, for example.
There was a problem hiding this comment.
I think it's not. object detection is where you detect bounding boxes, but in this one outputs look like these:
There was a problem hiding this comment.
Agree with @merveenoyan, keypoint-detection can be seen as a task at the same level of object detection, it's not a subtask
| @@ -131,6 +131,7 @@ export const TASKS_MODEL_LIBRARIES: Record<PipelineType, ModelLibraryKey[]> = { | |||
| "image-to-image": ["diffusers", "transformers", "transformers.js"], | |||
| "image-to-text": ["transformers", "transformers.js"], | |||
| "image-to-video": ["diffusers"], | |||
| "keypoint-detection": ["transformers"], | |||
There was a problem hiding this comment.
we currently have SuperPoint and will soon have ViTPose
I think we should tag those models then before the PR is merged
| name: "Keypoint Detection", | ||
| modality: "cv", | ||
| color: "red", | ||
| hideInDatasets: true, |
There was a problem hiding this comment.
Any datasets for this on the Hub?
There was a problem hiding this comment.
actually there seems to be a lot, opening PRs now
https://huggingface.co/datasets?sort=trending&search=mediapipe
https://huggingface.co/datasets?sort=trending&search=keypoint
https://huggingface.co/datasets?sort=trending&search=pose
There was a problem hiding this comment.
just set to false
| @@ -131,6 +131,7 @@ export const TASKS_MODEL_LIBRARIES: Record<PipelineType, ModelLibraryKey[]> = { | |||
| "image-to-image": ["diffusers", "transformers", "transformers.js"], | |||
| "image-to-text": ["transformers", "transformers.js"], | |||
| "image-to-video": ["diffusers"], | |||
| "keypoint-detection": ["transformers"], | |||
There was a problem hiding this comment.
Nice (I opened https://github.com/huggingface-internal/moon-landing/issues/11015 internally )
|
lol I opened too many PRs Hub thinks I'm spamming |
Co-authored-by: Merve Noyan <mervenoyan@Merve-MacBook-Pro.local> Co-authored-by: Pedro Cuenca <pedro@huggingface.co>
This reverts commit 678eba8.
I have opened PR to keypoint detection because we already have around 30 keypoint detection models on Hub (currently opening PRs to change pipeline tag), one model in transformers, and this doesn't fall under any other task.