Skip to content

Commit

Permalink
Deploying to gh-pages from @ 5914f48 🚀
Browse files Browse the repository at this point in the history
  • Loading branch information
tbirdso committed Sep 6, 2024
1 parent 6205bb5 commit f1376a0
Showing 1 changed file with 1 addition and 1 deletion.
2 changes: 1 addition & 1 deletion aggregate_metadata.json
Original file line number Diff line number Diff line change
Expand Up @@ -1629,7 +1629,7 @@
"workdir": "holohub_bin"
}
},
"readme": "# \ud83d\udcf7 Holoscan SAM2\n\nThis application demonstrates how to run [SAM2](https://github.com/facebookresearch/segment-anything-2) models on live video feed with the possibility of changing query points in real-time.\n\nThe application currently uses a single query point as a foreground point that moves on the perimeter of a circle with a configured angular speed. \nThe models returns three masks, the best mask is selected based on the model scores. For visualization, two options exist. Select between \"logits\" or \"masks\". \n- \"logits\": predictions of the network, mapped onto a colorscale that matches matplotlib.pyplot's \"viridis\" \n- \"masks\": binarized predictions\n\nSAM2, recently announced by Meta, is the next iteration of the Segment Anything Model (SAM). This new version expands upon its predecessor by adding the capability to segment both videos and images.\nThis sample application wraps the ImageInference class, and applies it on a live video feed.\n\nNote: This demo currently uses [\"sam2_hiera_l.yaml\"](https://huggingface.co/Efficient-Large-Model/Llama-3-VILA1.5-8b-AWQ), but any of the sam2 models work. You only need to adjust [segment_one_thing.yaml](./segment_one_thing.yaml).\n\n## \u2699\ufe0f Setup Instructions\nThe app defaults to using the video device at `/dev/video0`\n\nTo debug if this is the correct device download `v4l2-ctl`:\n```bash\nsudo apt-get install v4l-utils\n```\nTo check for your devices run:\n```bash\nv4l2-ctl --list-devices\n```\nThis command will output something similar to this:\n```bash\nNVIDIA Tegra Video Input Device (platform:tegra-camrtc-ca):\n /dev/media0\n\nvi-output, lt6911uxc 2-0056 (platform:tegra-capture-vi:0):\n /dev/video0\n\nDummy video device (0x0000) (platform:v4l2loopback-000):\n /dev/video3\n```\nDetermine your desired video device and edit the source device in [segment_one_thing.yaml](segment_one_thing.yaml)\n\n## \ud83d\ude80 Build and Run Instructions\n\n### ARM64 and x86\nThis application uses a custom Dockerfile based on a pytorch container.\nBuild and run the application using\n```sh\n ./dev_container build_and_run sam2 --docker_file applications/sam2/Dockerfile --img holohub:sam2\n``` \nOr first build the container, then launch it and run.\n\n```sh\n ./dev_container build --docker_file applications/sam2/Dockerfile --img holohub:sam2\n```\n```sh\n./dev_container launch --img holohub:sam\n```\n```sh\npython holohub/applications/sam2/segment_one_thing.py\n```\n\n### x86 only\nIf you are only using an x86 system, you may use a Dockerfile based on the Holoscan container. Replace the [Dockerfile](./Dockerfile) with this [alternative Dockerfile](./alternative_docker/Dockerfile). \nThen, from the Holohub main directory run the following command:\n```sh\n./dev_container build_and_run sam2\n```\n\nAlternatively build and run: \n```bash\n./dev_container vscode sam2\n```\nRun the application in debug mode from vscode, or execute it by\n```sh\npython applications/sam2/segment_one_thing.py\n```\n\n\nYou can choose to output \"logits\" or \"masks\" in the configuration of the postprocessor and holoviz operator [segment_one_thing.yaml](segment_one_thing.yaml)\n\n## \ud83d\udcbb Supported Hardware\n- x86 w/ dGPU\n- IGX devKit w/ dGPU\n\n## \ud83d\ude4c Acknowledgements\n- Meta, [SAM2](https://github.com/facebookresearch/segment-anything-2): for providing these models and inference infrastructure\n",
"readme": "# \ud83d\udcf7 Holoscan SAM2\n\nThis application demonstrates how to run [SAM2](https://github.com/facebookresearch/segment-anything-2) models on live video feed with the possibility of changing query points in real-time.\n\n<p align=\"center\">\n <img src=\"./holohub-sam2.gif\" alt=\"Holohub Sam2\">\n</p>\n\nThe application currently uses a single query point as a foreground point that moves on the perimeter of a circle with a configured angular speed. \nThe models returns three masks, the best mask is selected based on the model scores. For visualization, two options exist. Select between \"logits\" or \"masks\". \n- \"logits\": predictions of the network, mapped onto a colorscale that matches matplotlib.pyplot's \"viridis\" \n- \"masks\": binarized predictions\n\nSAM2, recently announced by Meta, is the next iteration of the Segment Anything Model (SAM). This new version expands upon its predecessor by adding the capability to segment both videos and images.\nThis sample application wraps the ImageInference class, and applies it on a live video feed.\n\nNote: This demo currently uses [\"sam2_hiera_l.yaml\"](https://huggingface.co/Efficient-Large-Model/Llama-3-VILA1.5-8b-AWQ), but any of the sam2 models work. You only need to adjust [segment_one_thing.yaml](./segment_one_thing.yaml).\n\n## \u2699\ufe0f Setup Instructions\nThe app defaults to using the video device at `/dev/video0`\n\nTo debug if this is the correct device download `v4l2-ctl`:\n```bash\nsudo apt-get install v4l-utils\n```\nTo check for your devices run:\n```bash\nv4l2-ctl --list-devices\n```\nThis command will output something similar to this:\n```bash\nNVIDIA Tegra Video Input Device (platform:tegra-camrtc-ca):\n /dev/media0\n\nvi-output, lt6911uxc 2-0056 (platform:tegra-capture-vi:0):\n /dev/video0\n\nDummy video device (0x0000) (platform:v4l2loopback-000):\n /dev/video3\n```\nDetermine your desired video device and edit the source device in [segment_one_thing.yaml](segment_one_thing.yaml)\n\n## \ud83d\ude80 Build and Run Instructions\n\n### ARM64 and x86\nThis application uses a custom Dockerfile based on a pytorch container.\nBuild and run the application using\n```sh\n ./dev_container build_and_run sam2 --docker_file applications/sam2/Dockerfile --img holohub:sam2\n``` \nOr first build the container, then launch it and run.\n\n```sh\n ./dev_container build --docker_file applications/sam2/Dockerfile --img holohub:sam2\n```\n```sh\n./dev_container launch --img holohub:sam\n```\n```sh\npython holohub/applications/sam2/segment_one_thing.py\n```\n\n### x86 only\nIf you are only using an x86 system, you may use a Dockerfile based on the Holoscan container. Replace the [Dockerfile](./Dockerfile) with this [alternative Dockerfile](./alternative_docker/Dockerfile). \nThen, from the Holohub main directory run the following command:\n```sh\n./dev_container build_and_run sam2\n```\n\nAlternatively build and run: \n```bash\n./dev_container vscode sam2\n```\nRun the application in debug mode from vscode, or execute it by\n```sh\npython applications/sam2/segment_one_thing.py\n```\n\n\nYou can choose to output \"logits\" or \"masks\" in the configuration of the postprocessor and holoviz operator [segment_one_thing.yaml](segment_one_thing.yaml)\n\n## \ud83d\udcbb Supported Hardware\n- x86 w/ dGPU\n- IGX devKit w/ dGPU\n\n## \ud83d\ude4c Acknowledgements\n- Meta, [SAM2](https://github.com/facebookresearch/segment-anything-2): for providing these models and inference infrastructure\n",
"application_name": "sam2",
"source_folder": "applications",
"build_and_run": "./dev_container build_and_run sam2 --language python"
Expand Down

0 comments on commit f1376a0

Please sign in to comment.