Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add link to early prototype of the API #4

Open
wants to merge 1 commit into
base: main
Choose a base branch
from
Open

Add link to early prototype of the API #4

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 4 additions & 1 deletion README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -24,4 +24,7 @@ There are 5 potential use cases we could consider working on, intertwined but su

- Exposing native computer vision building blocks and/or components, in a way that leverages native processing of video frames before sending the pre-processed video frames (and associated data) into the web app (either into the GPU or CPU), perhaps leveraging a library like [Khronos' OpenVX](https://www.khronos.org/openvx/). (In fact, one way this could be thought about is by working with Khronos to define WebVX.) Being able to leverage optimized platform capabilities for doing well-known basic algorithms (image pyramids, simple feature extraction, image conversion, blur, etc) could speed up in-app CV, and also allow some of the effects that might be done in the synchronous GPU case to be done faster. Like the WebRTC discussion, if we wanted to pursue this, we would want to do that elsewhere, but the scenario has been brought up multiple times, so we should document, summarize and record it it here. Documented in [webvx.md](webvx.md).

The first two are the focus of this repository, and share a common need to have the available cameras exposed and the developer request access to them. They can likely be done in the same API (e.g., request cameras, direct data to CPU and/or GPU, guarantee that if the camera is synch with rAF that the data will be available before rAF and make this known if so), but are separable if we only want to tackle one first.
The first two are the focus of this repository, and share a common need to have the available cameras exposed and the developer request access to them. They can likely be done in the same API (e.g., request cameras, direct data to CPU and/or GPU, guarantee that if the camera is synch with rAF that the data will be available before rAF and make this known if so), but are separable if we only want to tackle one first.

## Early prototyping efforts
Early prototyping efforts to provide synchronous access to video frames from the camera in GPU memory can be found in the following [repository](https://github.com/bialpio/webxr-raw-camera-access). Initial prototype implementation is happening on Chrome for Android, with the goal of discovering potential issues with the implementation and API ergonomics.