Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support for runtime voice input to showcase █████ #110

Open
jbilcke-hf opened this issue Sep 10, 2024 · 0 comments
Open

Support for runtime voice input to showcase █████ #110

jbilcke-hf opened this issue Sep 10, 2024 · 0 comments
Assignees
Labels
huggingface See the "Hugging Face" milestone for more info
Milestone

Comments

@jbilcke-hf
Copy link
Owner

(note: this feature is unrelated to the existing ai assistant or the existing assistant voice control feature)

  1. add a microphone input to the embedded player (can be push to speak or continuous)
  2. inject the user recordings (mp3 or wav) as ClapSegmentCategory.DIALOGUE segments into the current timeline loop
  3. edit the sample OpenClap file (see Create a demo OpenClap project to showcase █████ #109) to add a ClapSegmentCategory.PHENOMENON (can optionally be tied to a "user" entity) which will process user dialogue using by emitting new ClapSegmentCategory.DIALOGUE segments (tied to the avatar entity).

This ClapSegmentCategory.PHENOMENON must use the █████* workflow.

  1. update the Simulator service to process ClapSegmentCategory.PHENOMENON in a loop

Note: I don't think we need to use ClapSegmentCategory.EVENT here, since our events already nicely fit the concept of ClapSegmentCategory.DIALOGUE

  • redacted
@jbilcke-hf jbilcke-hf added the huggingface See the "Hugging Face" milestone for more info label Sep 10, 2024
@jbilcke-hf jbilcke-hf added this to the Hugging Face milestone Sep 10, 2024
@jbilcke-hf jbilcke-hf self-assigned this Sep 10, 2024
@jbilcke-hf jbilcke-hf changed the title Support for runtime voice input Support for runtime voice input to showcase █████ Sep 10, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
huggingface See the "Hugging Face" milestone for more info
Projects
Status: No status
Development

No branches or pull requests

1 participant