A fun and interactive project that combines the power of SwiftUI (MacOS), Adobe Character Animator, and Arduino (ESP32 BLE) to switch between "Active" and "Idle" states based on human detection.
The Pumpkin King is a unique combination of software and hardware that detects human presence and switches modes in a SwiftUI application. When a person is detected, it transitions from an "Idle" state to an "Active" state, answering questions with the power of OpenAI and Eleven Labs.
Video of the beam/break detector
- SwiftUI: For creating a visually appealing and responsive iOS application.
- Arduino: Used for the hardware component to detect human presence.
- BLE (Bluetooth Low Energy): For seamless communication between the hardware and software components.
- Adobe Character Animator: Creates a face that you can project onto a blow-up pumpkin.
- OpenAI Whisper: Used to take the audio recording of the user's question and turn it into text.
- OpenAI Completions: Used to generate a response to the user's question in the style of the pumpkin king.
- Eleven Labs: Used to generate an audio response in a custom voice that is sure to spook the kids.
- Clone the repository:
git clone [repository_link]
. - Open
PumpkinKing.xcodeproj
in Xcode. - Build and run the project on a Mac or an iOS device.
- Goto the preferences in the main menu and enter in your API keys
- Setup your Arduino board This project assumes HiLetgo 0.96" ESP32 OLED (Amazon Non-Affiliate Link).
- Connect the required sensors and components Tutorial for Photoresistor, Laser Diode (Amazon Non-Affiliate Link, Photoresistor (Amazon Non-Affiliate Link).
- Open the Arduino IDE and load the provided
.ino
file. (Configure the boards and libraries needed) - Upload the code to your Arduino board.
I've uploaded the 3D models to Thangs these models make it easier to setup the beam break sensor outside.
- Start the SwiftUI application.
- Power on the Arduino setup.
- Approach the Arduino sensor.
- Watch as the SwiftUI application transitions from "Idle" to "Active" mode, playing a synthesized response.
Pull requests are welcome. For major changes, please open an issue first to discuss what you would like to change. Please make sure to update tests as appropriate.
Pumpkin King Human Detector by David Bates is licensed under Attribution 4.0 International