Very fast capture
If you want to increase performance in Windows 10/11, be sure to disable the Windows Hardware GPU scheduler.
Firefly Luciferin is written in Java using AWT's Robot class, Robots is the only way to screen capture using Java (without exotic libs).
With that thing you can almost never get above 5FPS (in 4K) because as you can see in the OpenJDK implementation, robot.createScreenCapture() is synchronized and the native calls it uses are pretty slow.
Fast enough for screenshots but too slow for screen capture. If one Robot can capture at about 5FPS, what about 2 Robots in a multi threaded producer/consumer environment?
What about adding GPU Hardware Acceleration to the mix?
Firefly Luciferin is a very optimized software and it has nearly no impact on your system performance.
By default Firefly Luciferin captures at 30FPS, if you want you can unlock the framerate.
If you are using a slow microcontroller, capturing at a very high framerate will not help. If you right click the tray icon and then click FPS,
you can see the output as shown in the image below. In that output you can see how fast the software is capturing the screen (producing)
and how fast your microcontroller is able to process (consume) this data.
Increase baud rate accordingly to your microcontroller's serial speed, 500K is generally more than enough for 30FPS. Producers framerate should not exceed the consuming one, all data that is not consumed in time, is lost.
Higher baud rate allows for higher framerates when driving LEDs using USB cable but it can cause flickering.
Producer framerate should not exceed the consuming one, all data that is not consumed in time, is lost and may cause stuttering.
A good WiFi signal strength is required for wireless stream.
Wireless stream using UDP stream is faster than USB cable and it is generally recommended when using the FULL firmware.
Please check what is the best GPIO for your use case for increased performance.
| 100 LEDs | FPS |
|---|---|
| WiFi UDP GPIO2 | 240 FPS |
| WiFi UDP GPIO3 | 250 FPS |
| WiFi UDP GPIO5 | 220 FPS |
| USB GPIO2 | 85 FPS |
| 300 LEDs | FPS |
|---|---|
| WiFi UDP GPIO2 | 70 FPS |
| WiFi UDP GPIO3 | 100 FPS |
| WiFi UDP GPIO5 | 60 FPS |
| USB GPIO2 | 30 FPS |
| 500 LEDs | FPS |
|---|---|
| WiFi UDP GPIO2 | 45 FPS |
| WiFi UDP GPIO3 | 60 FPS |
| WiFi UDP GPIO5 | 40 FPS |
| USB GPIO2 | 20 FPS |
GPIO in use does not affect performance on ESP32.
| 100 LEDs | FPS |
|---|---|
| WiFi UDP | 300 FPS |
| USB | 120 FPS |
| 300 LEDs | FPS |
|---|---|
| WiFi UDP | 90 FPS |
| USB | 50 FPS |
| 500 LEDs | FPS |
|---|---|
| WiFi UDP | 60 FPS |
| USB | 30 FPS |
Screen capturing is pretty slow and very CPU intensive in Windows systems (Linux is much more efficient in this regard), for this reason I wrapped the Windows GDI32 C class using Java Native Access to access Windows hardware acceleration.
This API captures and delivers captured frames in GPU memory.
If you are running Windows 8 or Windows 10 you can use Desktop Duplication API (DDUPL), it's the fastest implementation yet, no lag,
no stutter, very small usage of resources. DDUPL is accessed via JNA using the GStreamer bindings for Java.