BYOC: add streaming

[ad-astra-video]

What does this pull request do? Explain your changes. (required)

Adds configurable streaming for BYOC entrypoint to go-livepeer. Uses trickle protocol to handle streaming for similar entrypoints and outputs from go-livepeer as live-video-to-video.

Streams can be any or a mix of the following:

• video ingress via WHIP (with Gateway) or RTMP (with MediaMTX)
• video egress via RTMP (with MediaMTX) or WHEP (with mediamtx)
• SSE data output

Control and Events channels are created for every stream.

Streams are created with a POST request to /ai/stream/start that will start the stream and reserve the capacity with an Orchestrator that is providing the BYOC capability. If video ingress is enabled, the client should then start a stream with WHIP or RTMP to the provided ingress URLs provided in the response. URLs for egress video, data, updates (control) and events are also included in the response as well as the stream_id. The stream_id is an integral part of the URLs provided to interact with the stream and is combined with a provided stream name in the /ai/stream/start request.

Streams are stopped with a POST request to /ai/stream/stop. Orchestrators and Gateways track payment balance and the Gateway adjusts to the Orchestrators provided balance in new JobTokens provided at each payment interval every minute. Orchestrators will shutdown a stream when payment balance is zero.

Specific updates (required)

• Add job_stream.go and job_stream_test.go
• refactor job_rpc.go to reuse stream setup where made sense
• updates go routines to ignore to enable tests to pass in common/testutil.go.

How did you test each of these updates (required)

Used byoc-stream to test end to end: https://github.com/ad-astra-video/livepeer-app-pipelines/tree/main/byoc-stream
Added tests to job_stream_test.go and some additional tests to job_rpc_test.go.

Does this pull request close any open issues?

Checklist:

• Read the contribution guide
• make runs successfully
• All tests in ./test.sh pass
• README and other documentation updated
• Pending changelog updated

[codecov]

Codecov Report

❌ Patch coverage is 48.22547% with 744 lines in your changes missing coverage. Please review.
✅ Project coverage is 32.88168%. Comparing base (aca2275) to head (f8189f0).
⚠️ Report is 1 commits behind head on master.

Files with missing lines	Patch %	Lines
server/job_stream.go	55.44768%	338 Missing and 75 partials ⚠️
server/job_rpc.go	50.18450%	105 Missing and 30 partials ⚠️
server/ai_live_video.go	30.55556%	69 Missing and 6 partials ⚠️
core/external_capabilities.go	2.94118%	66 Missing ⚠️
core/livepeernode.go	0.00000%	35 Missing ⚠️
common/testutil.go	0.00000%	10 Missing ⚠️
server/rpc.go	0.00000%	4 Missing ⚠️
core/ai_orchestrator.go	0.00000%	3 Missing ⚠️
server/ai_mediaserver.go	72.72727%	2 Missing and 1 partial ⚠️

Additional details and impacted files

@@                 Coverage Diff                 @@
##              master       #3727         +/-   ##
===================================================
+ Coverage   31.52544%   32.88168%   +1.35624%     
===================================================
  Files            159         160          +1     
  Lines          47901       40223       -7678     
===================================================
- Hits           15101       13226       -1875     
+ Misses         31911       25956       -5955     
- Partials         889        1041        +152     

Files with missing lines	Coverage Δ
server/ai_process.go	2.01913% <ø> (+0.35938%)	⬆️
core/ai_orchestrator.go	30.53735% <0.00000%> (+1.16817%)	⬆️
server/ai_mediaserver.go	9.53912% <72.72727%> (+2.88383%)	⬆️
server/rpc.go	68.86228% <0.00000%> (+1.13612%)	⬆️
common/testutil.go	16.39344% <0.00000%> (+0.26441%)	⬆️
core/livepeernode.go	51.08696% <0.00000%> (-9.13403%)	⬇️
core/external_capabilities.go	29.29293% <2.94118%> (-57.54918%)	⬇️
server/ai_live_video.go	32.64095% <30.55556%> (+32.64095%)	⬆️
server/job_rpc.go	44.83627% <50.18450%> (+14.81318%)	⬆️
server/job_stream.go	55.44768% <55.44768%> (ø)

... and 144 files with indirect coverage changes

---

Continue to review full report in Codecov by Sentry.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update aca2275...f8189f0. Read the comment docs.

Files with missing lines	Coverage Δ
server/ai_process.go	2.01913% <ø> (+0.35938%)	⬆️
core/ai_orchestrator.go	30.53735% <0.00000%> (+1.16817%)	⬆️
server/ai_mediaserver.go	9.53912% <72.72727%> (+2.88383%)	⬆️
server/rpc.go	68.86228% <0.00000%> (+1.13612%)	⬆️
common/testutil.go	16.39344% <0.00000%> (+0.26441%)	⬆️
core/livepeernode.go	51.08696% <0.00000%> (-9.13403%)	⬇️
core/external_capabilities.go	29.29293% <2.94118%> (-57.54918%)	⬇️
server/ai_live_video.go	32.64095% <30.55556%> (+32.64095%)	⬆️
server/job_rpc.go	44.83627% <50.18450%> (+14.81318%)	⬆️
server/job_stream.go	55.44768% <55.44768%> (ø)

... and 144 files with indirect coverage changes

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[pschroedl]

lets check/log this err here

[pschroedl]

Still hoping we can check and log this err

[ad-astra-video]

Added an error check and log in d73fe37. Note that this function cannot return an error currently, should I just update to only return the JobToken. I think I added an error here thinking as I built it out something could cause an error possibly.

[pschroedl]

After resolving conflicts, lets merge 👍