-
Notifications
You must be signed in to change notification settings - Fork 150
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[Feature] mp4 live stream and standard authentication #203
Comments
I'm open to it, but the devil's in the details. There's no "just an mp4 stream" for live viewing AFAIK. There's the browser APIs that client Javascript can execute (notably MSE, although I sadly learned iPhone Safari doesn't support even that), there are wire protocols (RTSP, WebRTC, RTMP, HLS, MPEG-DASH), and there's less overlap between them than one might think. What Moonfire does today is use MSE from its Javascript, over a simple but custom WebSocket-based protocol. If there's a comparatively simple, at least as well-supported protocol that I've missed, I'd be happy to just switch to it. If there are other protocols folks would like it to support, I'm open to adding them. But my progress for Moonfire NVR comes little by little, and there are other features I'd desperately like to have (analytics, audio, web-based config, traditional scrub bar UI, etc.), so I can't promise it will happen quickly unless someone is able to roll up their sleeves and send me PRs. Do you have anything more specific in mind? Is there a certain pre-existing Home Assistant integration that you were hoping to use? And would the http(s) requests to Moonfire be coming from the Home Assistant server or directly from the browser/app?
HTTP Digest is possible but has the huge downside requiring the server to store the passwords in plaintext. I'd prefer HTTP Basic for that reason.
Yeah, that's also possible. I understood (possibly incorrectly) folks asking about it before as wanting to use major Internet OAuth2 providers, so the proposal of a local one is new to me. But in either case, it's certainly possible, just a matter of priorities. |
Yeah, I should probably have said mjpeg live stream. Any integration within https://www.home-assistant.io/integrations/#camera. I guess mjpeg or anything that works w/ ffmpeg would work for me. Those integrations will proxify the streams through HA, so the answer to your question is: HA will do the requests, effectively hiding Moonfire credentials from the end user. Yeah you're right about Digest vs Basic. You either store or send the password in plaintext. Pest or cholera... Regarding OAuth, the most important rule about security is, IMHO, don't roll your own authn/authz. Keycloak will handle user registration/login/2FA/etc and it has a built-in module for authorization, meaning you can just remove all that stuff from Moonfire and delegate it to Keycloak. This would make Moonfire really tight to KC, but it would solve a lot of work/issues for you. |
I'd prefer to avoid MJPEG. As compared to passing through the encoded H.264, transcoding frames to JPEG uses more server-side CPU, uses more bandwidth, has worse image quality, and is choppier. Also, the
It looks the Home Assistant's generic camera platform platform supports RTSP, and the RTSPtoWebRTC platform (apparently written by an old teammate of mine! small world!) can turn that into WebRTC (by proxying through a server written in Go). If you want this to work right now, I suggest having HA talk to the cameras independently of Moonfire. All the cameras I've experimented with support several simultaneous RTSP clients. Mine commonly have several RTSP clients (including a couple/few Moonfire instances running for development/testing) and it's been fine. That said, I see how there could be advantages to HA talking to Moonfire. Having binary switches and visual overlays for Moonfire's (future) analytics, reducing the bandwidth to the cameras (important if they are wireless and/or remote), allowing a tighter firewall setup that better contains the (notoriously insecure) cameras, avoiding duplicating camera config, avoiding the need to run a separate RTSPtoWeb server, showing events in HA's media library as Frigate does. In the shiny future Moonfire could proxy over RTSP and/or WebRTC and even have its own HA integration which (given only a Moonfire URL and credentials) makes all the cameras just work.
I want this to be easy to setup and use, with as few moving parts as possible. E.g. I use SQLite rather than a database server, and I'd like to remove the need for a proxy server in front of Moonfire (#27). My instinct is that I don't want the user guide to say "first, go set up Keycloak". So I think if we do offer OAuth, it should be in addition to Moonfire's native auth. Moonfire's auth isn't anything crazy. It's missing some things (notably 2fa) but I feel pretty good about what's there. I didn't roll my own crypto. It's using session cookies with plenty of entropy and the best practice |
I'm trying to integrate Moonfire into Home Assistant. My integration can connect to Moonfire, get the list of cameras and import them into Home Assistant. That part works great. Now I'm having troubles with the video stream. I reversed the UI to understand the websocket protocol, and what I got is you have to:
I'm trying to read a file (or a fifo) created that way on VLC, but it will only play a few frames, then hang on the last frame (the file gets bigger over time, so I know the websocket is still receiving data). The video is still in "playing" state, it's not stopping or trying to play next. VLC logs the following repeatedly while hanging:
Do you have any idea what could be wrong? I can privately submit you a sample file, if you wanna investigate Here are the relevant part of my PoC, in case it would trigger something to you: def stream(self, camera_uuid: str, stream_init=True):
cookies = self.session.cookies.get_dict()
url = self.url(f"cameras/{camera_uuid}/sub/live.m4s").replace('https:', 'wss:').replace('http:', 'ws:')
ws = websocket.WebSocket()
self.streams[camera_uuid] = {'ws': ws, 'running': True}
ws.connect(url, cookie="; ".join(["%s=%s" %(i, j) for i, j in cookies.items()]))
def parse_part(data):
headers_b, data_b = data.split(b'\r\n\r\n')
headers = {h.split(': ')[0]: h.split(': ')[1] for h in headers_b.decode('utf-8').split('\r\n')}
return headers, data_b
if stream_init:
first_part = ws.recv()
headers, data_b = parse_part(first_part)
init_mp4 = self.stream_init(headers['X-Video-Sample-Entry-Id'])
yield init_mp4
yield data_b
while self.streams[camera_uuid]['running']:
headers, data_b = parse_part(ws.recv())
yield data_b
ws.close() while True:
try:
with open(fifo_path, 'wb') as fifo:
stream = moonfire.stream(camera['uuid'])
for part in stream:
print('part')
fifo.write(part)
except BrokenPipeError:
print('broken') Then I just run vlc /tmp/fifo.mp4 Chrome acts pretty much the same as VLC: it plays only a few frames, then hangs, even after recording for ~15 seconds. If I manage to get it working on HA, then this issue won't be relevant to me anymore. Thanks! |
I just grepped for this in VLC code: It apparently reads the sequence number from the Moonfire always writes that as 1 right now: moonfire-nvr/server/src/mp4.rs Lines 1247 to 1251 in a6bdf0b
I can hack together something to make it increment on the WebSocket live stream and see if it helps. |
This is an experiment to see if helps with #203.
Does 28cd864 (on the |
Oh, we'll probably need to do the same thing for moonfire-nvr/server/src/mp4.rs Line 1275 in a6bdf0b
|
I tried to build w/ Docker according to your build insrtuctions: docker buildx build --load --tag=moonfire-nvr -f docker/Dockerfile . But it's failing after 6-7 minutes: ...
docker buildx build --load --tag=moonfire-nvr -f docker/Dockerfile .
...
#19 1.086 test result: ok. 37 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.12s
#19 1.086
#19 1.090
#19 1.090 real 0m0.377s
#19 1.090 user 0m0.400s
#19 1.090 sys 0m0.182s
#19 1.090 + cargo build --profile=release-lto
#19 1.315 Finished release-lto [optimized + debuginfo] target(s) in 0.17s
#19 1.321
#19 1.321 real 0m0.231s
#19 1.321 user 0m0.164s
#19 1.321 sys 0m0.064s
#19 1.321 + sudo install -m 755 /var/lib/moonfire-nvr/moonfire-nvr /usr/local/bin/moonfire-nvr
#19 1.325 install: cannot stat '/var/lib/moonfire-nvr/moonfire-nvr': No such file or directory I don't have much time to debug that right now, I'll try again tomorrow, but if you have solution in the meantime, that'be great |
It works, thanks for the quick fix! |
I can confirm your commit fixes the issue! |
VLC is happy, but now ffmpeg is complaining:
Not sure if this is related? |
Cool, glad that helped you keep moving. In general I'm not sure what seq number to set when the caller can request chunks of basically arbitrary size, but at least for the I'm a bit glad it wasn't necessary to set a non-zero
Hmm, only the initialization segment should have a MOOV atom in it. So if you're just appending one of those at the beginning of the file (which should be fine as long as you don't change camera parameters mid-stream), I don't know why you'd be seeing this message. You could open the file with e.g. https://gpac.github.io/mp4box.js/test/filereader.html to inspect the structure. |
This one is on me, when refactoring the code, I resent the init segment with each chunk... Now the only ffmeg complain is
once at the begining. VLC won't play the generated m3u8 with ffmpeg -i /tmp/fifo.mp4 -f hls -hls_time 4 -hls_playlist_type event stream.m3u8
Then it just stays on the playlist view and won't display any frame at all. For some reason, Home Assistant hangs when trying downloading the m3u8 it is supposed to generate to play the live stream. HASS internally uses ffmeg (and generates the warning as above), so maybe both are related. I simply pass back to Home Assistant the path to my fifo (HASS expects any string input ffmeg can handle, like URLs and file names). This might be related to HASS internals. Not sure yet, still investigating. |
I've been thinking this over. I suspect the problem is the timestamps. The structure of the media segments is set up for using with my HTML5 Media Source Extensions code, which doesn't care about their sequence numbers and is happy with their timestamps starting at 0 each time. When ffmpeg is reading a single file and sees fragments with To be honest though I'm not super optimistic about the never-ending So I can give this another try or two but if it doesn't work out, we may just have to bite the bullet and adopt a better-supported way. I mentioned some protocols up-thread (RTSP, WebRTC, RTMP, HLS, MPEG-DASH). There are also a couple other formats that I would expect to work reliably over a simple stream (TCP/FIFO/whatever): RTP or MPEG-TS streams. I'm a little confused about some finicky details of how to make HLS/MPEG-DASH work [1], but in general, all of these protocols are possibilities, it's "just" a matter of sufficient elbow grease. [1] E.g. LL-HLS mandates some specific requirements about partial segments being of uniform duration that seem to be assuming we control the frame rate, position of IDR frames, etc., when in our use case the camera controls that and we just go along with it. I don't know what we're supposed to do if those parameters change mid-stream. But maybe players aren't super strict about stuff like this, it probably comes up rarely, etc., so we can probably get away with bending the rules a little. |
I do understand your point, non standard stuff, etc, I'm aware I'm doing hacky stuff, and my video/streaming skills are close to zero, as you probably realized :) From here, I can see 2 solutions:
I'd really love the first solution, but I understand it's more work for you. I definitely do not have the required skills to help on this, and not enough time to learn it for this single use-case. So it's up to you :) |
Is it possible to add (AFAIK, there's no feature like that) an endpoint to copy the live stream, as an mp4 stream, so it can be embedded into another app (I'm thinking about Home Assistant)? Currently, the live stream is a websocket, which doesn't seem usable outside of Moonfire.
The second point is about authentication: pretty much the same as above, this is specific to Moonfire and difficult to integrate elsewhere. HTTP Digest, or even preferably OIDC (OAuth), would be really helpful. I know you said you don't want OAuth because you want it to work offline, but OAuth has no "online" requirement: one could simply spin up a Keycloak instance next to moonfire, and it would be 100% local.
What do you think?
Thanks
The text was updated successfully, but these errors were encountered: