Replies: 11 comments
-
Thanks - I'm impressed how they implemented smoothing. Unfortunately, cannot use it to learn anything useful as @mediapipe solutions are written in C++ (JS is used just as loader for a custom compiled WASM binary). I do see that @mediapipe uses if anything, @mediapipe showcases just how much faster is a native binary code than any JS code that is interpreted in browser and uses a generic framework for backend (which is not unexpected). |
Beta Was this translation helpful? Give feedback.
-
i'm converting this to a discussion topic now that github supports discussions :) |
Beta Was this translation helpful? Give feedback.
-
I remembered I came across some discussions about the doubts in the future of TFJS, especially with the rise of WASM. I wonder if this really means a JavaScript solution of Tensorflow is not the way to go, or are there still some benefits of using JS for Tensorflow despite the performance disadvantage? |
Beta Was this translation helpful? Give feedback.
-
End-to-end But what you mention (I saw that discussion) is a bit misleading as So how can the question be So why is @mediapipe so much faster than TFJS+WASM? On the other hand WASM is superior solution for quick models - it has very low latency. But GPU integration within WASM is as difficult as it can be - I haven't seen much of it so far. So for complex models, WebGL is still a superior backend. Now, WebGL does have an issue of double translation (most browsers implement WebGL via DX first and then using actual GL calls later) plus much higher GPU memory usage. I am looking with anticipation of |
Beta Was this translation helpful? Give feedback.
-
To add, I'm working on developing some models of my own (not related to Btw, if you want absolutely maximum performance for ML, use C++ and link to CUDA libraries for GPU acceleration. And then use a higher level language to interact with library you wrote in C++/CUDA. But that language itself should support parallel execution - and JS doesn't (it's single-threaded by design). Python would be most popular because and honestly, it's not bad (horrible native performance, but can use external libraries with minimum overhead). |
Beta Was this translation helpful? Give feedback.
-
Yeah I know it's not fair to compare TFJS with WASM directly, since just like what you said, TFJS has a WASM backend as well. But assuming that you need to have the best performance on a web platform, this kind of all-WASM solution (obviously GPU accelerated as well in this case) saving all the slower JS preprocessing may be the way to go. PS: Been working on my own project that use TFJS + Facemesh + PoseNet + Handpose. Just when I got it working, then I came across your |
Beta Was this translation helpful? Give feedback.
-
Absolutely. And it's not just about ML - I know several graphic-heavy websites that that converted from JS to WASM and got massive performance boost. But that is not for casual developer for sure. Current state of WASM compilers (e.g.
Yeah, took me a while to get that one working, it's pretty messy. My code diverged from TFJS example quite a lot.
Quite a lot based on a quick initial glance.
|
Beta Was this translation helpful? Give feedback.
-
one more thought - as a casual developer, I don't even want to think about developing WASM solution other than a proof-of-concept if I still have to think about things like:
WASM promise was that dev will be able to take his existing binary code and convert it run natively in browser - we're still years from that. |
Beta Was this translation helpful? Give feedback.
-
I tried to use the MediaPipe API in my project, but unfortunately it doesn't seem to support web worker (a must in my case, since there are some intensive 3D animations, and there is little room in the mani UI thread for other CPU-intensive task). So at the end I tried your @vladmandic
|
Beta Was this translation helpful? Give feedback.
-
Um, just more info, Mediapipe doesn't even implement Multi-Threading. If they do, then that could lead to a very significant FPS increase from the already-awesome fps they have.... |
Beta Was this translation helpful? Give feedback.
-
FYI, I've managed to successfully convert and implement MediaPipe's |
Beta Was this translation helpful? Give feedback.
-
MediaPipe now has it's own javascript api that you can use instead of tensor flow.
https://ai.googleblog.com/2020/12/mediapipe-holistic-simultaneous-face.html
https://google.github.io/mediapipe/getting_started/javascript#ready-to-use-javascript-solutions
Beta Was this translation helpful? Give feedback.
All reactions