Use tf.js for no load on backend

[no-1ne]

hey Byron a thought, wouldn't it be server-free if we use tf.js, there is https://github.com/justadudewhohacks/face-api.js to get started (and if you want to go the facial landmark way)

To arrive at the eye part, a simpler way is posenet, https://github.com/tensorflow/tfjs-models/tree/master/posenet and feed that to a simple CNN which can classify if eye open or closed, https://towardsdatascience.com/real-time-face-liveness-detection-with-python-keras-and-opencv-c35dc70dafd3

[ByronHsu]

Hi, this is really a very clever solution for detector model's heavy delay.
But I used to write deep learning in pytorch and have no experience in tf.js.
If someone who experts in tf.js can help me, it would be better.

[no-1ne]

You just need JavaScript experience not much of tensorflow experience needed.

Two ways to get eye details one-way like in the original way using TF.js posenet another way is using harcascade and opencv.js which ever is closer to realtime.

For the tf.js way, the steps are
https://github.com/tensorflow/tfjs-models/tree/master/posenet#via-npm-1

if its via opencv.js, the first steps are
https://docs.opencv.org/4.1.0/js_face_detection.html

once the fastest one is determined, extract the eye canvas part and pass it thru a model that detemines if the eye is open or close, that I converted and attached is converted from https://github.com/Guarouba/face_rec/raw/master/model.h5 using https://github.com/tensorflow/tfjs-converter/blob/master/README.md

now in javascript, the model can be loaded using this

import * as tf from '@tensorflow/tfjs';

const MODEL_URL = 'https://...//model.json'; //attached in the zip 

const model = await tf.loadLayersModel(MODEL_URL); // For Keras use tf.loadLayersModel()
const cat = document.getElementById('cat');
model.predict(tf.browser.fromPixels(cat));

blink_tfjs_model.tar.gz
Good luck, just a point to ponder, AI is going to revolutionize the world, do you want to use the superpower to make blink games