face liveliness test by blinking eye

[ersaurabh101]

I am looking at this demo - https://storage.googleapis.com/tfjs-models/demos/face-landmarks-detection/index.html

How can i get a trigger when eye is blinked ?

If you can provide me any direction , it will be very helpful.

[sgowroji]

Hi @ersaurabh101, Could you please point us the MediaPipe use case in the above demo. Thanks!

[ersaurabh101]

Use case will be attendance capture, Before capturing i need to ensure, its human not a picture

[sgowroji]

Can you share your code changes which include MediaPipe scripts to understand the issue much better. Thanks!

[ersaurabh101]

I tried to use this code but it doesnt trigger any message in console, if you can help and get it to work or something similar - https://gist.github.com/kleysonr/d75494f239ad0dce561a55a624920693

I just need a valid trigger on blink.

[sgowroji]

Hi @ersaurabh101, I don't see any MediaPipe CDN scripts added in the codebase. And facemesh is taken from tensorflow model as stated below. @tyrmullen Does this need any support from MediaPipe.

<script src="https://cdn.jsdelivr.net/npm/@tensorflow-models/face-landmarks-detection@0.0.2/dist/face-landmarks-detection.min.js"></script>

[ersaurabh101]

I am sorry for that, I tried to tell you what i am doing and what i need to achieve. I came to mediapipe from here- https://blog.tensorflow.org/2020/11/iris-landmark-tracking-in-browser-with-MediaPipe-and-TensorFlowJS.html

I just need help , want to catch when eye blinks, if possible. If not , you can close this
Regards

[sgowroji]

I would recommend you to use face mesh Javascript solution. Eye blinking you can notice this point regarding the face mesh. Let us know if you are looking for anything more.

[kostyaby]

Hey @ersaurabh101,

I tried to use this code but it doesnt trigger any message in console, if you can help and get it to work or something similar - https://gist.github.com/kleysonr/d75494f239ad0dce561a55a624920693

Unfortunately, I can't help you with other person's code. Please, contact that person directly to get a fix

How can i get a trigger when eye is blinked ?

That can be done using MediaPipe JS solution mentioned by @sgowroji (please, explore the links in #2313 (comment)). To detect if someone blinked, you probably need to work with landmarks from the FACEMESH_LEFT_EYE and FACEMESH_RIGHT_EYE contours. For each eye, please pick face mesh landmarks on both vertical and horizontal edges of the eye. Then, compute the distance ratio between the vertical / horizontal edge segments - that ratio will be a good target for thresholding in a heuristic. I can't tell you specifics now as I didn't play with exact values, but there will probably be a different confidence interval of the horizontal_edge_segment / vertical_edge_segment expression value when an eye is closed and it is open - you can use that to detect whether there was a blink or not.

Sorry for the lack of specifics, hopefully this rough outline of one possible algorithm is a good starting point for your project!

[google-ml-butler]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you.

[google-ml-butler]

Closing as stale. Please reopen if you'd like to work on this further.