Can the library detect sun/glasses and other kinds of headwear?

[Nekronik]

Hi, I was wondering if the library had means to detect glasses, sunglasses, and other types of headwear.
Is there a way to plug this object detection in?

If any of the above is true:

• Is it lightweight on the network?
• What is the performance impact on real-time detections?

Also, do you know how much glasses are affecting iris detection?

Thanks.

[vladmandic]

I was wondering if the library had means to detect glasses, sunglasses, and other types of headwear.

it's not currently included, but i just created a separate git repository at https://github.com/vladmandic/anti-spoofing
which can be re-trained with any image set true/false and model itself simply takes input image of a face and returns true/false

model is currently trained on real/fake face, but can be retrained for other uses like
glasses/no-glasses, mask/no-mask, female/male, etc - pretty much any binary decision regarding a face

and for glasses/no-glasses, a good dataset is available at https://www.kaggle.com/jeffheaton/glasses-or-no-glasses,
so process of retraining the model should be simple

Is there a way to plug this object detection in?

once model is trained, it can be used inside human by extending human itself or outside of human
for best result, input should be image of a face and human itself can be used for that purpose
(there is a config paramemter to return tensor of all detected faces which can be then used for this specific purposes)

extending human with additional models is quite easy due to modular design
(i'll write some documentation around it if there is interest)

basically, each model is a separate ES6 module that:

• takes image tensor and config as input
• if needed implement any pre-and post processing fully self-contained
• must implement two methods: load (returns model) and predict (returns uniform output object)

models can be

• generic (detection using entire image) or
• face-specific (detection using pre-detected face)

Is it lightweight on the network?

model is extremely lightweight (less than 1MB trained)

What is the performance impact on real-time detections?

performance is less than 10ms, so applicable in real-time use-cases

plus human has extensive caching capabilities built-in so there is no need to re-run all models each frame
(e.g., what is the chance that same person removed glasses between two frames? it is sufficient to re-run only if a) person changed, b) re-run anyhow every xx frames)

Also, do you know how much glasses are affecting iris detection?

thin frame glasses with clear lenses should not impact iris detection much
glasses with thick frames that hide eye outline or non-clear lenses definitely have major impact up to the point where iris detection is just a guess

[Nekronik]

Thank you very much for your detailed answer, exactly what I was looking for.

[01coding]

@vladmandic Can I add headgear or glasses to the detected face?

[vladmandic]

@01coding you can, but thats up to you, library does not do that out-of-the-box

[01coding]

@vladmandic Can you provide an idea, or what method in human can be used to add hairstyles or eyes and other decorations to the face

[vladmandic]

in the future, can you start a new discussion thread for things like this?

anyhow, quick example for glasses:

• run detection using human.detect
• calculate face size by looking at left and right face edges: face.mesh[127] and face.mesh[356]
  (you can see all points at https://raw.githubusercontent.com/vladmandic/human/main/assets/facemesh.png)
• scale image of glasses to match face size
• rotate image of glasses to match face rotation in face.rotation.angle
• draw image of glasses as overlay on original image

[01coding]

(you can see all points at https://raw.githubusercontent.com/vladmandic/human/main/assets/facemesh.png)

How to display the number of 468 dots on this picture?