Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Feature: New tehcnique to compute the gaze vector #476

Open
vladmandic opened this issue Jun 25, 2024 Discussed in #465 · 3 comments
Open

Feature: New tehcnique to compute the gaze vector #476

vladmandic opened this issue Jun 25, 2024 Discussed in #465 · 3 comments
Labels
enhancement New feature or request question Further information is requested

Comments

@vladmandic
Copy link
Owner

Discussed in #465

Originally posted by saeidn95 May 16, 2024
Here is what I propose as an alternative to compute the gaze vector. The results are much better than the current technique. The current technique actually does not produces the right results (by a wide margin in the opposite direction for some samples.)

a. Select one of the eyes that is closer to the camera (same as the current technique)

b. Pick four points around the each eye
leftEyePoints = [33, 133, 159, 145]; //left, right, up, down, (we can replace 159, 145 with 27 and 23)
rightEyePoints = [362, 263, 374, 386]; //left, right, up, down, (we can replace 374, 386 with 257 and 253 )

compute the x and y coordinates of the centers of the eye by taking the averages of x and y coordinates of these four sets of points.

c. With the x, y, z coordinates of these four points build a linear system of equations as f = Ac, and solve for c (the center of the eyeball and the radius of the eyeball). The procedure is explained in (https://jekel.me/2015/Least-Squares-Sphere-Fit/)

d. Use the opencv linear solver or something similar to solve for c cv::solve(A, f, c, cv::DECOMP_SVD);

e. With x, y, z coordinates of center of the eyeball, the radius of the center of the eyeball, and x, y coordinates of c enter of the eye, compute the z component of the center of the eye. Change coordinates so that everything is with reference with the eyeball center (simple subtraction).

repeat the same procedure for iris. Note that for iris we might have to choose an offset. The offset I have chosen is (-5.0f, -10.0f) for the left eye, and (5.0f, -10.0f) for the right eye.

Note that we do sqrt to compute the z component of eye and iris center, we need to choose a right sign. For leftEyeCenter.z and leftIrisCenter.z, the sign is +ve and for the right eye righEyeCenter.z and rightrisCenter.z the sign is -ve.

f. convert from x, y, z for the eye center and the iris center to polar coordinate using the equation
leftEyeCenterHor = atan2(leftEyeCenter.x, - leftEyeCenter.z);
leftIrisCenterVer = asin(leftIrisCenter.y / leftEyeSphereRadious);

Finally
verAngle = leftEyeCenterVer - leftIrisCenterVer;
horAngle = leftIrisCenterHor - leftEyeCenterHor;

That is all!

@xhc-code
Copy link

Big shot, it's too powerful.

@vladmandic
Copy link
Owner Author

can you provide actual examples - image and results before/after?

d. Use the opencv linear solver or something similar to solve for c cv::solve(A, f, c, cv::DECOMP_SVD);

"or something similar" - to make human portable, whatever is used must be a pure js solution.
do you know of any such solution?

@vladmandic vladmandic added enhancement New feature or request question Further information is requested labels Sep 11, 2024
@xhc-code
Copy link

I also really want to participate, but I don't have the ability and knowledge to participate in such projects, it's really uncomfortable.😶

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request question Further information is requested
Projects
None yet
Development

No branches or pull requests

2 participants