Deep 3D Gaze Estimation with Head Motions
Gaze Estimation, Eye Tracking, Human-robot Interaction
Intuitive
- This work is done during my internship at the Interactive Algorithms Group at Intuitive Surgical. Due to the regulation of the company, I’m not allowed to publish detailed information about this project, including images and videos.
- Deep Learning-based Gaze Estimation: Designed and implemented an end-to-end deep learning-based 3D gaze estimation algorithm that could track surgeons eyes. The algorithm is robust to head motions and subject appearance differences.
- Performance Improvement: The developed algorithm improves the gaze estimation performance by 84.5%.
- Synthetic Data: Generated more than 100k synthetic data and images with suitable domain randomization in Blender for gaze estimation training.
- Data Collection: Designed real-world gaze estimation data collection pipeline and conducted data collection. Did detailed analysis and visualization of the dataset.
- Semi-auto Labeling: Implemented a semi-auto labeling tool for pupil localization and segmentation using SAM2.
Jiaying Fang
Electrical Engineering Master Student
As a Master student, I am interested in Surgical Robot, Robot Learning, and the intersection of Computer Vision and Robotics.