If you find our work useful, please cite the paper:

@INPROCEEDINGS{10484276, author={Hempel, Thorsten and Jung, Magnus and Abdelrahman, Ahmed A. and Al-Hamadi, Ayoub}, booktitle={2024 IEEE/CVF Winter Conference on Applications of Computer Vision (WACV)}, title={NITEC: Versatile Hand-Annotated Eye Contact Dataset for Ego-Vision Interaction}, year={2024}, pages={4425-4434}, doi={10.1109/WACV57701.2024.00438}}

Thorsten Hempel, Magnus Jung, Ahmed A. Abdelrahman and Ayoub Al-Hamadi, "NITEC: Versatile Hand-Annotated Eye Contact Dataset for Ego-Vision Interaction", WACV 2024.

Eye contact is a crucial non-verbal interaction modality and plays an important role in our everyday social life. While humans are very sensitive to eye contact, the capabilities of machines to capture a person's gaze are still mediocre. We tackle this challenge and present NITEC, a hand-annotated eye contact dataset for ego-vision interaction. NITEC exceeds existing datasets for ego-vision eye contact in size and variety of demographics, social contexts, and lighting conditions, making it a valuable resource for advancing ego-vision-based eye contact research. Our extensive evaluations on NITEC demonstrate strong cross-dataset performance, emphasizing its effectiveness and adaptability in various scenarios, that allows seamless utilization to the fields of computer vision, human-computer interaction, and social robotics. We make our NITEC dataset publicly available to foster reproducibility and further exploration in the field of ego-vision interaction.

pip install face_detection@git+https://github.com/elliottzheng/face-detection pip install nitec

Example usage:

from nitec import NITEC_Classifier, visualize import cv2 nitec_pipeline = NITEC_Classifier( weights= CWD / 'models' / 'nitec_rs18_e20.pth', device=torch.device('cuda') # or 'cpu' ) cap = cv2.VideoCapture(0) _, frame = cap.read() # Process frame and visualize results = nitec_pipeline.predict(frame) frame = visualize(frame, results, confidence=0.5)

Prepare the dataset as explained here.

Download from here: https://drive.google.com/drive/folders/1zc6NZZ6yA4NJ52Nn0bgky1XpZs9Z0hSJ?usp=sharing

 python train.py \ --gpu 0 \ --num_epochs 50 \ --batch_size 64 \ --lr 0.0001 \

 python test.py \ --snapshot models/nitec_rs18_20.pth \ --gpu 0 \