AssistMe is an accessibility-first web app that turns eye movement into intentional, spoken interaction. Built in a team of four during MLH HackHive OTU, it uses a standard webcam, iris-based gaze tracking, dwell-based selection, calibration storage, and AI-assisted follow-up options to reduce interaction friction for users with limited mouse control.
Designed to work with a standard camera instead of expensive dedicated hardware.
Uses large focus regions so the interaction model is simpler and less fatiguing.
Turns a selected intent into generated options and spoken confirmation.
Traditional eye-typing interfaces can demand too much precision, too much calibration, or hardware that is out of reach for many people. AssistMe explores a different path.
Instead of asking users to type precisely with their gaze, the interface breaks choices into large regions and confirms intent through sustained focus. That makes the interaction easier to understand and easier to use under time pressure. I also implemented stabilization heuristics to improve cursor reliability and reduce jitter, plus persistent calibration storage in PostgreSQL so returning users did not need to restart setup from scratch.
MediaPipe FaceMesh reads iris landmarks from a standard webcam feed.
The interface maps focus to one of four large regions instead of tiny targets.
Dwell time prevents accidental clicks and turns sustained attention into input.
Gemini generates context-aware options, and the selected response is spoken aloud.