TrackHands App Uses On-Device AI to Break Face-Touching Habits

TrackHands, a new open-source desktop application, addresses the common but often overlooked habit of unconscious face-touching. Created by independent developer Joaquin Ossandon (@cacoos), the tool leverages on-device machine learning to detect when fingers approach the mouth area, triggering subtle reminders to help users break behaviors like nail-biting or face-touching that can spread germs or exacerbate skin conditions.

The application combines multiple technologies into a privacy-focused solution:

1. Computer Vision Core: Using Google's MediaPipe framework, TrackHands simultaneously runs two machine learning models: Face Mesh for facial landmark detection and Hand Landmarks for finger tracking. These models calculate spatial relationships between detected hand positions and the mouth area in real-time.

2. Local Processing Architecture: All detection occurs on-device through a Rust backend built with Tauri, ensuring no camera footage or personal data leaves the user's computer. The frontend uses React with Vite and Zustand for state management, creating a responsive interface that minimizes system resources.

3. Discreet Notification System: When fingers linger near the mouth for approximately 2 seconds, TrackHands overlays a semi-transparent warning message across the screen without interrupting workflow. Users can optionally enable screenshot capture of detected incidents to build self-awareness.

Key technical features include adjustable detection sensitivity (with Slow/Medium/Fast intervals), camera resolution settings (Low/Medium/High), and a system-tray mode that runs unobtrusively in the background. The application is compatible with macOS 11+, Windows 10+, and Linux systems with standard webcams.

Privacy was a fundamental design constraint. "All processing pipelines terminate on the user's machine," explains Ossandon. "The camera feed is processed frame-by-frame in memory without persistent storage, and the app requires no network permissions."

For developers interested in the implementation, the GitHub repository reveals how MediaPipe's models integrate with Tauri's Rust backend through a React interface. The project employs a multi-threaded architecture where the vision processing runs separately from the UI, enabling real-time performance even on lower-end hardware.

Potential applications extend beyond habit correction. The underlying technology could be adapted for accessibility interfaces, hands-free controls, or behavioral research—all while maintaining strict on-device privacy. Ossandon welcomes community contributions to improve detection accuracy or expand platform support.

TrackHands exemplifies how lightweight machine learning models can deliver practical functionality without cloud dependencies. As computer vision capabilities become increasingly accessible through frameworks like MediaPipe, developers gain new tools to address nuanced human behaviors through ethical, local-first applications.