AI-Driven App Detects Early Signs of Disease Using Only a Smartphone Camera
“Mobile phones with a camera are the most widely available yet least exploited health-analysis tools,” says Remi Koscielny, cofounder and CEO of MX Labs. “We want to squeeze as much capability as possible out of the smartphones for diagnostic purposes without the need for any additional hardware.”
While over five billion people worldwide use mobile phones, the World Bank and the World Health Organization report that at least half of the global population lacks access to essential health services.
“Sixty-three percent of people in the world have mobile phones,” Przemek Jaworski, cofounder and CTO of MX Labs, says, “and 2.87 billion of them have a smartphone equipped with a camera. Imagine turning every smartphone into a health-assessment device. That’s our mission.”
Health monitoring from MX Labs starts with a short video of the subject’s face. The app stabilizes the real-time images and maps them to a customized three-dimensional face-geometry mesh. Computer-vision techniques then extract data patterns from facial skin, eyes, and other features such as tiny movements of the head and muscles. In order to analyze heart activity and extract vital signs, the engine uses remote photoplethysmography (rPPG), a contactless technique of recording blood pulsations in the skin vasculature.
Based on advanced data engineering and the latest research results, MX Labs’s technology can measure various cardiovascular parameters (blood pressure, heart rate, and heart-rate variability), provide visual biofeedback, assess skin condition, and make a preliminary diagnosis with recommended next steps. The technology, named Shen.AI, requires only a smartphone, tablet, or computer with a camera. No other devices are needed. The Shen.AI platform will be available as a software development kit (SDK) for app developers and hardware manufacturers.
MX Labs aims to develop a family of mobile apps starting with Heart Monitor, which is already available for free on Apple’s App Store and Google Play. Heart Monitor has been designed as a support solution for monitoring cardiovascular health and detecting hypertension, which could significantly improve access to early diagnosis of hypertension, a leading preventable risk factor for cardiovascular diseases. A skincare app will be launched next in 2021.