Google is working on developing a new technology based on artificial intelligence that may enable smartphones to provide advanced health information using only the phone’s camera, in a step that could open the door for turning the Pixel phone into a tool for assessing certain indicators related to body health.

The innovation comes within a research project called PhotoScan, where Google researchers trained an artificial intelligence model using real medical data and images, including X-ray images with two-energy absorption technology (DXA), in addition to magnetic resonance images. They then compared this data with images taken by smartphones cameras.

Measuring the fat percentage from phone pictures
 

The basic idea of the technology is to analyze body pictures taken by the user’s phone, and then use artificial intelligence to estimate body composition, including the percentage of fat.

It does not only estimate the total fat percentage; the system also tried to analyze the distribution of fat in different areas of the body, such as the percentage of abdominal fat compared to waist and leg fat. In addition, it distinguishes between visceral fat surrounding organs and subcutaneous fat.

According to research results, PhotoScan showed promising ability to estimate body composition from images taken by phones, with better performance than some bioelectrical impedance analysis (BIA) techniques used in a number of wearable devices.
Artificial intelligence looks for indicators related to insulin resistance

Researchers did not stop at analyzing fats, but also tested the possibility of using information extracted from images to predict insulin resistance, a condition associated with several health and metabolic problems.

The study's results indicate that the model was able to predict insulin resistance in an encouraging manner in the research environment, opening the door for future use of mobile phone images as an additional source of health information.

Does a smartphone outperform smartwatches?

Some smartwatches and fitness trackers have the ability to estimate body fat percentage using sensors based on BIA technology, but these devices remain limited in the amount of information they can provide.

PhotoScan, on the other hand, attempts to provide a more comprehensive picture of body composition, including fat distribution, which could give it an edge in research related to cardiovascular health and metabolism.

However, this technology does not mean that smartphones can currently replace medical examinations or specialized imaging devices. DXA technology remains one of the most accurate tools for assessing body composition, but it requires medical equipment and facilities. Google's technology aims to provide a simpler and more scalable solution.

Challenges of privacy and accuracy

Despite the great potential of this technology, its widespread use raises important questions, most notably privacy, especially since the system needs body images to analyze health indicators.

Converting a research model into a feature available to users requires proving its accuracy and reliability through extensive testing, along with establishing clear controls for how images and health data are stored and used.

At present, there is no indication that PhotoScan has become an available feature on consumer Pixel phones, but the success of the research may pave the way in the future for integrating these capabilities into a health app or services based on Gemini and Google Health.