Artificial intelligence helping to feed hospital patients

Nutrition plays an essential role in the care of hospitalized patients, enabling them to regain good health. However, there is significant room for improvement in nutrition in healthcare facilities: it is estimated that between 20% and 40% of hospitalized patients are malnourished—a figure that can reach up to 50% in the case of hospitalized elderly people.

To limit the risk of malnutrition in hospitalized patients, it is essential to have accurate visibility into each patient's diet, both in terms of quality (which foods on the meal tray are consumed or left behind) and quantity. As part of the FoodInTech project, IT and artificial intelligence tools are being made available to caregivers to provide reliable quantitative and qualitative data on their patients' diets, in order to guide them in their nutritional monitoring.

Find out more in this article.

An automated three-step analysis of meal trays

A three-step method is being implemented thanks to the collaboration of the FoodInTech project partners (Dijon Bourgogne University Hospital, Yumain, and Atol Conseils et Développements):

1. Image capture
2. Measurements taken using artificial intelligence
3. Information displayed on a dashboard for dieticians and doctors.  

“Yumain's role was to develop the artificial intelligence tool capable of receiving and recognizing images of meal trays,” explains Michel Paindavoine.  

Image capture

Photos are taken using an Android mobile device installed directly on the meal tray trolley (hands-free system). This allows the team at Dijon University Hospital to automatically take two photos of each meal tray: one before serving and one when clearing away.

The images are then sent to a secure server at the Dijon University Hospital, anonymized, and translated into menus by Atol Conseils et Développements software, using a QR code printed on each meal tray: for example, salad with pasta, peas, and ground beef, pastry.

Automation of “ingesta” measurement: in terms of quality and quantity

The “before/after” image is simultaneously transmitted by the AtolCD software to Yumain's artificial intelligence solution, which processes it to identify and measure what has been eaten. “The goal is to measure food intake,” explains Michel Paindavoine—a term used in hospitals to refer to all food (solid or liquid) ingested by a patient.

Here, two automated processes had to be integrated into Yumain's artificial intelligence system:

• Food recognition, so that artificial intelligence can recognize the variety of meals served.
• Surface measurement, by cropping, to identify the amount ingested by the patient.

In just a few seconds, artificial intelligence can compare “before” and “after” images: for example, we can see that 51% of the peas and 11% of the ground beef were consumed—information that will be translated into operational terms for dietitians in the third stage.

“By combining our artificial intelligence tool with the IT expertise provided by Atol Conseils et Développements, we can limit certain recognition errors (for example, there may be confusion between polenta and mashed potatoes),” explains Michel Paindavoine. “This provides a truly industrializable solution, meaning it can be automated on a large scale while significantly reducing the margin of error.”

A dashboard for patient monitoring

Once the measurements have been taken, Atol Conseils et Développements translates them into food intake records using a digital interface designed for dietitians. These records are sent to each patient's medical file so that doctors can monitor changes in their patients' food consumption. 

Dieticians can access details of each patient's consumption for each food item and each meal on a digital dashboard: for example, consuming 76% of a portion of country pâté identified by artificial intelligence corresponds to 136.35kcal, 6g of protein, 11g of fat, etc. Graphs are also available to visualize changes in consumption by meal and day, consumption statistics, energy monitoring, etc. An alert engine also highlights patients whose food consumption is insufficient for their needs.

An additional application: limiting waste

“In addition, the data collected also makes it possible to track food waste on a dish-by-dish basis: whether as a percentage of the dish, in kilograms wasted, or even in relation to expenditure or CO2 consumption,” adds Michel Paindavoine.

Although this use of artificial intelligence is not a priority within the FoodInTech project, it offers many benefits in hospital catering and also in collective catering: identifying meals that are not appreciated in order to implement corrective measures (ingredients, recipes, menus, etc.) at the central kitchen level in order to improve the food served to guests and target their preferences while reducing the costs (economic and environmental) associated with waste.

Artificial intelligence: reliable and fast results

Objective: to increase the accuracy of the analysis

“Until now, qualitative measurements of what was consumed or not consumed on a meal tray were carried out by nursing staff, based on visual inspection,” explains Michel Paindavoine. “For example, staff could indicate that a quiche was eaten to 1/4.”

As part of the FoodInTech project, the artificial intelligence tool provides particularly accurate quantitative information, with around 80% of measurements being correct in the laboratory; in real life, several factors, particularly human ones, can impact the accuracy of the system. “There is still a 20% margin of error,” admits Michel Paindavoine, “because the photos we obtain are in 2D. Obtaining 3D photos, thanks in particular to new technologies in photography, would allow us to take into account the relief of quantities, and not just the surface area, to reduce this margin of error.”

“Training” artificial intelligence

Michel Paindavoine explains the ongoing work of his teams: "Our role at Yumain is to continuously improve our artificial intelligence tool. Today, the AI's food cropping and the surface area occupied by the identified pixels achieve 98% recognition accuracy. We are conducting comparative cropping tests to continuously improve our solution, regardless of the food item."

Thus, the Yumain teams are “training” their artificial intelligence solution. “A complete classic training session takes place over three days. At the end of this period, the tool becomes capable of comparing before/after photos in just two seconds to identify the foods consumed and their quantities.”

What role does human intelligence play?

Michel Paindavoine is aware of this: “There are many objections to artificial intelligence, and a suspicion that has always existed... The concern that it will replace humans is entirely understandable and legitimate.”

But he reassures us about the tool's limitations compared to human intelligence: "Artificial intelligence cannot do everything; it lacks consciousness. It certainly allows us to obtain very accurate and rapid measurement results... but the interpretation of the results is always carried out by humans. The software developed by AtolCD also includes a level of human validation of digitally estimated intakes, and dietitians can check the photos taken by the system at any time and modify the percentages assessed for each dish according to their own assessment. Thanks to artificial intelligence, humans can focus on high value-added tasks where their intelligence is truly put to good use: data analysis and exploitation, decision-making, customer service, interactions, etc.

It was in order to reemphasize the usefulness of artificial intelligence in serving humans (and not the other way around) that the company's leaders chose to rename Global Sensing Technologies “Yumain” in 2019. “With the FoodInTech project, our artificial intelligence technology serving humans takes on its full meaning, as our solution is made available to practitioners for monitoring their patients.”

Yumain: artificial intelligence serving the agri-food industry

Beyond improving food service for hospital patients, Yumain is putting its expertise to use in other applications.  

Protecting people in factories  

In the field of personnel protection in factories, Yumain offers a solution dedicated to protecting people in logistics sites (e.g., warehouses) in order to prevent accidents, particularly those involving forklifts. “Factories can sometimes be as dangerous in terms of traffic as highways: there are corridors where speeds can be high, but also numerous intersections,” explains Michel Paindavoine. “Our idea is to warn forklift operators of collision risks in real time using a smart sensor called ‘ECS’ that reacts in less than a second, without the need to equip pedestrian teams or vehicles with special detection equipment.”

Quality control

Several artificial intelligence applications can also be used for quality control.

Yumain, for example, has developed a solution for industries (including the food industry) that produce low volumes or whose production cycle changes every two or three days, and for which existing visual defect detection tools are not suitable: an onboard camera that can identify whether a product meets expectations (unlike a camera that would identify a defect, which would require days of training for the artificial intelligence to learn how to identify each pitfall). “This solution allows for the rapid deployment of a new production cycle, particularly suited to industries where manufacturing cycles change regularly, for example from a cardboard packaging production line to a canning production line.”

The development of an “artificial nose” to analyze polluting gases (such as ammonia), is also underway as part of a CIFRE thesis led by Ludmilla Grzelak in collaboration with the ICB physics and chemistry laboratory at the University of Dijon (thesis supervised by Jérôme Rossignol and co-supervised by Olivier Brousse from Yumain).

Go further 

To learn more about Yumain's expertise or to be put in touch with Michel Paindavoine and his team, please contact me by email: pascal.bertrand@vitagora.com.  

A lire également :   

• Foodintech application - https://foodin.tech/
• Van Wymelbeke-Delannoy, V., Juhel, C., Bole, H., Sow4, A.K., Guyot,C., Beldaghdadi,F., Brousse,O., & Paindavoine,M. (2022). A Cross-Sectional Reproducibility Study of a Standard Camera Sensor Using Artificial Intelligence to Assess Food Items: The FoodIntech Project. Nutrients. 

Keywords

Artificial intelligence, solutions, decision support, nutrition, nutritional improvement, health, catering, hospitals, seniors, anti-waste