How AI can improve food safety

Japanese IT giant Fujitsu has developed a computer vision model that recognises hand washing gestures as food businesses across Japan brace themselves for stricter food handling rules amid the Covid-19 pandemic.

Under the new regulations, which will come into effect in June 2020 in Japan, food businesses are required to implement stronger measures to ensure hygiene in accordance with international food safety standards.

This has created an urgent need for a non-invasive approach to quickly and accurately confirm that handwashing is carried out in a proper manner.

Fujitsu’s model builds on its existing behavioural analytics capabilities, which can already recognise a variety of subtle and complex human movements using deep learning techniques without relying on large amounts of training data.

It has refined those recognition capabilities for hand movements to automatically recognise complicated hand movements performed during hand washing – a six-step process recommended by Japan’s Ministry of Health, Labour and Welfare.

The technology is expected to reduce the number of man-hours required for visual checks by inspectors for on-site sanitation management.

gesture recognition using deep learning is a common technique for identifying hand and finger movements. The technique can detect multiple feature points – such as joints and fingertips – from an image of the hand and determine the hand gesture based on the positional relationship of the feature points.

However, one issue with the existing technology was that when people wash their hands correctly, both hands overlap and are lathered with soap, which obscures the detection points on the fingers and prevents accurate gesture recognition.

To address this challenge, Fujitsu’s technology captures images of complex handwashing movements as a combination of hand shape and repetitive rubbing motions, using two deep learning engines: hand shape recognition and motion recognition.

With a hand-wash video dataset comprising about 2,000 variations of people, camera positions, and soap types, Fujitsu said its technology was able to detect the six-step hand washing process with an accuracy rate of over 95%.

When implemented on the ground, a display will also indicate to food handlers that each step of the hand-washing process has been completed before they can move on to the next step. The data is automatically recorded, including timestamps and duration of each hand-washing action.

Besides food businesses, Fujitsu said its technology can be deployed at medical facilities, schools, hotels and venues for large events. It plans to conduct field trials and additional research and development to develop it into a potential offering in its artificial intelligence portfolio.

Apart from Fujitsu, public cloud giant Amazon Web Services has also developed a machine learning model that recognises hand gestures used in sign languages to facilitate communication with deaf-mute persons.

Built in about 10 days, the experimental model recognises and translates sign language gestures into text, as well as convert spoken words into text for a deaf-mute person to see.