mmEat: Millimeter wave-enabled environment-invariant eating behavior monitoring

Abstract

Dietary habits are closely related to people’s health condition. Unhealthy diet can cause obesity, diabetes, heart diseases, as well as increase the risk of cancers. It is necessary to have a monitoring system that helps people keep tracking his/her eating behaviors. Traditional sensor-based and camera-based dietary monitoring systems either require users to wear dedicated devices or may potentially incur privacy concerns. WiFi-based methods, though yielding reasonably robust performance in certain cases, have major limitations. The wireless signals usually carry substantial information that is specific to the environment where eating activities are performed. To overcome these limitations, we propose mmEat, a millimeter wave-enabled environment-invariant eating behavior monitoring system. In particular, we propose an environment impact mitigation method by analyzing mmWave signals in Dopper-Range domain. To differentiate dietary activities with various utensils (i.e., eating with fork, fork and knife, spoon, chopsticks, bare hand) for fine-grained eating behavior monitoring, we construct Spatial–Temporal Heatmap by integrating multiple dimensional measurements. We further utilize an unsupervised learning-based 2D segmentation method and an eating period derivation algorithm to estimate time duration of each eating activity. Our system has the potential to infer the food categories and eating speed. Extensive experiments with over 1000 eating activities show that our system can achieve dietary activity recognition with an average accuracy of 97.5% and a false detection rate of 5%.