Trustworthy Acceptance: A New Metric for Trustworthy Artificial Intelligence Used in Decision Making in Food–Energy–Water Sectors

Abstract

We propose, for the first time, a trustworthy acceptance metric and its measurement methodology to evaluate the trustworthiness of AI-based systems used in decision making in Food Energy Water (FEW) management. The proposed metric is a significant step forward in the standardization process of AI systems. It is essential to standardize the AI systems’ trustworthiness, but until now, the standardization efforts remain at the level of high-level principles. The measurement methodology of the proposed includes human experts in the loop, and it is based on our trust management system. Our metric captures and quantifies the system’s transparent evaluation by field experts on as many control points as desirable by the users. We illustrate the trustworthy acceptance metric and its measurement methodology using AI in decision-making scenarios of Food-Energy-Water sectors. However, the proposed metric and its methodology can be easily adapted to other fields of AI applications. We show that our metric successfully captures the aggregated acceptance of any number of experts, can be used to do multiple measurements on various points of the system, and provides confidence values for the measured acceptance.