[2206.07772] Deep Learning and Handheld Augmented Reality Based System for Optimal Data Collection in Fault Diagnostics Domain

Compared to current AI or robotic systems, humans navigate their environment
with ease, making tasks such as data collection trivial. However, humans find
it harder to model complex relationships hidden in the data. AI systems,
especially deep learning (DL) algorithms, impressively capture those complex
relationships. Symbiotically coupling humans and computational machines’
strengths can simultaneously minimize the collected data required and build
complex input-to-output mapping models. This paper enables this coupling by
presenting a novel human-machine interaction framework to perform fault
diagnostics with minimal data. Collecting data for diagnosing faults for
complex systems is difficult and time-consuming. Minimizing the required data
will increase the practicability of data-driven models in diagnosing faults.
The framework provides instructions to a human user to collect data that
mitigates the difference between the data used to train and test the fault
diagnostics model. The framework is composed of three components: (1) a
reinforcement learning algorithm for data collection to develop a training
dataset, (2) a deep learning algorithm for diagnosing faults, and (3) a
handheld augmented reality application for data collection for testing data.
The proposed framework has provided above 100\% precision and recall on a novel
dataset with only one instance of each fault condition. Additionally, a
usability study was conducted to gauge the user experience of the handheld
augmented reality application, and all users were able to follow the provided
steps.