[2212.08324] Mobile Augmented Reality with Federated Learning in the Metaverse

The Metaverse is deemed the next evolution of the Internet and has received
much attention recently. Metaverse applications via mobile augmented reality
(MAR) require rapid and accurate object detection to mix digital data with the
real world. As mobile devices evolve, they become more potent in computing.
Hence, their computational resources can be leveraged to train machine learning
models. In light of the increasing concerns of user privacy and data security,
federated learning (FL) has become a promising distributed learning framework
for privacy-preserving analytics. In this article, FL and MAR are brought
together in the Metaverse. We discuss the necessity and rationality of the
combination of FL and MAR. The prospective technologies that power FL and MAR
in the Metaverse are also identified. In addition, existing challenges that
prevent the fulfilment of FL and MAR in the Metaverse and several application
scenarios are presented. Finally, two case studies of Metaverse FL-MAR systems
are demonstrated.