Anomaly detection system for data quality assurance in IoT infrastructures based on machine learning

Abstract

The inclusion of IoT in digital platforms is very common nowadays due to the ease of deployment, low power consumption and low cost. It is also common to use heterogeneous IoT devices of ad-hoc or commercial development, using private or third-party network infrastructures. This scenario makes it difficult to detect invalid packets from malfunctioning devices, from sensors to application servers. These invalid packets generate low quality or erroneous data, which negatively influence the services that use them. For this reason, we need to create procedures and mechanisms to ensure the quality of the data obtained from IoT infrastructures, regardless of the type of infrastructure and the control we have over them, so that the systems that use this data can be reliable. In this work we propose the development of an Anomaly Detection System for IoT infrastructures based on Machine Learning using unsupervised learning. We validate the proposal by implementing it on the IoT infrastructure of the University of Alicante, which has a multiple sensing system and uses third-party services, over a campus of one million square meters. The contribution of this work has been the generation of an anomaly detection system capable of revealing incidents in IoT infrastructures, without knowing details about the infrastructures or devices, through the analysis of data in real time. This proposal allows to discard from the IoT data flow all those packets that are suspected to be anomalous to ensure a high quality of information to the tools that consume IoT data.