Unleashing the Potential of Knowledge Distillation for IoT Traffic Classification
-
Abbasi, Mahmoud
2
-
Shahraki, Amin
3
-
Prieto, Javier
1
-
Arrieta, Angélica González
1
-
Corchado, Juan M.
1
- 1 Department of Computer Science and Automation Control, University of Salamanca, Salamanca, Spain
- 2 BISITE Research Group, University of Salamanca, Salamanca, Spain
- 3 Department of Informatics, University of Oslo, Oslo, Norway
ISSN: 2831-316X
Año de publicación: 2024
Volumen: 2
Páginas: 221-239
Tipo: Artículo
Otras publicaciones en: IEEE Transactions on Machine Learning in Communications and Networking
Resumen
The Internet of Things (IoT) has revolutionized our lives by generating large amounts of data, however, the data needs to be collected, processed, and analyzed in real-time. Network Traffic Classification (NTC) in IoT is a crucial step for optimizing network performance, enhancing security, and improving user experience. Different methods are introduced for NTC, but recently Machine Learning solutions have received high attention in this field, however, Traditional Machine Learning (ML) methods struggle with the complexity and heterogeneity of IoT traffic, as well as the limited resources of IoT devices. Deep learning shows promise but is computationally intensive for resource-constrained IoT devices. Knowledge distillation is a solution to help ML by compressing complex models into smaller ones suitable for IoT devices. In this paper, we examine the use of knowledge distillation for IoT traffic classification. Through experiments, we show that the student model achieves a balance between accuracy and efficiency. It exhibits similar accuracy to the larger teacher model while maintaining a smaller size. This makes it a suitable alternative for resource-constrained scenarios like mobile or IoT traffic classification. We find that the knowledge distillation technique effectively transfers knowledge from the teacher model to the student model, even with reduced training data. The results also demonstrate the robustness of the approach, as the student model performs well even with the removal of certain classes. Additionally, we highlight the trade-off between model capacity and computational cost, suggesting that increasing model size beyond a certain point may not be beneficial. The findings emphasize the value of soft labels in training student models with limited data resources.
Información de financiación
Financiadores
-
IoTalentum project within the framework of Marie Sklodowska-Curie Actions Innovative Training Networks-European Training Networks
- 953442
- European Union Horizon 2020 research and innovation program
Referencias bibliográficas
- 10.1016/j.jnca.2020.102538
- Abbasi, (2021), Eng. Appl. Artif. Intell., 102, 10.1016/j.engappai.2021.104234
- 10.1049/ntw2.12007
- 10.1109/ACCESS.2022.3205351
- 10.1109/ACCESS.2022.3205023
- 10.1007/s12243-020-00770-7
- 10.1109/MNET.2018.1700202
- 10.1007/s13042-021-01415-4
- 10.1007/s10462-020-09816-7
- 10.1016/j.scs.2020.102177
- 10.48550/arXiv.1503.02531
- 10.1109/ICC45855.2022.9838256
- 10.1109/ICC45855.2022.9839130
- 10.1109/CVPR42600.2020.01294
- 10.1609/aaai.v34i04.5963
- 10.1007/978-3-030-01252-6_17
- 10.1145/3240508.3240567
- 10.1109/ACCESS.2020.2983174
- Liu, (2019), arXiv:1904.09482
- 10.1109/CVPR46437.2021.00497
- Zhang, (2022), Ad Hoc Netw., 137, 10.1016/j.adhoc.2022.102984
- 10.1109/TPDS.2022.3225185
- 10.1109/JIOT.2021.3063497
- Wu, (2023), arXiv:2301.05849
- 10.1109/JIOT.2023.3297210
- 10.1109/JIOT.2021.3131981
- 10.1109/TNSM.2022.3211254
- Polino, (2018), arXiv:1802.05668
- 10.1109/TII.2023.3241689
- 10.1109/ACCESS.2022.3181135
- 10.1109/JIOT.2021.3121517
- 10.1109/ACCESS.2022.3153331
- 10.1109/SMARTCOMP55677.2022.00055
- 10.1145/1064212.1064220
- 10.1016/j.engappai.2022.105607
- 10.5220/0006639801080116
- Zagoruyko, (2016), arXiv:1612.03928
- 10.1016/j.neucom.2021.07.045