UAV Path Planning Optimization Strategy: Considerations of Urban Morphology, Microclimate, and Energy Efficiency Using Q-Learning Algorithm

  1. Souto, Anderson 1
  2. Alfaia, Rodrigo 1
  3. Cardoso, Evelin 12
  4. Araújo, Jasmine 1
  5. Francês, Carlos 1
  6. González Aguilera, Diego 3
  7. Georgiev, Georgi
  8. Bauer, Friedrich-Wilhelm
  9. Sindelar, Ralf
  10. Rodríguez-Gonzálvez, Pablo
  1. 1 Postgraduate Program in Electrical Engineering, Federal University of Pará (UFPA), Belém 66075110, Brazil
  2. 2 Computer Science Area, Federal Rural University of the Amazon (UFRA), Belém 66077830, Brazil
  3. 3 Universidad de Salamanca
    info

    Universidad de Salamanca

    Salamanca, España

    ROR https://ror.org/02f40zc51

Mostrar afiliaciones +
Revista:
Drones

ISSN: 2504-446X

Año de publicación: 2023

Volumen: 7

Número: 2

Páginas: 123

Tipo: Artículo

DOI: 10.3390/DRONES7020123 GOOGLE SCHOLAR lock_openAcceso abierto editor

Otras publicaciones en: Drones

Resumen

The use of unmanned aerial vehicles (UAVS) has been suggested as a potential communications alternative due to their fast implantation, which makes this resource an ideal solution to provide support in scenarios such as natural disasters or intentional attacks that may cause partial or complete disruption of telecommunications services. However, one limitation of this solution is energy autonomy, which affects mission life. With this in mind, our group has developed a new method based on reinforcement learning that aims to reduce the power consumption of UAV missions in disaster scenarios to circumvent the negative effects of wind variations, thus optimizing the timing of the aerial mesh in locations affected by the disruption of fiber-optic-based telecommunications. The method considers the K-means to stagger the position of the resource stations—from which the UAVS launched—within the topology of Stockholm, Sweden. For the UAVS’ locomotion, the Q-learning approach was used to investigate possible actions that the UAVS could take due to urban obstacles randomly distributed in the scenario and due to wind speed. The latter is related to the way the UAVS are arranged during the mission. The numerical results of the simulations have shown that the solution based on reinforcement learning was able to reduce the power consumption by 15.93% compared to the naive solution, which can lead to an increase in the life of UAV missions.

Referencias bibliográficas

  • Dagooc, (2010), Philipp. Star. Retrieved March, 3, pp. 2011
  • Mosannenzadeh, (2014), TeMA-J. Land Use Mobil. Environ., 16, pp. 684
  • Mulero, (2019), Int. J. Distrib. Sens. Netw., 15, pp. 1550147719853984
  • Visvizi, (2018), J. Sci. Technol. Policy Manag., 9, pp. 126, 10.1108/JSTPM-07-2018-079
  • Silva, (2018), Sustain. Cities Soc., 38, pp. 697, 10.1016/j.scs.2018.01.053
  • (2015), Serv. Sci., 7, pp. 2
  • Gupta, (2015), IEEE Commun. Surv. Tutorials, 18, pp. 1123, 10.1109/COMST.2015.2495297
  • Mehta, (2020), Comput. Commun., 151, pp. 518, 10.1016/j.comcom.2020.01.023
  • Maddikunta, (2021), IEEE Sensors J., 21, pp. 17608, 10.1109/JSEN.2021.3049471
  • Kalla, (2021), IEEE Open J. Commun. Soc., 2, pp. 836, 10.1109/OJCOMS.2021.3071496
  • Tisdale, (2009), IEEE Robot. Autom. Mag., 16, pp. 35, 10.1109/MRA.2009.932529
  • Grasso, C., and Schembra, G. (2018, January 25–29). Design of a UAV-based videosurveillance system with tactile internet constraints in a 5G ecosystem. Proceedings of the 2018 4th IEEE Conference on Network Softwarization and Workshops (NetSoft), Montreal, QC, Canada.
  • Ullah, (2019), Future Gener. Comput. Syst., 97, pp. 425, 10.1016/j.future.2019.01.028
  • Tang, C., Zhu, C., Wei, X., Rodrigues, J.J., Guizani, M., and Jia, W. (2020, January 15–19). UAV placement optimization for Internet of Medical Things. Proceedings of the 2020 International Wireless Communications and Mobile Computing (IWCMC), Limassol, Cyprus.
  • Mozaffari, (2018), IEEE Trans. Wirel. Commun., 18, pp. 357, 10.1109/TWC.2018.2879940
  • Li, (2018), IEEE Internet Things J., 6, pp. 2241, 10.1109/JIOT.2018.2887086
  • Pham, (2021), IEEE Trans. Veh. Technol., 70, pp. 3944, 10.1109/TVT.2021.3065084
  • Zeng, (2016), IEEE Commun. Mag., 54, pp. 36, 10.1109/MCOM.2016.7470933
  • Kose, (2022), Aircr. Eng. Aerosp. Technol., 94, pp. 1228, 10.1108/AEAT-06-2021-0180
  • Zeng, (2017), IEEE Trans. Wirel. Commun., 16, pp. 3747, 10.1109/TWC.2017.2688328
  • Wu, (2019), IEEE Access, 7, pp. 79617, 10.1109/ACCESS.2019.2922651
  • PAHL, J. (July, January 29). Flight recorders in accident and incident investigation. Proceedings of the 1st Annual Meeting, Washington, DC, USA.
  • Lester, A. (November, January 31). Global air transport accident statistics. Proceedings of the Aviation Safety Meeting, Toronto, ON, Canada.
  • Li, (2019), IEEE Wirel. Commun. Lett., 8, pp. 1520, 10.1109/LWC.2019.2925796
  • Hermand, E., Nguyen, T.W., Hosseinzadeh, M., and Garone, E. (2018, January 19–22). Constrained control of UAVs in geofencing applications. Proceedings of the 2018 26th Mediterranean Conference on Control and Automation (MED), Zadar, Croatia.
  • Ariante, G., Ponte, S., Papa, U., Greco, A., and Del Core, G. (2022). Ground Control System for UAS Safe Landing Area Determination (SLAD) in Urban Air Mobility Operations. Sensors, 22.
  • Jayaweera, H.M., and Hanoun, S. (2022). Path Planning of Unmanned Aerial Vehicles (UAVs) in Windy Environments. Drones, 6.
  • Cao, (2019), IEEE Access, 7, pp. 179252, 10.1109/ACCESS.2019.2958680
  • Tseng, C.M., Chau, C.K., Elbassioni, K.M., and Khonji, M. (2017). Flight Tour Planning with Recharging Optimization for Battery-Operated Autonomous Drones. CoRR, Available online: https://www.researchgate.net/profile/Majid-Khonji/publication/315695709_Flight_Tour_Planning_with_Recharging_Optimization_for_Battery-operated_Autonomous_Drones/links/58fff4cfaca2725bd71e7a69/Flight-Tour-Planning-with-Recharging-Optimization-for-Battery-operated-Autonomous-Drones.pdf.
  • Thibbotuwawa, A. (2019). Unmanned Aerial Vehicle Fleet Mission Planning Subject to Changing Weather Conditions. [Ph. D. Thesis, Og Naturvidenskabelige Fakultet, Aalborg Universitet].
  • Thibbotuwawa, A., Bocewicz, G., Zbigniew, B., and Nielsen, P. (2019). A solution approach for UAV fleet mission planning in changing weather conditions. Appl. Sci., 9.
  • Thibbotuwawa, A., Bocewicz, G., Radzki, G., Nielsen, P., and Banaszak, Z. (2020). UAV Mission planning resistant to weather uncertainty. Sensors, 20.
  • Dorling, (2016), IEEE Trans. Syst. Man, Cybern. Syst., 47, pp. 70, 10.1109/TSMC.2016.2582745
  • Klaine, (2018), Cogn. Comput., 10, pp. 790, 10.1007/s12559-018-9559-8
  • Zhao, (2021), IEEE J. Sel. Areas Commun., 39, pp. 3193, 10.1109/JSAC.2021.3088669
  • Hu, (2018), IEEE Internet Things J., 6, pp. 6177, 10.1109/JIOT.2018.2876513
  • Saxena, (2019), IEEE Trans. Cogn. Commun. Netw., 5, pp. 1101, 10.1109/TCCN.2019.2948324
  • Liu, (2018), IEEE J. Sel. Areas Commun., 36, pp. 2059, 10.1109/JSAC.2018.2864373
  • Kim, (2018), J. Commun. Netw., 20, pp. 452, 10.1109/JCN.2018.000070
  • Zhang, Z., Wu, J., and He, C. (2019, January 27–30). Search Method of disaster inspection coordinated by Multi-UAV. Proceedings of the 2019 Chinese Control Conference (CCC), Guangzhou, China.
  • Zhang, S., and Cheng, W. (2019, January 9–13). Statistical QoS Provisioning for UAV-Enabled Emergency Communication Networks. Proceedings of the 2019 IEEE Globecom Workshops (GC Wkshps), Waikoloa, HI, USA.
  • Reina, (2019), Future Gener. Comput. Syst., 90, pp. 129, 10.1016/j.future.2018.07.048
  • Toral, (2016), Int. J. Distrib. Sens. Netw., 12, pp. 8132812, 10.1155/2016/8132812
  • Ghamry, K.A., Kamel, M.A., and Zhang, Y. (2017, January 13–16). Multiple UAVs in forest fire fighting mission using particle swarm optimization. Proceedings of the 2017 International Conference on Unmanned Aircraft Systems (ICUAS), Miami, FL, USA.
  • Sanci, (2019), Eur. J. Oper. Res., 279, pp. 335, 10.1016/j.ejor.2019.06.012
  • Mekikis, (2017), IEEE Trans. Consum. Electron., 63, pp. 291, 10.1109/TCE.2017.014904
  • Agrawal, (2019), J. Light. Technol., 37, pp. 2352, 10.1109/JLT.2019.2904328
  • Tran, (2016), J. Light. Technol., 34, pp. 5226, 10.1109/JLT.2016.2607171
  • Ma, (2015), IEEE/OSA J. Opt. Commun. Netw., 7, pp. B81, 10.1364/JOCN.7.000B81
  • Msongaleli, (2016), J. Light. Technol., 34, pp. 4293, 10.1109/JLT.2016.2587719
  • Dikbiyik, (2014), J. Light. Technol., 32, pp. 3175, 10.1109/JLT.2014.2334713
  • Abdallah, A., Ali, M.Z., Mišić, J., and Mišić, V.B. (2019). Efficient Security Scheme for Disaster Surveillance UAV Communication Networks. Information, 10.
  • Sutton, (1999), J. Cogn. Neurosci., 11, pp. 126
  • Rummery, G.A., and Niranjan, M. (1994). On-Line Q-Learning Using Connectionist Systems, University of Cambridge, Department of Engineering.
  • Watkins, C.J.C.H. (1989). Learning from Delayed Rewards, University of Cambridge.
  • Watkins, (1992), Mach. Learn., 8, pp. 279, 10.1007/BF00992698
  • Tokic, M., and Palm, G. (2011). Annual Conference on Artificial Intelligence, Springer.
  • Tsitsiklis, (1994), Mach. Learn., 16, pp. 185, 10.1007/BF00993306
  • Thrun, S.B. (2022, October 24). Efficient Exploration in Reinforcement Learning. Available online: https://www.ri.cmu.edu/pub_files/pub1/thrun_sebastian_1992_1/thrun_sebastian_1992_1.pdf.
  • Auer, (2002), J. Mach. Learn. Res., 3, pp. 397
  • Kumar, V., and Webster, M. (2021). Importance Sampling based Exploration in Q Learning. arXiv.
  • Dabney, W., Ostrovski, G., and Barreto, A. (2020). Temporally-extended ε-greedy exploration. arXiv.
  • Ludwig, N. (2022, October 24). 14 CFR Part 107 (UAS)–Drone Operators Are Not Pilots. Available online: https://www.suasnews.com/2017/12/14-cfr-part-107-uas-drone-operators-not-pilots/.
  • Eole, S. (2022, October 24). Windenergie-Daten der Schweiz. Available online: http://www.wind-data.ch/windkarte/.
  • Delgado, (2015), AIP Conference Proceedings, Volume 1734, pp. 030011, 10.1063/1.4949063
  • Fang, (2020), J. Appl. Meteorol. Climatol., 59, pp. 637, 10.1175/JAMC-D-18-0327.1
  • Mohandes, (2021), Appl. Artif. Intell., 35, pp. 605, 10.1080/08839514.2021.1922850
  • Cui, (2021), IEEE Access, 9, pp. 59486, 10.1109/ACCESS.2021.3073704
  • Chung, (2020), IEEE Trans. Ind. Inform., 17, pp. 3032, 10.1109/TII.2020.3004816
  • Hou, Y., Huang, W., Zhou, H., Gu, F., Chang, Y., and He, Y. (2019, January 3–5). Analysis on Wind Resistance Index of Multi-rotor UAV. Proceedings of the 2019 Chinese Control And Decision Conference (CCDC), Nanchang, China.
  • (2022, October 24). Die Website Für Windenergie-Daten der Schweiz. Available online: http://www.wind-data.ch/tools/.
  • Cardoso, E., Natalino, C., Alfaia, R., Souto, A., Araújo, J., Francês, C.R., Chiaraviglio, L., and Monti, P. (2020, January 19–23). A heuristic approach for the design of UAV-based disaster relief in optical metro networks. Proceedings of the 2020 22nd International Conference on Transparent Optical Networks (ICTON), Bari, Italy.
  • Fawaz, (2018), IEEE Commun. Mag., 56, pp. 70, 10.1109/MCOM.2017.1700320
  • Zhou, (2009), Front. Phys. China, 4, pp. 170, 10.1007/s11467-009-0036-4
  • Jarchi, D., and Casson, A.J. (2016). Description of a database containing wrist PPG signals recorded during physical exercise with both accelerometer and gyroscope measures of motion. Data, 2.
  • Yu, L., Yan, X., Kuang, Z., Chen, B., and Zhao, Y. (2019). Driverless bus path tracking based on fuzzy pure pursuit control with a front axle reference. Appl. Sci., 10.