Optimización de aplicaciones científicas y de aprendizaje automático en entornos de altas prestaciones heterogéneos

Abstract

The improvement of high-performance computing platforms has leveraged the acceleration and optimization of computationally intensive applications. Since these applications consume a large amount of resources and time, its optimization has been a key point of research. In high-performance computing platforms the optimization of these applications have been addressed using multiple techniques. In this sense, the workload distribution technique is widely used. This technique consists in the distribution of the workload between the processes deployed on the different computing devices that compose the platform. Current distributed applications usually perform a homogeneous workload partitioning between processes composing the application without taking into account the heterogeneous features of the resources in which they execute. As a consequence, non-optimal partitioning leads to longer execution times. Thus, a heterogeneous distribution according to the capabilities of each process is needed. In order to achieve an optimal workload distribution it is necessary to model the heterogeneity of the platform resources. The analytical computation and communication models have traditionally been used for modeling that resources capabilities. This thesis proposes different methodologies with the objective of improving the performance of high computational cost applications in heterogeneous platforms. The idea is to characterize the computational capabilities of the processes involved in the execution of the applications, and then perform a partition and distribution of the workload heterogeneously in terms of such capabilities. To evaluate our proposal, experiments with common scientific kernels and neural network based applications are performed to demonstrate the advantages of our proposal.