Treinamento de funções de ativação em redes neurais artificiais: uma abordagem com regressão polinomial

Abstract

Artificial neural networks are widely applied to complex problems; however, selecting an optimal activation function (AF) remains a challenge, particularly due to the lack of consistent criteria for relating data characteristics to the most appropriate function. In light of these limitations, recent advances highlight the potential of trainable activation functions, which dynamically adjust during training to improve network performance. Such a mechanism can also be leveraged to insert new hidden layers into stacked autoencoder networks, promoting greater structural depth without compromising training stability. In this context, this work proposes a novel approach based on polynomial regression to develop trainable activation functions. The methodology introduces a collaborative layer insertion mechanism, enabling the integration of new layers without degrading the knowledge acquired in previous ones, while simultaneously enhancing the network’s adaptive capacity and performance through the use of error estimation as a criterion for dynamically adjusting the outputs of hidden neurons, in comparison with fixed activation functions. The proposed approach is evaluated on eight benchmarking datasets extracted from the OpenML platform, totaling 4,800 experiments, analyzing its impact on training stability, adaptability, and overall network performance compared to the traditional sigmoid activation function.