Some mathematical properties of flexible hyperbolic tangent activation function with application to deep neural networks

Abstract

This study rigorously delves into some analytic properties of an improved activation function (referred as flx tanh). The determined results appear as a generalization of some results known in the literature. The flx tanh is created by taking into account symmetry property, parameterizability, and deformability of the classical tanh function. Under the regime of certain parameters, we examine the behaviours of flx tanh. Moreover, these dynamic properties of the function flx tanh yields promising results as an activation function in deep neural networks. We utilize the PyTorch library running on Python 3.9 to evaluate the performance of our activation function. Additionally, we aim to encourage the readers to improve their computer programming language skills by making the Python 3.9 codes available on GitHub.