INTERNATIONAL JOURNAL OF ARTIFICIAL INTELLIGENCE AND AGENT SYSTEMS

Parameterized Nonlinearities in Feedforward Networks: An Adaptive Function Evaluation for Efficient Regression

1.Compunnel Inc.

Published in issue

October 2026

Volume / Issue

Vol. 1 No. 1

Access

Open access

View PDF

The article details presented on this page reflect the published version of record. Readers may access the full-text PDF, review the citation information, and export the reference in supported citation formats.

Abstract

Adaptive activation functions have emerged as a promising approach for improving neural network expressiveness without increasing architectural complexity. This paper investigates whether trainable nonlinearities can enhance regression performance in compact feedforward networks operating under limited computational budgets. We propose a modified Extreme Learning Machine (ELM) framework in which activation functions are parameterized and learned through two adaptation strategies: layer-wide parameter sharing and neuron-specific parameterization. Four activation functions, quadratic, PReLU, sigmoid, and swish, are evaluated in both their standard and adaptive forms across ten diverse regression benchmark datasets. Experimental results demonstrate that adaptive parameterization consistently improves predictive performance over fixed activation functions while introducing only a modest increase in computational cost. Neuron-specific adaptation achieves the lowest root mean squared error (RMSE) in the majority of datasets, whereas layer-wide adaptation offers a favorable trade-off between accuracy and parameter efficiency. Among the investigated nonlinearities, adaptive quadratic functions yield the largest improvements, with average error reductions exceeding 18% relative to their standard counterparts. Statistical analyses using one-way ANOVA and Tukey's post-hoc tests confirm that the observed performance gains are significant across most datasets. These findings indicate that functional plasticity through trainable activation parameters provides an effective alternative to increasing network depth or width, enabling more accurate and computationally efficient regression models for resource-constrained applications.
Keywords:Parameterized NonlinearitiesAdaptive Activation FunctionsResource-Constrained LearningRegression ModelingEfficient Neural NetworksExtreme Learning Machine (ELM)

Cite this Article

Use the following citation when referencing this publication in academic or professional work.

J. Gupta, "Parameterized Nonlinearities in Feedforward Networks: An Adaptive Function Evaluation for Efficient Regression", Int. J. Artif. Intell. Agent Syst., vol. 1, no. 1, Oct. 2026, doi: https://doi.org/10.67231/4hh9ns95.

DOWNLOAD CITATION

Parameterized Nonlinearities in Feedforward Networks: An Adaptive Function Evaluation for Efficient Regression | International Journal of Artificial Intelligence and Agent Systems