On the improvement of generalization and stability of forward-only learning via neural polarization

Vista sencilla de ítem
dc.contributor.authorTerres Escudero, Erik B.
dc.contributor.authorSer Lorente, Javier del
dc.contributor.authorGarcía Bringas, Pablo
dc.date.accessioned2025-06-03T10:55:22Z
dc.date.available2025-06-03T10:55:22Z
dc.date.issued2024-10-16
dc.date.updated2025-06-03T10:55:22Z
dc.descriptionPonencia presentada en la 27th European Conference on Artificial Intelligence, ECAI, celebrada en Santiago de Compostela, entre el 19 y el 24 de octubre de 2024es
dc.description.abstractForward-only learning algorithms have recently gained attention as alternatives to gradient backpropagation, replacing the backward step of this latter solver with an additional contrastive forward pass. Among these approaches, the so-called Forward-Forward Algorithm (FFA) has been shown to achieve competitive levels of performance in terms of generalization and complexity. Networks trained using FFA learn to contrastively maximize a layer-wise defined goodness score when presented with real data (denoted as positive samples) and to minimize it when processing synthetic data (corr. negative samples). However, this algorithm still faces weaknesses that negatively affect the model accuracy and training stability, primarily due to a gradient imbalance between positive and negative samples. To overcome this issue, in this work we propose a novel implementation of the FFA algorithm, denoted as Polar-FFA, which extends the original formulation by introducing a neural division (polarization) between positive and negative instances. Neurons in each of these groups aim to maximize their goodness when presented with their respective data type, thereby creating a symmetric gradient behavior. To empirically gauge the improved learning capabilities of our proposed Polar-FFA, we perform several systematic experiments using different activation and goodness functions over image classification datasets. Our results demonstrate that Polar-FFA outperforms FFA in terms of accuracy and convergence speed. Furthermore, its lower reliance on hyperparameters reduces the need for hyperparameter tuning to guarantee optimal generalization capabilities, thereby allowing for a broader range of neural network configurations.en
dc.description.sponsorshipThe authors thank the Basque Government for its funding support via the consolidated research groups MATHMODE (ref. T1256-22) and D4K (ref. IT1528-22), and the colaborative ELKARTEK project KK-2023/00012 (BEREZ-IA). E. B. Terres-Escudero is supported by a PIF research fellowship granted by the University of Deustoen
dc.identifier.citationTerres-Escudero, E. B., Del Ser, J., & Garcia-Bringas, P. (2024). On the improvement of generalization and stability of forward-only learning via neural polarization. Frontiers in Artificial Intelligence and Applications, 392, 1919-1926. https://doi.org/10.3233/FAIA240706
dc.identifier.doi10.3233/FAIA240706
dc.identifier.eissn1879-8314
dc.identifier.isbn9781643685489
dc.identifier.issn0922-6389
dc.identifier.urihttp://hdl.handle.net/20.500.14454/2920
dc.language.isoeng
dc.publisherIOS Press BV
dc.rights© 2024 The Authors
dc.titleOn the improvement of generalization and stability of forward-only learning via neural polarizationen
dc.typeconference paper
dcterms.accessRightsopen access
oaire.citation.endPage1926
oaire.citation.startPage1919
oaire.citation.titleFrontiers in Artificial Intelligence and Applications
oaire.citation.volume392
oaire.licenseConditionhttps://creativecommons.org/licenses/by-nc/4.0/
oaire.versionVoR
Archivos
Bloque original
Mostrando 1 - 1 de 1
Miniatura
Nombre:
terres_ontheimprovement_2024.pdf
Tamaño:
698.43 KB
Formato:
Adobe Portable Document Format
Descargar
Colecciones
Ponencias