Interplay between depth and width for interpolation in neural ODEs

Vista sencilla de ítem
dc.contributor.authorÁlvarez López, Antonio
dc.contributor.authorSlimane, Arselane Hadj
dc.contributor.authorZuazua, Enrique
dc.date.accessioned2025-03-06T11:44:19Z
dc.date.available2025-03-06T11:44:19Z
dc.date.issued2024-12
dc.date.updated2025-03-06T11:44:19Z
dc.description.abstractNeural ordinary differential equations have emerged as a natural tool for supervised learning from a control perspective, yet a complete understanding of the role played by their architecture remains elusive. In this work, we examine the interplay between the width p and the number of transitions between layers L (corresponding to a depth of L+1). Specifically, we construct explicit controls interpolating either a finite dataset D, comprising N pairs of points in Rd, or two probability measures within a Wasserstein error margin ɛ>0. Our findings reveal a balancing trade-off between p and L, with L scaling as 1+O(N/p) for data interpolation, and as 1+Op−1+(1+p)−1ɛ−d for measures. In the high-dimensional and wide setting where d,p>N, our result can be refined to achieve L=0. This naturally raises the problem of data interpolation in the autonomous regime, characterized by L=0. We adopt two alternative approaches: either controlling in a probabilistic sense, or by relaxing the target condition. In the first case, when p=N we develop an inductive control strategy based on a separability assumption whose probability increases with d. In the second one, we establish an explicit error decay rate with respect to p which results from applying a universal approximation theorem to a custom-built Lipschitz vector field interpolating D.en
dc.description.sponsorshipThis paper was supported by the Madrid Government (Comunidad de Madrid – Spain) under the multiannual Agreement with UAM in the line for the Excellence of the University Research Staff in the context of the V PRICIT (Regional Programme of Research and Technological Innovation). A. Álvarez-López has been funded by a contract FPU21/05673 from the Spanish Ministry of Universities. A. Hadj Slimane has been funded by École Normale Supérieur Paris-Saclay and Université Paris-Saclay. E. Zuazua has been funded by the Alexander von Humboldt-Professorship program, ModConFlex Marie Curie Action, HORIZON-MSCA-2021-DN-01, COST Action MAT-DYN-NET, Transregio 154 Project ‘‘Mathematical Modelling, Simulation and Optimization Using the Example of Gas Networks’’ of the DFG, grants PID2020-112617GB-C22 and TED2021-131390B-I00 of MICINN (Spain)en
dc.identifier.citationÁlvarez-López, A., Slimane, A. H., & Zuazua, E. (2024). Interplay between depth and width for interpolation in neural ODEs. Neural Networks, 180. https://doi.org/10.1016/J.NEUNET.2024.106640
dc.identifier.doi10.1016/J.NEUNET.2024.106640
dc.identifier.eissn1879-2782
dc.identifier.issn0893-6080
dc.identifier.urihttp://hdl.handle.net/20.500.14454/2468
dc.language.isoeng
dc.publisherElsevier Ltd
dc.rights© 2024 The Authors
dc.subject.otherDepth
dc.subject.otherNeural ODEs
dc.subject.otherSimultaneous controllability
dc.subject.otherTransport control
dc.subject.otherWasserstein distance
dc.subject.otherWidth
dc.titleInterplay between depth and width for interpolation in neural ODEsen
dc.typejournal article
dcterms.accessRightsopen access
oaire.citation.titleNeural Networks
oaire.citation.volume180
oaire.licenseConditionhttps://creativecommons.org/licenses/by/4.0/
oaire.versionVoR
Archivos
Bloque original
Mostrando 1 - 1 de 1
Miniatura
Nombre:
alvarez_interplay_2024.pdf
Tamaño:
1.57 MB
Formato:
Adobe Portable Document Format
Descargar
Colecciones
Artículos