Duality Based Generalization Bounds for Parallel Positively Homogeneous Networks
statistical learning theory;non-convex optimization;high dimension;sample complexity;matrix sensing;neural network;positively homogeneous;generalization
I have read and agree with the venue's withdrawal policy on behalf of myself and my co-authors.
A general framework for deriving generalization bounds for parallel positively homogeneous neural networks.
@misc{
tadipatri2024duality,
title={Duality Based Generalization Bounds for Parallel Positively Homogeneous Networks},
author={Uday Kiran Reddy Tadipatri and Benjamin David Haeffele and Joshua Agterberg and Rene Vidal},
year={2024},
url={https://openreview.net/forum?id=LWKHLHroPk}
}
We propose a general framework for deriving generalization bounds for parallel positively homogeneous neural
networks--a class of neural networks whose input-output map decomposes as the sum of positively homogeneous maps. 
Examples of such networks include matrix factorization/sensing, two-layer linear/ReLU networks, 
single-layer multi-head attention mechanisms, and tensor factorization.
Our general framework is based on linking the non-convex empirical risk minimization (ERM) problem of interest to a 
closely related convex optimization problem over prediction functions, which provides a global, achievable 
lower-bound to the original ERM problem. We exploit this convex lower-bound to perform generalization
analysis in the convex space while controlling the discrepancy between the convex model and its non-convex
counterpart. We apply our general framework to a wide variety of models ranging from low-rank matrix sensing,
to structured matrix sensing, two-layer linear networks, two-layer ReLU networks, and single-layer 
multi-head attention mechanisms, achieving generalization bounds with a sample complexity that scales nearly
linear with the network width.
I certify that there is no URL (e.g., github page) that could be used to find authors’ identity.
~Uday_Kiran_Reddy_Tadipatri1 ~Benjamin_David_Haeffele1 ~Joshua_Agterberg1 ~Rene_Vidal1
Uday Kiran Reddy Tadipatri Benjamin David Haeffele Joshua Agterberg Rene Vidal
I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics.
statistical learning theory non-convex optimization high dimension sample complexity matrix sensing neural network positively homogeneous generalization
I certify that there is no acknowledgement section in this submission for double blind review.
tadipatri|duality_based_generalization_bounds_for_parallel_positively_homogeneous_networks
learning theory
I understand the reciprocal reviewing requirement as described on https://iclr.cc/Conferences/2025/CallForPapers. If none of the authors are registered as a reviewer, it may result in a desk rejection at the discretion of the program chairs. To request an exception, please complete this form at https://forms.gle/Huojr6VjkFxiQsUp6.
I certify that this submission complies with the submission instructions as described on https://iclr.cc/Conferences/2025/AuthorGuide.
Duality Based Generalization Bounds for Parallel Positively Homogeneous Networks
ICLR 2025 Conference Withdrawn Submission
ICLR.cc/2025/Conference/Withdrawn_Submission