Divergence of Neural Tangent Kernel in Classification Problems
neural tangent kernel;neural network;cross-entropy loss;classification problem
Thank you very much for taking the time to review our work. We truly appreciate your valuable feedback and thoughtful comments. There may be some misunderstandings between us, and we would like to take this opportunity to address them one by one. 

---

 ### 1.  ”*The main contributions of this paper are trivial since the derivation in Eqs. (2.7)-(3.2) is common-used.*“

In our paper, the content of Eqs. (2.7)-(3.2) is as follows:

**Equation (2.7):**
$$
\begin{aligned}
\frac{\mathrm{d}}{\mathrm{~d} t} \theta_{t} & =-\nabla_{\theta} \mathcal{L}\left(\theta_{t}\right)=-\sum_{i=1}^{n} \ell^{\prime}\left(\left(2 y_{i}-1\right) f\left(x_{i} ; \theta_{t}\right)\right)\left(2 y_{i}-1\right) \nabla_{\theta} f\left(x_{i} ; \theta_{t}\right) 
 =\sum_{i=1}^{n} \nabla_{\theta} f\left(x_{i} ; \theta_{t}\right)\left(2 y_{i}-1\right) u_{i} .
\end{aligned}
$$

**Equation (3.1):**
$$
K_{t}\left(x, x^{\prime}\right)=\left\langle\nabla_{\theta} f\left(x ; \theta_{t}\right), \nabla_{\theta} f\left(x^{\prime} ; \theta_{t}\right)\right\rangle .
$$

**Equation (3.2):**
$$
\begin{aligned}
\frac{\mathrm{d}}{\mathrm{~d} t} f\left(x ; \theta_{t}\right)  =\left[\nabla_{\theta} f\left(x ; \theta_{t}\right)\right]^{T}\left[\frac{\partial \theta_{t}}{\partial t}\right] 
 =\sum_{i=1}^{n}\left[\nabla_{\theta} f\left(x ; \theta_{t}\right)\right]^{T}\left[\nabla_{\theta} f\left(x_{i} ; \theta_{t}\right)\right]\left(2 y_{i}-1\right) u_{i}  =\sum_{i=1}^{n} K_{t}\left(x, x_{i}\right)\left(2 y_{i}-1\right) u_{i} .
\end{aligned}
$$

These three equations are not our core results but are used to set up the problem. They are standard in NTK theory. Specifically:
1. Equation (2.7) describes the dynamics of neural network parameters under gradient descent.
2. Equation (3.1) provides the standard definition of the (empirical) Neural Tangent Kernel (NTK).
3. Equation (3.2) describes the dynamics of the neural network function during gradient descent.

Our theoretical contribution lies in **Theorem 2**, which demonstrates the divergence property of the NTK. In Theorem 2, we state that as the training time  $t \to \infty$ , the empirical NTK  $K_t(x, x{\prime})$  may not converges, which offers a perspective that differs from the conventional assumption of NTK convergence in regression tasks. We greatly value your feedback and hope this clarification helps address your concerns.

---

### 2. ” *Besides, the finding that the empirical NTK does not consistently converge but instead diverges as time approaches infinity has been observed by existing studies.*“

In fact, to the best of our knowledge, we have not seen studies addressing the divergence of the NTK in previous literature, especially in classification problem. For regression problems under the Mean Squared Error (MSE) loss, it is commonly believed that the NTK converges [Allen-Zhu et al., 2019; Xu and Zhu, 2024].  If you are aware of works that explore the divergence of the NTK, we would be sincerely grateful if you could share them with us. Thank you for your valuable insights. 

---

### References

[Allen-Zhu et al., 2019] Allen-Zhu, Z., Li, Y., and Song, Z. (2019). "A convergence theory for deep learning via over-parameterization." In _International Conference on Machine Learning_, pages 242–252. PMLR.  
[Xu and Zhu, 2024] Xu, J., and Zhu, H. (2024). "Overparametrized Multi-layer Neural Networks: Uniform Concentration of Neural Tangent Kernel and Convergence of Stochastic Gradient Descent." _Journal of Machine Learning Research_, 25(94), 1–83.
Yes
No ethics review needed.
See previous sections.
The result appears to be sound and certainly has relevance insofar as it demonstrates the non-applicability of many NTK results in a common scenario (training with cross-entropy).  The presentation is mostly readable and the derivation appears to be correct, though admittedly I did not dive too deeply into the appendices, and certainly the material appears novel (at least to me).
The paper considers the convergence of the neural tangent kernel of fully-connected ReLU networks and ResNets for classification problems trained with cross-entropy loss functions.  The key result of the paper is a proof that, during training (in the gradient-flow approximation), the network parameters diverge due to the strict positive definiteness of the NTK, implying that many results from NTK theory may not be applicable in this context.
While I don't question to novelty of the result in literature, I don't necessarily find it too surprising.  To truly minimize the cross-entropy would require $f(x_i,\theta) \to +\infty$ for $y_i=+1$ (and similarly for the other class).  Without regularization and given infinite time, recalling that the NN is a universal approximator in the infinite-width limit, you should expect precisely this, in which light theorem 1 (and subsequently corollary 1 and theorem 2) is to be expected.  Nevertheless it is important to see this intuition proven formally, so I do not consider this a serious criticism.

Minor points - the presentation of the paper could certainly be improved, for example:

- line 163: you don't optimize the parameters with gradient flow.  Gradient flow is used to (approximately) analyze the optimization process.
- line 185: this is badly written.  You already defined $K_t$ in (3.1) - why are you repeating it in (3.3) as though it was new?
- line 283: "...will converges to NTK with probability as width comes to infinity..." Do you mean "will converge with probability 1" or "will converge in probability"?
- Theorem 2: (4.4) is precisely equivalent to (4.3).  Are these meant to refer to different networks (ie should (4.4) be modified to refer to ResNet)?
Yes
N/A
No ethics review needed.
- The obtained results rely on the positivity of the NNK. Could you clarify the required conditions (e.g., number of neurons) for this assumption?

- Could you discuss the relationships with the related papers mentioned in the weaknesses?
- The paper proves the divergence of NNK during training, indicating the inapplicability of NTK theory for regression problems.
- The paper is well-structured and easy to read.
This paper investigates the divergent properties of neural network kernels (NNK, empirical NTK) in classification problems using the softmax loss. Specifically, it demonstrates that the NTKs of fully connected neural networks and ResNets are strictly positive, and that model outputs and parameters will diverge as long as the NNK is bounded below by a positive constant during training (Theorem 1 and Corollary 1). Additionally, it is shown that, unlike in regression problems, the NNK is not fixed during training (Theorem 2).
- The divergence result of the NNK is not particularly surprising, as it is evident that parameters and outputs must diverge to minimize standard classification losses such as logistic, softmax, or exponential losses. For example, refer to the following papers: 

[D. Soudry, et al] THE IMPLICIT BIAS OF GRADIENT DESCENT ON SEPARABLE DATA. ICLR 2018.

[Z. Ji and M. Telgarsky] The implicit bias of gradient descent on nonseparable data. COLT, 2019.

- Given the scale of $\lambda_0$, which typically degenerates as $n\to \infty$, I am curious about the significance of the deviation $\lambda_0/2n^2$, which goes to $0$ as well.

- The results do not address the convergence of gradient descent. If the positivity of the NNK (Eq. (5.3)) holds, convergence can be shown as in NTK theory, but this requirement seems redundant for classification problems. The NTK separability assumption, a weaker condition than positivity, is sufficient for proving the convergence of gradient descent. See the following papers for details:

[A. Nitanda, G. Chinot, and T. Suzuki] Gradient descent can learn less over-parameterized two-layer neural networks on classification problems. 2019.

[Z. Ji and M. Telgarsky] Polylogarithmic width suffices for gradient descent to achieve arbitrarily small test error with shallow relu networks. ICLR 2020.

[H. Taheri and C. Thrampoulidis] Generalization and Stability of Interpolating Neural Networks with Minimal Width. JMLR 2024.

- These related works are not discussed in the paper. 

- (Minor) 
A few typos: (1) Line 110: $f(x_1, f(x_2),$ (2) In the second statement of Theorem 2: Should this be the result for $K_t^{Res}$?
Yes
No ethics review needed.
1. In Theorem 1, it is assumed that the minimum eigenvalue of the NNK matrix has a lower bound. However, the eigenvalues of the NNK Gram matrix approach zero as the number of training examples increases, as demonstrated in the two papers below. Can this observation offer any insights into NTK theory for classification problems?

[1] Lili Su and Pengkun Yang. On learning over-parameterized neural networks: A functional approximation perspective. In Advances in Neural Information Processing Systems, pp. 2637–2646, 2019.

[2] Atsushi Nitanda and Suzuki Taiji. Optimal rates for averaged stochastic gradient descent under neural tangent kernel regime. In International Conference on Learning Representations, 2021.

2. Could the scope be broadened to incorporate other classification loss functions in order to gain a more comprehensive understanding of NTK behavior?

3. What will happen if the data is completely separable or follows a Gaussian mixture in classification problems?
1.	The paper provides valuable insights into the behavior of the NTK in classification problems, highlighting a significant divergence from established understandings in regression tasks.

2.	By demonstrating that NTK does not uniformly converge in classification scenarios, the authors identify a critical limitation of NTK theory.
This paper explores the convergence properties of the Neural Tangent Kernel (NTK) in classification tasks, with a focus on multi-layer fully connected neural networks and residual neural networks. The authors establish that the NTK of both types of networks is strictly positive definite, a key characteristic for comprehending the training behavior of neural networks. Using a contradiction argument, the authors demonstrate that when employing the cross-entropy loss function, the parameters of the neural network tend to diverge due to the strictly positive definiteness of the NTK. The results imply that NTK theory may not hold in the context of classification problems, leading to significant theoretical implications for understanding the dynamics of neural networks.
1. The claim that current NTK theories might not fully explain generalization properties in classification tasks could be seen as an overgeneralization without comprehensive evidence. A more nuanced discussion on the conditions under which NTK theory might still apply could add depth to the argument.

2. Although the paper recognizes the need for new theoretical tools, it doesn't provide specific directions or approaches for developing these frameworks. Offering some initial ideas or suggestions could help guide future research.
Yes
No ethics review needed.
Nothing.
This paper is easy to follow.
This paper showed that the multi-layer fully connected neural networks and residual neural networks satisfy the NTK in classification problems. Based on this building, this paper shows the positive definiteness and the convergence of NTK.
The main contributions of this paper are trivial since the derivation in Eqs. (2.7)-(3.2) is common-used. Besides, the finding that the empirical NTK does not consistently converge but instead diverges as time approaches infinity has been observed by existing studies.
This paper investigates the divergence of NTK in fully connected and residual networks during classification tasks, showing that NTK theory is not applicable under the cross-entropy loss function as the empirical NTK fails to converge over time.
@inproceedings{
anonymous2024divergence,
title={Divergence of Neural Tangent Kernel in Classification Problems},
author={Anonymous},
booktitle={Submitted to The Thirteenth International Conference on Learning Representations},
year={2024},
url={https://openreview.net/forum?id=VEJzjAvaIy},
note={under review}
}
This paper primarily investigates the convergence of the Neural Tangent Kernel (NTK) in classification problems. This study firstly show the strictly positive definiteness of NTK of multi-layer fully connected neural networks and residual neural networks. Then, through a contradiction argument,  it indicates that, during training with the cross-entropy loss function, the neural network parameters diverge due to the strictly positive definiteness of the NTK. Consequently, the empirical NTK does not consistently converge but instead diverges as time approaches infinity. This finding implies that NTK theory is not applicable in this context, highlighting significant theoretical implications for the study of neural networks in classification problems. These results can also  be easily generalized to other network structures, provided that the NTK is strictly positive definite.
I certify that there is no URL (e.g., github page) that could be used to find authors’ identity.
I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics.
neural tangent kernel neural network cross-entropy loss classification problem
I certify that there is no acknowledgement section in this submission for double blind review.
/pdf/87f5a882a914e8af1d7b27ef3bf9a8549d73304d.pdf
learning theory
I understand the reciprocal reviewing requirement as described on https://iclr.cc/Conferences/2025/CallForPapers. If none of the authors are registered as a reviewer, it may result in a desk rejection at the discretion of the program chairs. To request an exception, please complete this form at https://forms.gle/Huojr6VjkFxiQsUp6.
I certify that this submission complies with the submission instructions as described on https://iclr.cc/Conferences/2025/AuthorGuide.
Divergence of Neural Tangent Kernel in Classification Problems
ICLR 2025 Conference Submission
ICLR.cc/2025/Conference/Submission