For an @math -variate order- @math tensor @math , define @math to be the maximum value taken by the tensor on the unit sphere. It is known that for a random tensor with i.i.d @math entries, @math w.h.p. We study the problem of efficiently certifying upper bounds on @math via the natural relaxation from the Sum of Squares (SoS) hierarchy. Our results include: - When @math is a random order- @math tensor, we prove that @math levels of SoS certifies an upper bound @math on @math that satisfies [ B A_ ( n q^ ,1-o(1) )^ q 4-1 2 w.h.p. ] Our upper bound improves a result of Montanari and Richard (NIPS 2014) when @math is large. - We show the above bound is the best possible up to lower order terms, namely the optimum of the level- @math SoS relaxation is at least [ A_ ( n q^ ,1+o(1) )^ q 4-1 2 . ] - When @math is a random order- @math tensor, we prove that @math levels of SoS certifies an upper bound @math on @math that satisfies [ B A_ ( O (n) q )^ d 4 - 1 2 w.h.p. ] For growing @math , this improves upon the bound certified by constant levels of SoS. This answers in part, a question posed by Hopkins, Shi, and Steurer (COLT 2015), who established the tight characterization for constant levels of SoS.
@cite_11: We study a statistical model for the tensor principal component analysis problem introduced by Montanari and Richard: Given a order- @math tensor @math of the form @math , where @math is a signal-to-noise ratio, @math is a unit vector, and @math is a random noise tensor, the goal is to recover the planted vector @math . For the case that @math has iid standard Gaussian entries, we give an efficient algorithm to recover @math whenever @math , and certify that the recovered vector is close to a maximum likelihood estimator, all with high probability over the random choice of @math . The previous best algorithms with provable guarantees required @math . In the regime @math , natural tensor-unfolding-based spectral relaxations for the underlying optimization problem break down (in the sense that their integrality gap is large). To go beyond this barrier, we use convex relaxations based on the sum-of-squares method. Our recovery algorithm proceeds by rounding a degree- @math sum-of-squares relaxations of the maximum-likelihood-estimation problem for the statistical model. To complement our algorithmic results, we show that degree- @math sum-of-squares relaxations break down for @math , which demonstrates that improving our current guarantees (by more than logarithmic factors) would require new techniques or might even be intractable. Finally, we show how to exploit additional problem structure in order to solve our sum-of-squares relaxations, up to some approximation, very efficiently. Our fastest algorithm runs in nearly-linear time using shifted (matrix) power iteration and has similar guarantees as above. The analysis of this algorithm also confirms a variant of a conjecture of Montanari and Richard about singular vectors of tensor unfoldings.
@cite_8: We consider the Principal Component Analysis problem for large tensors of arbitrary order @math under a single-spike (or rank-one plus noise) model. On the one hand, we use information theory, and recent results in probability theory, to establish necessary and sufficient conditions under which the principal component can be estimated using unbounded computational resources. It turns out that this is possible as soon as the signal-to-noise ratio @math becomes larger than @math (and in particular @math can remain bounded as the problem dimensions increase). On the other hand, we analyze several polynomial-time estimation algorithms, based on tensor unfolding, power iteration and message passing ideas from graphical models. We show that, unless the signal-to-noise ratio diverges in the system dimensions, none of these approaches succeeds. This is possibly related to a fundamental limitation of computationally tractable estimators for this problem. We discuss various initializations for tensor power iteration, and show that a tractable initialization based on the spectrum of the matricized tensor outperforms significantly baseline methods, statistically and computationally. Finally, we consider the case in which additional side information is available about the unknown signal. We characterize the amount of side information that allows the iterative algorithms to converge to a good estimate.
Montanari and Richard @cite_8 presented a @math -time algorithm that can certify that the optimal value of @math for a random @math -tensor is at most @math with high probability. Hopkins, Shi, and Steurer @cite_11 improved it to @math with the same running time. They also asked how many levels of SoS are required to certify a bound of @math for @math .