• ### Detection of Planted Solutions for Flat Satisfiability Problems(1502.06144)

March 6, 2019 math.ST, stat.TH, cs.CC, cs.LG
We study the detection problem of finding planted solutions in random instances of flat satisfiability problems, a generalization of boolean satisfiability formulas. We describe the properties of random instances of flat satisfiability, as well of the optimal rates of detection of the associated hypothesis testing problem. We also study the performance of an algorithmically efficient testing procedure. We introduce a modification of our model, the light planting of solutions, and show that it is as hard as the problem of learning parity with noise. This hints strongly at the difficulty of detecting planted flat satisfiability for a wide class of tests.
• ### Optimal link prediction with matrix logistic regression(1803.07054)

March 19, 2018 math.ST, stat.TH, stat.ML
We consider the problem of link prediction, based on partial observation of a large network, and on side information associated to its vertices. The generative model is formulated as a matrix logistic regression. The performance of the model is analysed in a high-dimensional regime under a structural assumption. The minimax rate for the Frobenius-norm risk is established and a combinatorial estimator based on the penalised maximum likelihood approach is shown to achieve it. Furthermore, it is shown that this rate cannot be attained by any (randomised) algorithm computable in polynomial time under a computational complexity assumption.
• ### Fast Rates for Bandit Optimization with Upper-Confidence Frank-Wolfe(1702.06917)

Sept. 6, 2017 math.OC, cs.LG, stat.ML
We consider the problem of bandit optimization, inspired by stochastic optimization and online learning problems with bandit feedback. In this problem, the objective is to minimize a global loss function of all the actions, not necessarily a cumulative loss. This framework allows us to study a very general class of problems, with applications in statistics, machine learning, and other fields. To solve this problem, we analyze the Upper-Confidence Frank-Wolfe algorithm, inspired by techniques for bandits and convex optimization. We give theoretical guarantees for the performance of this algorithm over various classes of functions, and discuss the optimality of these results.
• ### Exact recovery in the Ising blockmodel(1612.03880)

Feb. 2, 2017 math.ST, stat.TH, cs.DS
We consider the problem associated to recovering the block structure of an Ising model given independent observations on the binary hypercube. This new model, called the Ising blockmodel, is a perturbation of the mean field approximation of the Ising model known as the Curie-Weiss model: the sites are partitioned into two blocks of equal size and the interaction between those of the same block is stronger than across blocks, to account for more order within each block. We study probabilistic, statistical and computational aspects of this model in the high-dimensional case when the number of sites may be much larger than the sample size.
• ### Statistical and computational trade-offs in estimation of sparse principal components(1408.5369)

Sept. 28, 2016 math.ST, stat.TH, stat.ML
In recent years, sparse principal component analysis has emerged as an extremely popular dimension reduction technique for high-dimensional data. The theoretical challenge, in the simplest case, is to estimate the leading eigenvector of a population covariance matrix under the assumption that this eigenvector is sparse. An impressive range of estimators have been proposed; some of these are fast to compute, while others are known to achieve the minimax optimal rate over certain Gaussian or sub-Gaussian classes. In this paper, we show that, under a widely-believed assumption from computational complexity theory, there is a fundamental trade-off between statistical and computational performance in this problem. More precisely, working with new, larger classes satisfying a restricted covariance concentration condition, we show that there is an effective sample size regime in which no randomised polynomial time algorithm can achieve the minimax optimal rate. We also study the theoretical performance of a (polynomial time) variant of the well-known semidefinite relaxation estimator, revealing a subtle interplay between statistical and computational efficiency.
• ### Average-case Hardness of RIP Certification(1605.09646)

May 31, 2016 math.ST, stat.TH, cs.CC, cs.LG, stat.ML
The restricted isometry property (RIP) for design matrices gives guarantees for optimal recovery in sparse linear models. It is of high interest in compressed sensing and statistical learning. This property is particularly important for computationally efficient recovery methods. As a consequence, even though it is in general NP-hard to check that RIP holds, there have been substantial efforts to find tractable proxies for it. These would allow the construction of RIP matrices and the polynomial-time verification of RIP given an arbitrary matrix. We consider the framework of average-case certifiers, that never wrongly declare that a matrix is RIP, while being often correct for random instances. While there are such functions which are tractable in a suboptimal parameter regime, we show that this is a computationally hard task in any better regime. Our results are based on a new, weaker assumption on the problem of detecting dense subgraphs.
• ### Optimal Testing for Planted Satisfiability Problems(1401.2205)

Feb. 8, 2015 math.PR, math.ST, stat.TH, cs.CC
We study the problem of detecting planted solutions in a random satisfiability formula. Adopting the formalism of hypothesis testing in statistical analysis, we describe the minimax optimal rates of detection. Our analysis relies on the study of the number of satisfying assignments, for which we prove new results. We also address algorithmic issues, and give a computationally efficient test with optimal statistical performance. This result is compared to an average-case hypothesis on the hardness of refuting satisfiability of random formulas.
• ### Resource Allocation for Statistical Estimation(1412.6613)

Dec. 20, 2014 stat.AP
Statistical estimation in many contemporary settings involves the acquisition, analysis, and aggregation of datasets from multiple sources, which can have significant differences in character and in value. Due to these variations, the effectiveness of employing a given resource (e.g., a sensing device or computing power) for gathering or processing data from a particular source depends on the nature of that source. As a result, the appropriate division and assignment of a collection of resources to a set of data sources can substantially impact the overall performance of an inferential strategy. In this expository article, we adopt a general view of the notion of a resource and its effect on the quality of a data source, and we describe a framework for the allocation of a given set of resources to a collection of sources in order to optimize a specified metric of statistical efficiency. We discuss several stylized examples involving inferential tasks such as parameter estimation and hypothesis testing based on heterogeneous data sources, in which optimal allocations can be computed either in closed form or via efficient numerical procedures based on convex optimization.
• ### Optimal detection of sparse principal components in high dimension(1202.5070)

Sept. 19, 2013 math.ST, stat.TH, stat.ML
We perform a finite sample analysis of the detection levels for sparse principal components of a high-dimensional covariance matrix. Our minimax optimal test is based on a sparse eigenvalue statistic. Alas, computing this test is known to be NP-complete in general, and we describe a computationally efficient alternative test using convex relaxations. Our relaxation is also proved to detect sparse principal components at near optimal detection levels, and it performs well on simulated datasets. Moreover, using polynomial time reductions from theoretical computer science, we bring significant evidence that our results cannot be improved, thus revealing an inherent trade off between statistical and computational performance.
• ### Computational Lower Bounds for Sparse PCA(1304.0828)

April 26, 2013 math.ST, stat.TH, cs.CC, stat.ML
In the context of sparse principal component detection, we bring evidence towards the existence of a statistical price to pay for computational efficiency. We measure the performance of a test by the smallest signal strength that it can detect and we propose a computationally efficient method based on semidefinite programming. We also prove that the statistical performance of this test cannot be strictly improved by any computationally efficient method. Our results can be viewed as complexity theoretic lower bounds conditionally on the assumptions that some instances of the planted clique problem cannot be solved in randomized polynomial time.
{"ft_lang_label":"__label__en","ft_lang_prob":0.89359564,"math_prob":0.9305527,"size":7902,"snap":"2020-10-2020-16","text_gpt3_token_len":1539,"char_repetition_ratio":0.11952393,"word_repetition_ratio":0.004219409,"special_character_ratio":0.16881803,"punctuation_ratio":0.08486239,"nsfw_num_words":0,"has_unicode_error":false,"math_prob_llama3":0.9858846,"pos_list":[0],"im_url_duplicate_count":[null],"WARC_HEADER":"{\"WARC-Type\":\"response\",\"WARC-Date\":\"2020-03-30T16:01:20Z\",\"WARC-Record-ID\":\"<urn:uuid:be8d60f2-661c-4193-9f0b-f5e9a88de334>\",\"Content-Length\":\"74849\",\"Content-Type\":\"application/http; msgtype=response\",\"WARC-Warcinfo-ID\":\"<urn:uuid:c4b75531-fcd0-4fa2-8ddb-c792367d2561>\",\"WARC-Concurrent-To\":\"<urn:uuid:7cccdd60-8271-45fb-9883-b04198bfc695>\",\"WARC-IP-Address\":\"194.95.114.13\",\"WARC-Target-URI\":\"https://labs.tib.eu/arxiv/?author=Quentin%20Berthet\",\"WARC-Payload-Digest\":\"sha1:ZPPG2ORY755TY54PCMSIV7XQ2WYYR4VN\",\"WARC-Block-Digest\":\"sha1:AGAGBXBLW7DCZOM4ZDDT6HM5KYIRX7ZP\",\"WARC-Identified-Payload-Type\":\"text/html\",\"warc_filename\":\"/cc_download/warc_2020/CC-MAIN-2020-16/CC-MAIN-2020-16_segments_1585370497171.9_warc_CC-MAIN-20200330150913-20200330180913-00087.warc.gz\"}"}