We introduce an approach to high-dimensional extremes. Based on concepts from high-dimensional statistics and modern convex programming, we can obtain fine-grained models within seconds on a standard laptop. We will illustrate these properties with finite-sample theories and empirical analyses.

The surge of massive image databases has spurred the development of scalable machine learning methods particularly convolutional neural network (CNNs), for filtering and processing such data. Current theoretical advancements in CNNs primarily focus on standard nonparametric denoising problems. However, in image classification datasets, the variability arises not from additive noise but from variations in object shape and other characteristics of the same object across different images. To address this problem, we consider a simple supervised classification problem for object detection in grayscale images. While from a function estimation point of view, every pixel is a variable and large images lead to high-dimensional function recovery tasks suffering from the curse of dimensionality, increasing the number of pixels in our image deformation model enhances image resolution and simplifies object classification problem easier. We introduce and theoretically analyze two procedures: one based on support alignment, demonstrating perfect classification under minimal separation conditions, and another fitting CNNs to the data showcasing a misclassification error depending on the sample size and number of pixels. Both methods are empirically validated using the MNIST handwritten digits database.

This is joint work with Johannes Schmidt-Hieber (Twente).

We investigate the statistical behavior of gradient descent iterates with dropout in the linear regression model. In particular, non-asymptotic bounds for expectations and covariance matrices of the iterates are derived. In contrast with the widely cited connection between dropout and ell_2-regularization in expectation, the results indicate a much more subtle relationship, owing to interactions between the gradient descent dynamics and the additional randomness induced by dropout.

For more details see [2306.10529] on arXiv.

We present a significance test for any given variable in nonparametric regression with many variables {via estimating derivatives of a nonparametric function}. The test is based on the moment generating function of the partial derivative of an estimator of the regression function, where the estimator is a deep neural network whose structure is allowed to become more complex as the sample size grows.
This test finds applications in model specification and variable screening for high-dimensional data. To render our test applicable to high-dimensional inputs, whose dimensions can also increase with sample size, we make the assumption that the observed high-dimensional predictors can effectively serve as proxies for certain latent, lower-dimensional predictors that are actually involved in the regression function. Additionally, we finely adjust the regression function estimator, enabling us to achieve the desired asymptotic normality under the null hypothesis, as well as consistency for any fixed scenarios and certain local alternatives.