Nonasymptotic Analysis of Semiparametric Regression Models with High-Dimensional Parametric Coefficients

We consider a two-step projection based Lasso procedure for estimating a partially linear regression model where the number of coefficients in the linear component can exceed the sample size and these coefficients belong to the l_{q}-“balls” for q in [0,1]. Our theoretical results regarding the properties of the estimators are nonasymptotic. In particular, we establish a new non-asymptotic “oracle” result: Although the prediction error of the nonparametric projection per se has the scaling t_{n} in the first step, it only contributes a scaling t_{n}^{2} in the l_{2}-error of the second-step estimator for the linear coefficients. This new “oracle” result holds for a large family of nonparametric least squares procedures and regularized nonparametric least squares procedures for the first-step estimation and the driver behind it lies in the projection strategy. We specialize our analysis to the estimation of a semiparametric sample selection model and provide a simple method with theoretical guarantees for choosing the regularization parameter in practice