arXiv:2611.21877v1 Announce Type: new Abstract: For smooth penalties and group Lasso, analyzed approximate leave-one-out (ALO) cross-validation by comparing its weights with the trace correction used in mean-field inference. For a $C^2$ penalty $R$, the matrix entering these corrections is $A=[\nabla^2R(\widehat b)+X^TDX]^{-1}$, where $\widehat b$ is the full-data estimator and $D$ is the diagonal matrix of fitted loss curvatures; for group Lasso, the penalty and loss curvatures are first restricted to the active support before taking the pseudoinverse. The obstacle to extending this argument to general convex penalties was the lack of an explicit formula for the matrix $A$ entering these corrections. For every finite-valued strongly convex penalty, we construct such a matrix from ordinary derivatives of the penalty response at the observed subgradient. Its projection--pseudoinverse formula separates the response space and penalty curvature, fixed by conditioning, from the explicit design dependence. This supplies the missing ingredient to extend the probabilistic argument of Part~I. The construction recovers the inverse Hessian for smooth penalties and the active-group inverse for group Lasso, including inactive boundary groups. In the proportional regime, for Gaussian designs with general positive-definite covariance and covariance-scaled strong convexity, we obtain unconditional bounds comparing both ALO and mean-field predictions with exact leave-one-out predictions in regression and single-index models. The expected average squared prediction error is $O(n^{-1})$, and the expected absolute criterion error is $O(n^{-1/2})$, under bounded, 1-Lipschitz loss scores and suitable test functions. The matrix and ALO weights use only observed data; the mean-field correction additionally uses the design covariance. The construction, recovery identities and statistical guarantees are formalized in Lean~4.
Continue to arXiv Mathematics →You are being forwarded in 6s — stay on this page