Linking the statistic and the machine learning literature, we provide new general results on the convergence of stochastic approximation schemes and inexact Newton methods. Building on these results, we put forward a new optimization scheme that we call generalized inexact Newton method (GINM)....

Strong consistency of least squares estimators of the slope parameter in simple linear regression models is established for predetermined stochastic regressors. The main result covers a class of models which falls outside the applicability of what is presently available in the literature. An...