Linking the statistic and the machine learning literature, we provide new general results on the convergence of stochastic approximation schemes and inexact Newton methods. Building on these results, we put forward a new optimization scheme that we call generalized inexact Newton method (GINM)....