When there are more moment conditions than observations, the usual GMM weighting matrix is singular. We show that using the generalized inverse is not a good idea. With continuous updating, the criterion function equals one for every parameter value.