We analyze the fluctuation of the loss from default around its large portfolio limit in a class of reduced-form models of correlated firm-by-firm default timing. We prove a weak convergence result for the fluctuation process and use it for developing a conditionally Gaussian approximation to the loss distribution. Numerical results illustrate the accuracy and computational efficiency of the approximation.