We develop inferential tools for latent factor analysis in short panels. The pseudo maximum likelihood setting under a large cross-sectional dimension n and a fixed time series dimension T relies on a diagonal T × T covariance matrix of the errors without imposing sphericity nor Gaussianity. We outline the asymptotic distributions of the latent factor and error covariance estimates as well as of an asymptotically uniformly most powerful invariant (AUMPI) test for the number of factors based on the likelihood ratio statistic. We derive the AUMPI characterization from inequalities ensuring the monotone likelihood ratio property for positive definite quadratic forms in normal variables. An empirical application to a large panel of monthly U.S. stock returns separates month after month systematic and idiosyncratic risks in short subperiods of bear vs. bull market based on the selected number of factors. We observe an uptrend in the paths of total and idiosyncratic volatilities while the systematic risk explains a large part of the cross-sectional total variance in bear markets but is not driven by a single factor. Rank tests show that observed factors struggle spanning latent factors with a discrepancy between the dimensions of the two factor spaces decreasing over time.