We clarify the rationale and differences between the two main categories of linear factor models, namely dominant-residual and systematic-idiosyncratic. We discuss the five different, yet interconnected areas of quantitative finance where linear factor models play an essential role: multivariate estimation theory, asset pricing theory, systematic strategies, portfolio optimization, and risk attribution. We present a comprehensive list of common pitfalls and misunderstandings on linear factor models. An appendix details all the calculations. Supporting code is available for download.