In today’s data-rich financial markets, trading strategies increasingly rely on analyzing large sets of correlated indicators. With this complexity, identifying the key drivers—or “factors”—behind asset returns has become both more difficult and more essential. Enter latent factor models: techniques designed to distill high-dimensional data into a few informative signals. But how do we find these hidden factors effectively?
Traditionally, analysts have taken one of two routes: select a few known variables as factor proxies based on theory or use statistical techniques like principal component analysis (PCA) to uncover factors from the data itself. Each approach has its strengths—but also clear limitations. A recent methodological advance aims to bridge this divide by combining domain expertise with data-driven techniques to create more accurate and robust factor models.
Rethinking Factor Discovery: A New Hybrid Approach
Factor models are central to analyzing how multiple variables are linked by a smaller set of common drivers. In this context, “latent factors” refer to unobserved variables that explain much of the behavior of observed time series—like asset returns across sectors or countries. However, estimating these latent factors is not straightforward.
The first method, often used in finance, relies on economic intuition to choose factor proxies—observed variables believed to represent broader economic themes. For example, the Fama-French three-factor model is built on such pre-selected proxies like market risk, size, and value. The downside? It risks overlooking important influences not captured by the chosen proxies.
The second method is purely statistical. Techniques like PCA estimate latent factors by identifying patterns in the data itself. While elegant, this approach ignores any prior knowledge and may include irrelevant or noisy influences that weaken the model.
A new solution, proposed by researchers Runzhe Wan, Yingying Li, Wenbin Lu, and Rui Song, combines both ideas. They introduce a technique called Factor Model estimation with Sufficient Proxies (FMSP). This method uses a large set of candidate proxies and applies penalized reduced rank regression (RRR) to uncover the most meaningful factors.
What Makes Reduced Rank Regression Effective?
Reduced rank regression is a multivariate modeling technique well-suited to high-dimensional data. It seeks to identify a lower-dimensional structure within a large number of predictors by finding linear combinations that best explain the response variables. Instead of estimating coefficients for each individual variable, RRR focuses on a smaller number of composite variables, thereby reducing complexity and multicollinearity.
The innovation in FMSP lies in how it handles real-world data characteristics. Markets often produce heavy-tailed distributions and noisy inputs. The researchers enhance standard RRR by adding penalization to improve stability and robustness, especially when dealing with many potential proxies. The model then estimates the latent factor structure by projecting observed outcomes onto the compressed space derived from these proxies.
This hybrid approach also adapts as more data becomes available. It scales with the number of proxies, allowing for richer factor modeling as the financial “factor zoo” continues to grow.
From Theory to Practice: Application in Equity Markets
To test the method’s usefulness, the researchers applied it to U.S. equity returns using a dataset of 99 known factor proxies spanning from 1980 to 2016. These included well-established factors such as those from the Fama-French and q-factor models, as well as additional long-short portfolios based on firm characteristics like value, profitability, and trading frictions.
Their approach was benchmarked against traditional methods using a rolling-window analysis to assess how accurately the factor structures could be estimated over time. The results were compelling: FMSP consistently outperformed existing models in terms of estimation accuracy. The improvements were not only statistically significant but also robust across market conditions and data subsamples.
This is particularly noteworthy in the context of the “factor zoo” problem in finance—where hundreds of proposed factors complicate model selection and performance attribution. FMSP avoids the trap of selecting a small subset of proxies upfront and instead leverages the full breadth of information in a disciplined way.
Why This Matters for Quantitative Trading
As datasets expand in size and complexity, the ability to extract reliable signals becomes a key differentiator in systematic investing. The FMSP framework offers a powerful tool for improving factor model accuracy, particularly when working with overlapping or correlated inputs.
Its strength lies in its flexibility. By balancing expert-driven intuition with machine learning techniques, it enables practitioners to capture a more complete picture of market dynamics without falling into the pitfalls of overfitting or oversimplification.
Ultimately, FMSP provides a pathway for refining trading strategies by uncovering the latent forces that drive asset prices—forces that are often hidden beneath the surface but critical for making informed investment decisions.
