Nowcasting when predictors keep changing

Who am I?

Economist working at the intersection of applied macroeconomics, data science, and research engineering
Lead of Data and Economic Modeling at Kapitalo, a Brazilian multi-manager hedge fund managing over USD 5 billion in assets
I develop quantitative models, data products, and data pipelines for investment research
Much of my recent work has focused on inflation nowcasting using alternative data

Why Inflation Nowcasting?

Predicting inflation in Brazil is a highly competitive task

Inflation expectations are central to asset allocation in Brazil
A deep and liquid market for inflation-linked bonds creates strong incentives to improve predictive accuracy
Professional forecasters rely on a wide range of high-frequency information
- These include commodity prices, exchange rates, wholesale prices, among many others
- One of the most valuable information sources is the FGV Monitor, a proprietary daily indicator based on brick-and-mortar price collection
- By closely following the methodology of Brazil’s official statistical agency, it provides an early signal of short-term inflation dynamics

The FGV Monitor Leaves Room for Improvement

It is hard to beat for some components, but less informative for others

Where Can Additional Information Come From?

Building an additional high-frequency signal from web-scraped prices

Rather than replacing the FGV Monitor, we seek an additional source of high-frequency information based on web-scraped prices
The greatest potential lies in components where the FGV Monitor is less informative
Recent advances in computational tools have made collecting web prices substantially more accessible
Even when starting from similar raw data, the resulting indicators can differ because of coverage, preprocessing, classification, and modeling choices

The challenge is no longer access to data, but transforming it into useful signals.

The First Challenge

Which products should we use?

We observe hundreds, sometimes thousands, of products with daily price updates
The available product universe is far larger than the official CPI basket
The official CPI basket is expected to be embedded within this larger universe

This looks like a standard variable selection problem, right?

Not Quite a Selection Problem

The predictor set evolves continuously

Products continuously enter and leave the basket
- Electronics have short lifecycles and are constantly replaced
- Vehicles are effectively a new basket every year
- Even “stable” categories exhibit gradual turnover

Product turnover makes the set of available products generally unstable over time

What Does Turnover Look Like?

Product availability is highly dynamic over time

Why Do Standard Approaches Struggle?

High turnover creates methodological and operational challenges

Most methods are designed to learn stable relationships from a fixed set of predictors
Many products have short lifecycles, leaving little time to estimate, validate, and re-optimize models before the underlying cross-section changes again
In production environments, daily pipelines benefit from methods that remain stable over time rather than requiring frequent re-specification

The challenge is not only statistical, but also operational

Reframing the Task

From selecting products to aggregating information

Variable selection relies on learning stable relationships over time
Most products in our data have short and irregular histories
Time dimension is scarce, but the cross-section is rich

Instead of selecting predictors, we exploit the information contained in the cross-section

A Simple Benchmark

Despite its simplicity, the mean captures a useful amount of information

A Structured Approach

From a Mean to a Factor Model

The mean already captures a useful amount of information
Rather than replacing it, we can view it as a restricted factor model
This perspective suggests a natural path for introducing additional structure

A factor model allows us to extend the mean while preserving its intuition

The Mean as the Simplest Factor Model

A single latent factor under strong restrictions

Unit loadings, no factor dynamics, and homoskedastic Gaussian errors

\[ \require{cancel} \begin{aligned} y_{i,t} &= f_t + \varepsilon_{i,t} \\ f_t &= \eta_t \\ \varepsilon_{i,t} &\sim {\mathcal{N}}(0, {\sigma_\varepsilon}) \\ \eta_t &\sim \mathcal{N}(0, \sigma_\eta) \end{aligned} \]

The Mean as the Simplest Factor Model

A single latent factor under strong restrictions

Unit loadings, no factor dynamics, and homoskedastic Gaussian errors

\[ \require{cancel} \begin{aligned} y_{i,t} &= f_t + \varepsilon_{i,t} \\ f_t &= \eta_t \\ \varepsilon_{i,t} &\sim {\mathcal{N}}(0, {\sigma_\varepsilon}) \\ \eta_t &\sim \mathcal{N}(0, \sigma_\eta) \end{aligned} \]

The Mean as the Simplest Factor Model

A single latent factor under strong restrictions

Unit loadings, no factor dynamics, and homoskedastic Gaussian errors

\[ \require{cancel} \begin{aligned} y_{i,t} &= f_t + \varepsilon_{i,t} \\ f_t &= \eta_t \\ \varepsilon_{i,t} &\sim {\mathcal{N}}(0, {\sigma_\varepsilon}) \\ \eta_t &\sim \mathcal{N}(0, \sigma_\eta) \end{aligned} \]

This naturally leads to potential extensions

Extension 1: Factor Dynamics

Allow the latent signal to evolve over time

\[ \require{cancel} \begin{aligned} y_{i,t} &= f_t + \varepsilon_{i,t} \\ f_t &= \color{#D97706}{f_{t-1}} + \eta_t \\ \varepsilon_{i,t} &\sim \mathcal{N}(0, {\sigma_\varepsilon}) \\ \eta_t &\sim \mathcal{N}(0, \sigma_\eta) \end{aligned} \]

Extension 1: Factor Dynamics

Allow the latent signal to evolve over time

Extension 2: Time-Varying Volatility

Measurement uncertainty changes with cross-sectional dispersion

\[ \require{cancel} \begin{aligned} y_{i,t} &= f_t + \varepsilon_{i,t} \\ f_t &= \eta_t \\ \varepsilon_{i,t} &\sim \mathcal{N}(0, \color{#D97706}{\sigma_{\varepsilon,t}}) \\ \color{#D97706}{\log(\sigma_{\varepsilon,t})} &\color{#D97706}{=} \color{#D97706}{\alpha + \beta \times IQR_t} \\ \eta_t &\sim \mathcal{N}(0, \sigma_\eta) \end{aligned} \]

Extension 2: Time-Varying Volatility

Measurement uncertainty changes with cross-sectional dispersion

Extension 3: Student-t Errors

Robustness to extreme product-level price changes

\[ \require{cancel} \begin{aligned} y_{i,t} &= f_t + \varepsilon_{i,t} \\ f_t &= \eta_t \\ \varepsilon_{i,t} &\sim \color{#D97706}{t_\nu(0, \sigma_\varepsilon)} \\ \eta_t &\sim \mathcal{N}(0, \sigma_\eta) \end{aligned} \]

Extension 3: Student-t Errors

Robustness to extreme product-level price changes

Do Extensions Improve Forecasting Performance?

Mean Absolute Error - Model with Student-t errors vs. sample mean

Is there information beyond the FGV Monitor?

Yes, there are periods when alternative data provides information beyond the FGV Monitor.

What about combining them into a single indicator?

Simple forecast combinations often outperform their individual components

Business Impact

Our forecasts have consistently ranked among the Brazilian Central Bank’s Top 5

Takeaways

What we learned from high-turnover datasets

In high-turnover datasets, variable selection is often not a viable strategy
When predictors change faster than relationships can be learned, aggregation may be more effective than selection
The cross-section contains valuable information beyond simple averaging
Even simple extensions can deliver meaningful forecasting gains
Alternative data are not silver bullets, but they can provide complementary information beyond traditional indicators

THANK YOU!

Contact

📧 joao@rleripio.com
🌐 rleripio.com

Slides