Financial machine learning

Fractional differentiation: stationary features that keep their memory

Financial machine learning · Published 2026-10-29

Statistical models generally need stationary inputs: series whose mean, variance and autocorrelation do not drift over time. Prices are not stationary, so researchers usually work with returns. But differencing a price series to get returns removes almost all of its memory, the long-range dependence that might carry predictive information. Fractional differentiation offers a compromise between the two extremes.

The dilemma: prices or returns

A log price series is typically integrated of order one: it wanders, and its level depends on its whole history. Taking the first difference, today's log price minus yesterday's, produces log returns, which are close to stationary. In the language of time series, we have differenced with order d = 1.

The cost is memory. Returns tell you how much the price moved today but nothing about where the price stands relative to its history. Many intuitive trading ideas, such as distance from a long-run level or the extent of a trend, depend on that information. Using prices keeps it but makes models unstable, because relationships estimated in one price range break down in another. The choice between d = 0 (prices) and d = 1 (returns) is a choice between memory and stationarity.

Differencing by a fraction

The idea of non-integer differencing comes from time series econometrics. Clive Granger and Roselyne Joyeux (1980) and Jonathan Hosking (1981) introduced fractionally integrated models to describe long-memory processes, in which correlations decay slowly. The key tool is the backshift operator B, defined by B X_t = X_t−1. Ordinary first differencing is (1 − B) X_t. For any real number d, the expression (1 − B)^d can be expanded as a binomial series:

The fractionally differenced series is therefore a weighted sum of current and past values: X̃_t = Σ ω_k X_t−k. For d = 1 the weights are 1, −1, 0, 0, …, which gives ordinary returns. For d between 0 and 1, the weights start at 1, the second weight is −d, and the remaining weights are small, negative and decay slowly toward zero. Older observations still contribute a little, which is how memory is preserved.

A worked look at the weights

With d = 0.4, the recursion gives ω₀ = 1, ω₁ = −0.4, ω₂ = −0.12, ω₃ = −0.064, ω₄ = −0.0416, and so on, each smaller in magnitude than the last but never exactly zero. The transformed series is mostly today's price minus 0.4 times yesterday's, with gradually fading adjustments for the more distant past. This arithmetic follows directly from the formula above and is easy to verify in a spreadsheet.

Applying it in practice: the fixed-width window

Because the weights never reach zero, the full expansion uses the entire history. Marcos López de Prado (2018, chapter 5) describes two practical approaches:

• Expanding window: use all available past observations, so early values of the series are computed with fewer weights than later ones. This introduces a drift in the series' properties.
• Fixed-width window: drop weights once their absolute value falls below a small threshold, such as 10⁻⁵, and apply the same truncated set of weights at every date. Every value is computed the same way, which keeps the series consistent.

The fixed-width window is generally preferred for features used in machine learning.

Choosing d

The goal is the smallest d that makes the series stationary. A standard procedure:

1. Take a log price series (or a cumulative series such as cumulative dollar volume).
2. For a grid of values, say d from 0 to 1 in steps of 0.05, compute the fractionally differenced series with a fixed window.
3. Run an augmented Dickey-Fuller (ADF) test on each, recording the test statistic.
4. Pick the lowest d whose ADF statistic is below the critical value at the chosen confidence level, typically 95 per cent.
5. Record the correlation between the transformed series and the original prices, as a measure of memory retained.

In his book, López de Prado reports that for many liquid futures series the minimum d passing the test is well below 1, often below 0.6, while the correlation with the original series remains high. This means a large part of the memory discarded by ordinary returns was not needed to achieve stationarity. The exact value of d depends on the instrument, the sample and the bar type, so it should be re-estimated rather than copied.

Pitfalls

• Look-ahead in choosing d. If d is chosen using the whole sample, information from the future leaks into the features. Estimate it on training data only.
• Stationarity is not predictability. A stationary feature with memory is a better candidate input; it is not evidence of an edge.
• Structural breaks. ADF tests have low power around regime changes, and the appropriate d can change over time.
• Interpretation. Fractionally differenced values have no natural unit, so features built on them are harder to explain.
• Window length. A very small threshold produces long windows that consume much of the sample before the first valid value.

Implementation notes

The transformation is a convolution of the series with a fixed weight vector, which makes it easy to implement with NumPy or pandas. The ADF test is available in statsmodels as adfuller. Open-source implementations of the fixed-width method exist in several Python packages; whichever is used, it is worth testing that d = 1 reproduces simple differences and d = 0 returns the original series, a quick check that the weights are correct.

Fractional differentiation is one step of the research pipeline described in our laboratory for machine learning in finance.