A laboratory for machine learning in finance: bars, labels and honest validation

Applying machine learning to markets is not mainly a question of choosing a clever algorithm. Most failures happen earlier, in how the data are sampled, how the target is labelled and how the model is validated. This article walks through a research "laboratory" for financial machine learning, largely following the framework set out by Marcos López de Prado in Advances in Financial Machine Learning (2018), and explains why each step exists.
Educational content about research methods. It describes no live strategy and makes no performance claim.
Why market data are a hard case
Standard machine-learning practice assumes observations that are independent and identically distributed, a target that is cleanly defined, and a test set that resembles the training set. Financial time series violate all three. Returns are noisy, with a low signal-to-noise ratio; their distribution changes over time; observations overlap and are correlated; and the strongest-looking patterns are often artefacts of the testing procedure itself. Each stage of the laboratory addresses one of these issues.
Step 1: sample by information, not by the clock
Most price data are sampled at fixed time intervals: one bar per minute, hour or day. But markets do not process information at a constant rate. Activity clusters around the open, the close and news events. Time bars therefore oversample quiet periods and undersample busy ones, and their returns show strong volatility clustering and fat tails.
Alternatives sample a new bar each time a fixed amount of market activity has occurred:
- Tick bars: a new bar every N transactions.
- Volume bars: a new bar every N shares or contracts traded.
- Dollar bars: a new bar every N units of currency traded, which also adjusts for price changes and corporate actions such as splits.
The idea draws on older research into "business time" and the volume clock, including Easley, López de Prado and O'Hara (2012). Returns sampled this way are typically closer to normal and less autocorrelated in their volatility than time-bar returns, which makes them better behaved inputs for statistical models. López de Prado also describes "imbalance" and "run" bars, which sample when buying or selling pressure departs from expectations.
Step 2: decide which bars are events
Not every bar is a decision point. A filter such as the symmetric CUSUM filter flags bars where cumulative price change since the last event exceeds a threshold. Only these events are labelled and used for training. This focuses the model on moments where something has happened and reduces the number of redundant, highly overlapping samples.
Step 3: label with the triple-barrier method
The naive label is the sign of the return over a fixed horizon. It ignores the path: a trade that would have hit a stop-loss along the way is labelled a winner if the price recovered by the horizon. The triple-barrier method sets three barriers at each event:
- an upper barrier (profit-taking), at a multiple of recent volatility above the entry price;
- a lower barrier (stop-loss), at a multiple of volatility below;
- a vertical barrier, a maximum holding period.
The label is determined by which barrier is touched first: +1 for the upper, −1 for the lower, and either 0 or the sign of the return for the vertical. Scaling barriers by volatility means labels adapt to the market regime. The method produces labels that reflect how a position would actually have been managed.
When the side of the trade is already given by another model, the triple-barrier labels can be turned into a binary "act or don't act" target. This is the basis of meta-labelling.
Step 4: weight overlapping samples
Triple-barrier labels span several bars, so labels from nearby events overlap in time and share the same price moves. Treating them as independent overstates the information in the data. López de Prado proposes measuring each label's uniqueness, the average share of its lifespan it does not share with other labels, and using it to weight samples or to bootstrap them sequentially so that more unique samples are drawn more often.
Step 5: make features stationary without erasing memory
Prices are non-stationary; returns are stationary but have lost most of the memory that might carry predictive information. Fractional differentiation offers a middle path: differencing by a fractional order just large enough to pass a stationarity test, preserving as much memory as possible.
Step 6: validate with purging and embargo
Ordinary k-fold cross-validation shuffles data into folds. With overlapping labels, a training sample can contain information that also determines a test label, which leaks the answer. Purged k-fold cross-validation removes from the training set any observation whose label period overlaps the test period. An embargo also removes a short window of training data immediately after each test fold, to guard against serial correlation. The result is lower but more honest scores.
For backtesting, López de Prado further proposes combinatorial purged cross-validation, which generates many train/test splits and therefore a distribution of backtest paths instead of a single historical path.
Step 7: ask which features matter
Feature importance is how the laboratory learns rather than merely predicts. Three methods are common:
- Mean decrease impurity (MDI): for tree ensembles, how much each feature reduces impurity across splits. Fast, in-sample, and biased toward features with many possible splits.
- Mean decrease accuracy (MDA): permutation importance, the loss in out-of-sample score when a feature's values are shuffled. Slower, out-of-sample, but affected when features are correlated.
- Single feature importance (SFI): the out-of-sample score of a model trained on each feature alone, which avoids substitution effects but ignores interactions.
Using several methods together and clustering correlated features before testing them gives a more reliable picture. Features that matter consistently are the ones worth economic interpretation.
Running the laboratory
These steps are building blocks, not a guaranteed path to profit. Each adds parameters, such as bar size, barrier widths and embargo length, that should be fixed in advance or validated carefully. The most useful habit is to keep a research log of every variation tried, so that the final result can be judged against the number of attempts. Open implementations of many of these methods exist; our article on the book's techniques and the open-source tools around them maps the landscape.
Sources
- López de Prado, M. (2018). Advances in Financial Machine Learning. Wiley. Chapters 2–8.
- Easley, D., López de Prado, M. and O'Hara, M. (2012). The Volume Clock: Insights into the High-Frequency Paradigm. Journal of Portfolio Management, 39(1), 19–29.
- Clark, P. (1973). A Subordinated Stochastic Process Model with Finite Variance for Speculative Prices. Econometrica, 41(1), 135–155.
- scikit-learn documentation, Permutation feature importance.
Momentum & factorsMomentum strategies
How momentum portfolios are built, what three decades of research found, and what erodes the paper returns.
Systematic tradingBuilding a fully automated trading system
Data, signals, risk, orders and monitoring: the parts of an automated trading system and how they fail.
Financial machine learningDreaming, thinking machines
Automata, Descartes, Turing, ELIZA and the Chinese room: what the history of machine intelligence says about markets.