Running a research project in the open: a practical guide
Deciding to research a systematic strategy in public is the easy part. Making the project useful to others, and trustworthy, takes structure: clear repositories, licences that say what people may do, environments that can be rebuilt, a record of experiments, and rules about who decides what. This guide covers the practical mechanics, drawing on how well-run open-source and open-science projects work.
This is a guide to organising research. It is not legal advice, and an open research project is not an investment vehicle.
Start with a clear scope
Write a short statement of purpose before writing code. It should say what the project studies (for example, "momentum strategies on liquid U.S. equities and ETFs"), what it will publish (code, documentation, test results), and what it will not do (manage money, publish live signals, give advice). The statement protects the project from drift and makes it easier for contributors to judge whether their ideas fit. Our overview of open-source hedge funds discusses why that last boundary matters.
Repository structure
A predictable layout lets newcomers find their way. A common pattern for research code:
data/containing download scripts and data documentation, not the data itself if licences forbid redistribution;src/containing reusable functions for signals, labels, portfolio construction and performance measurement;tests/with unit tests for those functions;experiments/with one folder or notebook per study, each with a short description of the hypothesis and the result;docs/with explanations of methods and decisions;- a
README, aLICENSEfile, aCONTRIBUTINGguide and a lock file recording dependency versions.
Version control and review
Git, hosted on a public platform, is the backbone. Changes arrive as pull requests that someone other than the author reviews before merging. For research code, review should look beyond style to the questions that most often invalidate results:
- Does any calculation use information not available at the time (look-ahead bias)?
- Are prices adjusted for splits and dividends consistently?
- Are delisted securities included where they should be (survivorship bias)?
- Are transaction costs and slippage modelled, and are the assumptions stated?
- Is the test period separated from the period used for design choices?
Continuous integration that runs tests automatically on every change catches regressions, such as a refactoring that silently changes how returns are calculated.
Licences: code, data and text
Without a licence, published code is legally "all rights reserved", and others cannot safely reuse it. Choose one at the start:
- Permissive licences (MIT, BSD, Apache 2.0) allow almost any use, including commercial, with attribution. Apache 2.0 also includes an explicit patent grant.
- Copyleft licences (GPL, AGPL) require derivative works to be released under the same terms, which keeps improvements open but discourages some commercial users.
- Documentation and text are often licensed under Creative Commons, for example CC BY 4.0.
Data are the trickiest part. Most market data are licensed by exchanges and vendors with strict redistribution limits. An open project should publish scripts that download data from their source rather than the data themselves, unless the licence clearly allows sharing. Free sources such as the Kenneth French Data Library and FRED make reproducible public research possible, but each has its own terms of use.
Reproducible environments
A result that cannot be reproduced cannot be checked. Record exact package versions in a lock file, specify the Python or R version, and consider a container definition for complex setups. Fix random seeds where models use randomness, and document hardware-dependent behaviour. Store the commit identifier alongside every published result, so anyone can return to the exact code that produced it.
Record every experiment, including failures
The biggest threat to quantitative research is selection: trying many variations and remembering only the best. An open project can do better than most private research by keeping a public log of every experiment, with its hypothesis, parameters and outcome. Negative results are valuable: they save others from repeating the work, and they allow readers to judge the final result against the number of trials, for example with the deflated Sharpe ratio of Bailey and López de Prado.
Pre-registration, writing down the hypothesis and test design before running the test, is common in medicine and increasingly discussed in finance. Even an informal version, a dated issue describing the planned test, strengthens credibility.
Governance
Small projects can run on informal agreement, but conflicts eventually arise over direction, credit or standards. A short governance document should say who can merge changes, how disagreements are resolved, how contributors are credited (for example, through the version control history and a contributors file listing those who agree to be named), and what code of conduct applies. Credit should be recorded accurately; never list someone as a contributor without their consent.
Legal lines to respect
- No investment advice or solicitation. Publishing research is different from recommending trades or inviting investment, which are regulated activities in most countries.
- No performance marketing. Presenting hypothetical results as an offer to manage money is restricted by securities regulators; label backtests as simulations with stated assumptions.
- Respect data and software licences, including for any third-party libraries.
- Protect personal data of contributors, and do not publish information about individuals beyond what they choose to share.
Next step
With the structure in place, the research itself can begin. Creating a strategy step by step walks through the stages from hypothesis to paper trading.
Communicating results
Each completed experiment deserves a short written summary: the question, the data and period, the method, the result with uncertainty, and the limitations. Charts should show the full sample and the hold-out period distinctly. Avoid language that implies a recommendation, and state clearly that results are simulated where they are.
Sources
- Open Source Guides (GitHub), How to contribute, start and lead open-source projects.
- Choose a License, choosealicense.com.
- Creative Commons, Attribution 4.0 International.
- Wilson, G. et al. (2017). Good Enough Practices in Scientific Computing. PLOS Computational Biology, 13(6), e1005510.
- Bailey, D. and López de Prado, M. (2014). The Deflated Sharpe Ratio. Journal of Portfolio Management, 40(5), 94–107.