Getting Started: Building a Fully Automated Trading System.

 

By Jacques Joubert

For the last 6 months I have been focused on the process of building the full technology stack of an automated trading system. I have come across many challenges and learnt a great deal about the two different methods of backtesting  (Vectorised and Event driven). In my journey to building an event driven backtester, it came to my surprise that what you would end up with is close to the full technology stack needed to build a strategy, backtest it, and run live execution.

My biggest problem when tackling the problem was a lack of knowledge. I looked in many places for an introduction to building the technology or a blog that would guide me. I did find a few resources that I am going to share with you today.

For Beginners:

For the readers new to quantitative trading I would recommend Ernie P. Chan’s book titled: Quantitative Trading: How to build your own algorithmic trading business. This book is the basics. It’s actually the first book I read on quantitative trading and even then I found it very basic but there are some notes you should take.

From page 81-84 Ernie writes about how at the retail level a system architecture can be split up into semi-automated and fully automated strategies.

A semi-automated system is suitable if you want to place a few trades a week. Ernie recommends using Matlab, R, or even Excel. I have used all 3 platforms and this is my advice:

  1. Skip Matlab, it cost a lot of money and I could only get access to it at the university laboratories. There isn’t a lot of training material like blogs or books that will teach you how to code a strategy using Matlab.
  2. R has tons of resources that you can make use of in order to learn how to build a strategy. My favorite blog covering the topic is: QuantStratTradeR  run by Ilya Kipnis.
  3. Microsoft Excel is most likely where you will start if you don’t have programming experience. You can use Excel for semi-automated trading but it’s not going to do the trick when it comes to building the full technology stack.

EP Chan SemiAutoSemi-automatic framework pg 81

Completely automated trading systems are for when you want to automatically place trades based on a live data feed. I coded mine in C#, QuantConnect also uses C#, QuantStart walks the reader through building it in Python, Quantopian uses Python,  HFT will most likely use C++. Java is also popular.

EP Chan FullyAutoCompletely automated trading framework pg 84

Step 1: Getting a head start

Do the Executive Program in Algorithmic Trading offered by QuantInsti.com. I just started the course and the very first set of lectures was on system architecture. It would have saved me about 3 months of research if I had started here. The lectures walked me through each component that I would need as well as detailed description of what each component needs to do. Below is a screen shot of one of their slides used in the presentation:

Untitled

You can also use this general framework when evaluating other automatic trading systems.

At the time of writing I am only in the third week of lectures but I am confident that a practitioner will be able to build a fully automated trading strategy that could, with a bit of polish, be turned into the beginnings of a quantitative hedge fund.

Note: the course is not focused on building the technology stack.

Step 2: Code a basic event driven backtester

Michael Hallsmore’s blog quantstart.com & book “Successful Algorithmic Trading”

This book has sections dedicated to building a robust event driven backtester. He walks the reader through a number of chapters that will explain his choice of language, the different types of backtesting, the importance of event driven backtesting, and how to code the backtester.

Michael introduces the reader to the different classes needed in an object orientated design. He also teaches the reader to building a securities master database. It is here that you will see how the system architecture from QuantInsti fits in.

Note: You will need to purchase his book: “Successful Algorithmic Trading”, his blog leaves out too much information.

sat-cover

Step 3: Turn to TuringFinance.com

After completing:

  1. The EPAT program
  2. Reading “Successful Algorithmic Trading” & coding a backtester in a different language of your choice.

You should move onto a blog called TuringFinance.com and read the article titled “Algorithmic Trading System Architecture” By: Stuart Gordon Reid.   In his post he describes the architecture following the guidelines of the ISO/IEC/IEEE 42010 systems and software engineering architecture description standard.

I found this post very technical and it has some great ideas that you should incorporate into your own architecture.

turingFinance

A screen shot from his post

Step 4: Study open source trading systems.

4.1) Quantopian

It goes without saying that Quantopian should be added to this list and I am embarrassed to say that I haven’t spent a lot of time using their platform (due to my choice of language). Quantopian has many perks but the ones that stick out most to me are the following:

  1. Easy to learn Python
  2. Free access to many datasets
  3. A huge community and competitions
  4. I love how they host QuantCon!

Quantopian is the market leaders in this field and is loved by quants all over!  Their open source project is under the code name Zipline and this is a little bit about it:

“Zipline is our open-sourced engine that powers the backtester in the IDE. You can see the code repository in Github and contribute pull requests to the project. There is a Google group available for seeking help and facilitating discussions.”

Here is a link to their documentation:

4.2) QuantConnect

For those of you unfamiliar with QuantConnect, they provide a full open source algorithmic trading engine. Here is a link 

You should have a look at their code, study it, & give them praise. They are Quantopians competition.

I would like to take this opportunity to thank the QuantConnect team for letting me pick their brain and for the brilliant service they provide.

Here is a link to their documentation:

Concluding remarks:

I hope this guide helps the members of the community. I wish I had this insight 6 months ago when I started coding our system.

Questions to the Readers:

  1. I would like to reach out to the community and ask: “What good algorithmic trading courses do you know of?” I would like to write a post that looks into the topic and provides a ranking.
  2. Are there any recommendations to building a fully automated trading system that you would like to add to this post?

Kind regards
Jacques Joubert

6 replies
  1. Nicholas Stein
    Nicholas Stein says:

    Nice article. I wish I had it about 6 months ago. I use QuantConnect because I am a C# programmer. I found it very convenient to be able to download Lean and back test locally. Rummaging through their code is also valuable. Also they cut a deal with Tradier for $1 trades. That helps a lot. I am not as salient about Tradier spreads and execution. IB might be better for that.

    I will take a look at the course you mentioned.

    You did not mention Quantocracy or RBloggers. Both are very valuable resources.

    What do you use for charting results of back tests? I log OHLC and indicator values to csv from the OnData event and am really tired of using Excel to chart results. I would like to be able to point a charting package to a data file and have it just go.

    Do you have a tick stream vendor yet?

    I have one thought about event driven systems. The problem with events is that they are async and latent. It seems they are unavoidable as soon as you get a brokerage involved, So I have been dreaming of a more streaming system following the principles of functional programming.
    – Injest a tick or bar stream.
    – Run it through a process of calculating indicators, running analytics or ML, and so forth.
    – Get back a signal.
    – Make an order.
    – Send it off to the broker to execute.
    Then in a separate stream.
    – Get back a response from the broker.
    – React to it.
    The problem of course is state. Do I have enough margin to make the trade? What is in my portfolio? How is it performing? Usually the broker api can be queried to find out that stuff, but it takes time and is async. I am also looking at Rx extensions. That way the system can react to changes in the system through the observable pattern.

    Events are great for mouse clicks. Not so great for high volume transactional processing.

    Off soapbox.

    Nick

    Reply
    • Robert Carver
      Robert Carver says:

      Nick
      This is exactly the approach I took with my own stuff. Essentially I have a ‘normal’ program which wraps around a small part that is event driven to speak to the broker (IB API). Now for the problem of state. You have two choices; get state from the broker, or store it internally updating it when you get a fill back. This means there are times when you won’t know your state, or when the two sources of state are potentially in conflict (bad data, or a lag). Part of this depends on how fast you trade. Unless you are trading really quickly then pausing if you have a state conflict, or you are uncertain of state, is better than proceeding without knowing your state. I use a database ‘lock’ paradigm to deal with this.

      For a few more thoughts along these lines, see http://qoppac.blogspot.co.uk/2015/07/systems-building-execution.html

      Reply
    • Schalk Dormehl
      Schalk Dormehl says:

      Nicholas,

      Regarding almost everything you asked, you are close to the answer in Reactive Extension (Rx).

      With Rx going from Ticks to Candles is trivial.
      Going from Candles to Indicators is trivial.
      Composing Indicators from other Indicators is trivial.
      Composing Positions from Indicators is trivial.
      Composing Portfolios (as held over time) from Positions is trivial.
      Simulating the Risk Model is trivial.
      Back testing or trading live is simply deciding between a live stream of data or a simulated replay of database data.
      Executing is trivial.
      Implementation is possible in everything from C# to F# to JavaScript to C++ in almost identical code.
      Optimization is made fast because purely functional Rx is massively parallalizable to the GPU.
      Admittedly, optimization and feeding the effect of continuous optimization back into the back-test is non-trivial, but given that it is non-trivial anyway, I’m going to let that one slide 😉

      Purely Functional (or close to it) Rx is in my opinion the only way to tackle the infrastructure of this problem.

      Reply
  2. Larry Klein
    Larry Klein says:

    I know the system I want to trade. I dont want to program or learn something someone else already knows. So who can I hire to take the system I want to use and automate it. By automate it, I mean I don’t want to look at it. I will glance at the results once/week and the trades will be executed without my attention. Seems odd to me that in 2016, so much effort needs to go into taking a set of rules and having those rules execute at my broker.

    Reply
    • Jacques Joubert
      Jacques Joubert says:

      Hi Larry,

      I would suggest signing up with Quantopian and then finding someone inside the community there to build the strategy for you. They will be able to build it for you inside the IB brokers platform and be fully automated.

      Let me say though that I think you should monitor it closely, and not just “forget about it to”.

      Reply

Leave a Reply

Want to join the discussion?
Feel free to contribute!

Leave a Reply

Your email address will not be published. Required fields are marked *