I have met quite many traders lately who had very interesting ideas regarding building of automated trading systems (ATS) for algorithmic trading. Unfortunately, I often feel that many traders rely solely on the basic backtest result, as in the example of a trading system and its equity curve in Fig. 1.
These days, there are many trading platforms, such as NinjaTrader, MetaTrader, AmiBroker, and TradeStation, that can perform this basic backtesting. Each of these programs has its own specific programming language by the use of which traders define entry and exit conditions of their trading systems. The reason why I chose TradeStation was the possibility to carry out advanced robustness testing. Such a possibility is not offered by any other commercial platform.
Fig. 1: Example of crossing of two moving averages
A backtest itself may be the first indication of the system´s quality, but by no means it is a sufficient analysis on the basis of which you should conclude that the system will be profitable also in real trading. You must perceive the basic backtesting as only one part of the entire system performance evaluation. For example, I regard backtesting only as a test that shows me the potential profitability of the trading system on the historical data for the selected market. The problem is that trading systems work for example with:
- open, high, low, close of a certain number of bars for the selected market and timeframe,
- numeric constants, and so on.
In the following example I will explain why the basic backtest never sufficiently evaluates the quality of a trading system:
Imagine that you have created a trading system the basic entry and exit conditions of which are crossings of two moving averages (MA). In Fig. 2 you can see two examples of such crossings:
Fig. 2: Example of crossing of two moving averages
The “Fast” moving average (FMA) calculates and draws into the chart the arithmetic average of the last 15 CLOSE values (blue line) and the “Slow” moving average (SMA) draws the average of the last 30 CLOSE values (purple line). On the left side you can see that if FMA crosses above SMA, the trading system enters into a long position (LONG) by the BUY MARKET order on the CLOSE price of the bar. In the opposite situation, if FMA crosses below SMA, the system closes the long position by the SELL MARKET order and at the same time opens a SHORT position on the CLOSE price of the bar. Thus we are actually constantly in a position because we enter into long or short positions at every crossing of FMA and SMA. Of course, I definitely do not recommend you to trade via such trading system. I mentioned it here only as an illustrative example.
As traders we must think critically and we should ask ourselves whether our automated trading system (ATS) is sufficiently profitable regardless of periods (e.g. 5, 10, 15, etc.) of the used moving averages. Here we use a simple premise that we want to trade via such a system the backtesting of which with various indicator settings showed a sufficient profitability. Since markets are unsteady time series and historical data is always unique, we must be sure that the ATS´s backtests show profitability with most parameter settings. We cannot afford to make such a decision without a well-founded expectation that the parameters for which we have built our trading system will be profitable in the future. But if the backtesting shows us that the ATS was profitable with various parameter settings, we get an indication that the trading system is robust, i.e. able to generate profits in live trading regardless of setting of its parameter values. Coming back to our example, the FMA 15 and SMA 30 setting seems to be profitable at first glance, thus we need to test whether the system would still be profitable with other parameter values (for example FMA 10 and SMA 40). We will test different combinations of the system´s parameter values on historical data. For example:
FMA: 5 10 15 20 25 30 35 40 45 50
SMA: 55 60 65 70 75 80 85 90 95 100
Table 1: Optimisation results of a trading system using crossing of two exponential moving averages
The example clearly shows that from the total of 100 FMA and SMA values there were only 20 profitable ones (see the black Net Profit column in Table 1). I. e. only 20% of the combinations were profitable, which is clearly not a sufficient value. The aim is that most combinations within the optimisation were profitable. The presented example is really very simplified and inapplicable in practice. Precise optimisation tests represent a very substantial know-how and to my surprise, many traders are not familiar with this important part of trading system evaluation at all, or they do not understand it properly. I explain the specific procedures and methods which I use for evaluation of optimisation tests my course Building Winning Automated Trading Strategies with TradeStation.
In this chapter, we explained that optimisation tests represent one of the basic tools for robustness testing beyond the basic backtest. High-quality optimisation tests can be performed in the TradeStation platform. If you would like to learn more about this platform, click here. For me it is absolutely the crucial software for my algorithmic trading.