Backtesting (or backtest) is the process of testing a trading system´s efficiency (profitability) on historical data of a particular market or a group of markets. Backtests are used before the trading system´s application in live trading. Obtaining of usable results by testing of trading systems and sets of trading rules on the current market data would require too much time, therefore we use the method of backtesting instead. Backtesting enables application of our system´s trading rules on historical market data. The objective of backtesting is to find out whether our strategy has the potential to generate profits in live trading. Traders should take backtesting really seriously. Each trading system that is intended to be employed in live trading should always and in all circumstances be subjected to quality backtesting.
There are the following two basic methods of backtesting:
a) Manual backtesting which is used within discretionary trading. A trader manually applies the predefined entry and exit conditions on historical data. This means that he manually goes through historical charts and searches for potentially profitable trades that comply with the trading system´s parameters. Then records these trades and their results into a predefined table. The objective of manual backtesting is to evaluate whether the trading system had good enough performance characteristics in the past (such as the net profit and minimal loss) and whether it has a clear potential to be profitable in the future within the live trading with real money. The advantage of manual backtesting is that you really “get acquainted” with the market and learn its inherent laws. Manual backtesting is suitable for traders who like having everything under control. A disadvantage of manual backtesting, on the other hand, is its´ time-consumingness and innate tendencies of each trader to “improve” their trade results. As they say, it´s easy to be wise after the event – everybody can see winning trades on historical data. It is therefore necessary to worsen the results of manual backtests at least by 30%.
b) Automated backtesting which is used by traders employing Automated trading systems (ATS). The trader defines the trading system´s entry and exit conditions in the form of an algorithmic code and then applies the code to selected historical data of a particular market or group of markets. The advantage of automated backtesting is that unlike in the manual backtesting you immediately get a summary of the system´s performance characteristics. A disadvantage, however, is that you have to create a functional program code of input conditions.
Automated backtesting falls into more advanced areas of trading. In this chapter we will first explain the principles of manual backtesting.
Let’s show step by step how to perform a manual backtest:
1. We define the market (or markets) on which we will perform backtesting.
For example, we are interested in intraday trading of stock index futures, thus we choose E-mini S&P 500 for backtesting.
2. We choose the timeframe and trading sessions – i.e. the market´s timeframe and from when to when we intend to trade every day.
Suppose we decided to trade the E-mini S&P 500 market on the 15-minute timeframe during the main trading session. It means from 8:30 a.m. to 4:15 p.m. Chicago time. In the Czech Republic it is from 15:30 to 22:15. The reason for choosing the 15-minute timeframe may be, for example, that we want to take advantage of highly profitable opportunities that we found on higher timeframes than, for example, the 5-minute chart. You can see an example of the 15-minute E-mini S&P 500 chart in Fig. 1.
Fig. 1: 15-minute chart of E-mini S&P 500 (Created in TradeStation)
3. We choose the length of the historical time period to be backtested. In other words, how many days, months, or years back we want to test our trading system.
There is one general rule that we should take into account – the longer the backtested period, the better. It is because the backtesting will include many different situations. We can thus verify if the system works well in for example periods of steep growths or declines in market prices, periods of stagnation, or periods of high turbulences when the market behaves crazily and rises steeply in one moment and then suddenly falls, or vice versa. When testing, for example, 1-minute data it is ideal to choose about 10 years of historical data – i.e. for example from August 2003 to August 2013. If we wanted to backtest such a period manually, it would take us a lot of time. However, we recommend to backtest intraday systems by the use of at least 3-year historical data. In the case of positional trading it is suitable to use 20-30 years of historical data for backtesting.
4. We define a set of rules specifying entry and exit conditions.
In order to be able to perform backtesting we need to define the rules for opening and closing individual positions. We can employ technical analysis, namely various indicators, price patterns, or S/R levels.
Now we know which market and timeframe we are going to backtest. We also know how many years of historical data we are going to use and we have defined the entry and exit rules both for Long and Short positions. We can say that we have built a trading system. The aim of our backtest is to determine whether the system was sufficiently profitable and stable in the past. Ideally, the system should be profitable in each tested year.
5. We search all trades corresponding to the parameters of our trading system – the actual backtest.
Now we have to do a very lengthy, yet necessary and useful work. We have to go through the entire tested period day by day, bar after bar (or candle after candle) and search for situations when our entry conditions were met. We can scroll the chart manually or we can use the function for accelerated trading on historical data. This function is part of many software platforms – for example in Sierra Chart this function is called Replay. It allows us to accelerate our trading by a certain time ratio and even to simulate opening and closing of positions.
In case we find a situation where the entry conditions were met, we record it (i.e. the trading position) in a predefined table – we keep a Trading diary. Then we also record the exit from the position. As I mentioned earlier, a problem in the manual backtesting is the fact that traders often “cheat” about their successful entries into profitable trades. For this reason backtests often do not entirely correspond to reality and it is necessary to worsen their results by at least 30%. Yet as you will see soon, this problem does not concern traders using ATS.
What basic information should a trading diary contain?
Table 1: The basic trading diary entries
How can the trading diary for, for example, E-mini S&P 500 look?
Table 2: Trading diary containing 21 trades from September 3, 2003 to October 28, 2003, E-mini S&P 500
In Table 2 you can see 21 trades that were manually recorded within the backtest of the period from September 8, 2003 to October 28, 2003.
6. Evaluation of trading system´s performance
After we finished the backtesting there is another next piece of work ahead us – we have to evaluate the trading system´s potential. We must calculate various indicators of the system´s profitability, such as the equity curve, total net profit, Risk Reward Ratio (hereinafter “RRR”), or the maximum drawdown. You will learn more about the issue of evaluating a trading system´s performance in Chapter Trading System Performance Evaluation.
We have shown on the example how a manual backtesting can be performed. This kind of backtesting is a typical instrument of discretionary traders. Now we will focus on the method used by traders employing programmed algorithms.
We have said several times within the guidebook that traders using ATS employ both for backtesting and live trading programmed algorithms. Thus an ATS trader is inherently also a programmer. I know that many novice traders get scared at this point. I myself belonged among those who are frightened by programming. Programming was always a kind of abstract and difficult concept for me, something that was completely alien to my nature. Yet one day I decided (and now I´m really glad that I did it) to try to study the EasyLanguage programming language. This programming language was created purely for the purposes of trading. It allows programming of simply defined entry and exit conditions for mechanical trading systems and strategies. EasyLanguage is a part of the professional software and trading platform TradeStation. And we can also find it in the no-less professional software Multicharts.
I realised that programming may not be an obstacle. The thing is that the EasyLanguage programming language is very intuitive (because it was created for “non-programmers”). Furthermore, an appealing aspect of algorithmized trading for me was that uncomplicated trading systems and strategies with simple codes often work much better in the long term than complicated multi-parameter entry and exit conditions.
As we have already said, programming software platforms can greatly facilitate and accelerate all trading-related procedures. In Fig. 2 you can see a table containing backtest results. Such tables can be generated in programs like TradeStation or Multicharts in a split second. A manual processing of the same table would be a very lengthy process. Moreover software platforms provide other valuable information on the tested trading system´s performance.
Fig. 2: Automated trading system backtest results, period from September 8, 2003 to October 9, 2003, market E-mini S&P 500, 15-minute timeframe, main trading session (Created in TradeStation)
Simulated trading – papertrading
After a properly performed backtesting when the trader verifies whether his trading system has a real potential for profitability, he should use the system within a simulated trading- papertrading (you can also see the terms Forward Testing or Virtual Trading). Both discretionary traders and traders using ATS should perform papertrading. Papertrading is essentially a verification of the trading system´s functionality on live data in real time. It’s the same like live trading with only one difference – you trade with virtual money. The advantage of papertrading is that it allows you to accurately simulate live trading thanks to which you can verify (to a certain extent) whether you can manage the psychology of trading. The length of the papertrading period derives from the number of trades. In order to be able to compare the results of the backtest with the results of papertrading you need to have a statistically relevant sample of the papertrading results, i.e. at least 30 trades. Ideally, you should use a sample of 50 to 100 trades. Such a sample is usually sufficient to confirm the correlation (similarity) between the trading system´s performance in backtesting and papertrading.
Discretionary traders carry out papertrading much like backtesting. They also use trading diaries in which they record individual trades and their parameters. They trade live data in real time, yet they do so on simulated “demo” accounts. Traders using ATS simply apply their programmed trading system on the simulated account where trades are concluded in real time. Then they just check whether the entry and exit orders are correctly executed.
Backtesting and papertrading are still only simulations of the trading system.
In conclusion, the results of backtesting and papertrading have a real value, yet they do not always faithfully simulate slippage or failures to execute limit orders, i.e. phenomena that occasionally occur during live trading. This must be borne in mind both during backtesting and papertrading. In any case, I can say from my personal experience that a well-executed backtest and the subsequent papertrading are essential and fundamental prerequisites for a successful live trading. Therefore, do not neglect either of these steps. Only through them you can truly learn if your trading system has the potential to be profitable in live trading or not.
In this chapter we have explained the differences between the manual and automated backtesting. We have showed you how a trading diary in which you record individual trades that meet your system´s entry and exit parameters should look like. You learned that the backtesting results are also used for evaluation of your trading system´s performance (we will address the issue of trading systems in detail in the following chapter). The aim of backtesting is to determine whether a given trading system has the potential to be profitable in live trading with real money. We have also explained the concept of simulated trading (or papertrading).