TradeStation Walk Forward Optimizer (WFO) is an advanced tool for optimising automated trading systems (ATS) which automates the very complex multilevel process of statistical Walk Forward testing of ATS input parameters (Inputs). Where the possibilities of most classical optimisation tests end WFO enables performance of set of walk-forward analyses on optimised In-Sample historical data against the unknown and non-optimized Out-of-Sample historical data. The purpose of these tests is to simulate the unpredictability of live trading. The purpose of this simulation is to determine whether the ATS is likely to be profitable in live trading or not. We discussed a lot the principle of optimisation of input parameters and Out-of-Sample testing in the last and last but one chapters so we will not return to these issues.
We will concentrate solely on WFO which is a universal tool for ATS robustness testing using the TradeStation backtesting engine, EasyLanguage, and the advanced backtesting performance reports. In order to help you to fully understand this tool, we must first explain what Walk Forward testing means (Walk Forward tests are the backbone of the Walk Forward Analysis – WFA). To illustrate this concept we drew up the following scheme in which you can see a WFA containing a series of In-Sample and Out-of-Sample Sample historical data with individual Walk Forward tests (8 tests in total).
Fig. 1: Walk Forward Analysis
In Fig. 1 you can see an example of WFA for 12 months (each trader can set the historical period by himself, I often use historical data older than 10 years). This WFA includes 8 In-Sample parts (blue fields) and 8 Out-of-Sample parts (green fields). Thus from the fifth month we actually simulate live trading conditions on unknown data.
Specifically how do we proceed in the analysis?
The basic principle is very simple: We perform optimisation tests for all In-Sample runs and after identification of input parameter settings with the highest In-Sample Fitness Function (FF) we apply these settings to the Out-of-Sample data. If you are not acquainted enough with the issue of FF and optimisation testing, please read again the previous chapters which I have already referred to at the beginning.
It is always good to repeat everything on a simple example:
Imagine that you trade via an ATS with two input parameters (Inputs) for a longer and a shorter moving average. For the longer moving average we use for example the range from 50 to 100 with increments of 10 (i.e. 6 combinations) and for the shorter moving period 5-45 also with increments of 10 (i.e. 5 combinations). In total we get: 5 x 6 = 30 possible combinations of the two input parameters. Now imagine that our FF will be the highest Net Profit. In the first In-Sample run, i.e. the first to the fourth month, we found out that the highest net profit achieved the combination 5 for the shorter moving average and 60 for the longer moving average. We will therefore apply the parameter settings 5 and 60 to the first Out-of-Sample testing, i.e. the fifth month.
In the second In-Sample run, i.e. the second to the fifth month, we found out that the highest net profit achieved the combination 15 and 80. We will therefore apply the parameter settings 15 and 80 to the second Out-of-Sample testing, i.e. the sixth month.
We apply the same principle to the next runs (Walk Forward), up to the In-Sample run 8 with application on the last Out-of-Sample data, i.e. the 12th month.
This principle is called WFA “Rolling”. The point is that we divide the historical data into various parts thanks to which we obtain more Out-of-Sample data for evaluating of the ATS´s robustness. Then we evaluate the robustness potential by the predetermined test criteria. We will introduce these test criteria in some of the next chapters. To start off, we just say that the fundamental test criterion for us should be that the Out-of-Sample results were profitable enough and at least half as profitable as the best In-Sample optimised parameters.
All the above findings show that the WFA analysis is the most realistic simulation of the ATS behaviour in live trading. WFA helps us to answer these basic questions:
- Will the ATS be profitable even after optimisation (the aim of which is to find the most suitable input parameters for live trading)?
- What performance characteristics should the ATS have in order to have the potential to be profitable on unknown data in live trading?
- What impact on the ATS´s performance will have a change in trend, volatility, or liquidity in the future?
- How often should we re-optimise the system´s input parameters (Inputs)?
A much more advanced tool that goes beyond the possibilities of instruments offered by most software trading platforms and which is also included in the TradeStation platform is the Cluster Walk Forward Analysis. It is a set of many Walk Forward analyses the principle of which we will explain in the next chapter.