.. _user.replay: ======================================== Creating An Automated Backtesting Script ======================================== Motivation And Context ~~~~~~~~~~~~~~~~~~~~~~ The replay feature in SC can be useful for backtesting. As part of your strategy development effort, in order to optimize or validate parameter stability, you sometimes need to run many replays and this can be both time consuming and error prone. SC-Py's bridge includes methods we can use to automate this process. In this tutorial we will go demonstrate how we can use those functions for automation and create a simple optimization framework. Learning Objectives ~~~~~~~~~~~~~~~~~~~ * How to work with the SC-Py replay API. * How to work with SC-Py functions that update the chart and study settings. SC-Py Functions Used in this Tutorial ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ * :func:`~trade29.sc.SCBridge.start_replay` * :func:`~trade29.sc.SCBridge.subscribe_to_replay_updates` * :func:`~trade29.sc.SCBridge.set_chart_study_input` * :func:`~trade29.sc.SCBridge.set_intraday_bar_period_parameters` * :func:`~trade29.sc.SCBridge.get_flat_to_flat_trade_list` User Scenario ~~~~~~~~~~~~~ For this tutorial we start with the situation where you have a study that autotrades a fully automated strategy and you wish to backtest and test how the system behaves under different parameters. For our example we will use a simple ma crossover strategy that includes the following specification * Trades ES on an intraday chart * Includes a Fast MA and Slow MA * Long signal - when the fast ma crosses the slow from below * Short signal - when the fast ma crosses the slow from above * Has a stop and target defined in the SC trade window * Position size is fixed and defined in the study settings The user starts with the fast ma set to 10 and the slow ma set to 30 and a 10 min chart but is wondering what would happen if we use different ma settings and different chart timeframes. One way to do that is run replays with different permutations for each of those 3 parameters. The user decides to run a 2 month backtest for the following values .. list-table:: Parameters :widths: 33 33 33 :header-rows: 1 * - Fast MA Period - Slow MA Period - Minutes Per Bar * - 10 - 30 - 10 * - 20 - 60 - 10 * - 30 - 90 - 10 * - 10 - 30 - 30 * - 20 - 60 - 30 * - 20 - 90 - 30 In total the user needs to run 6 backtest replays. The general workflow would look something like this 1. Set the settings for 10, 30, 10 (i.e., using the values in the first row in the table) 2. Scroll the chart to the beginning 3. Start the replay 4. Wait for the replay to finish 5. Save the trades to a file Repeat steps 1-5 for next set of parameters. This would be done 6 times. This is a form of brute force optimization and can be fine for a few runs. But if you are doing this at scale, running it many times over long periods, it can be time consuming and error prone. In this tutorial we will show how this can be automated using SC-Py. Setting Up Sierra Chart ~~~~~~~~~~~~~~~~~~~~~~~ First we must create the SC custom study. Since this is an SC-Py tutorial, not an ACSIL tutorial, you shall simply be given :download:`this zip ` with the DLL and the .cpp to satiate your curiosity. Now put the DLL into your ``SierraChart/Data`` directory and start up SierraChart. Open an ES intraday chart, and apply the ``Simple MA Crossover`` custom study and the ``SC-dX`` custom study with the key ``A``. Make sure you note the ID of the ``Simple MA Crossover`` custom study for later. If it was applied first it should be ID 1. Implementing The Script ~~~~~~~~~~~~~~~~~~~~~~~ The script we want to build can be sketched into a few steps: 1. Set the two MA periods and the chart timeframe. 2. Start the replay. 3. Wait for the replay to finish. 4. Save the trades. 5. Go back to step 1 with a new parameter set. With that list we can create a skeleton of our python script: .. literalinclude:: examples/replay/skeleton.py :caption: skeleton.py :linenos: :lineno-match: Setting the parameters ---------------------- The :func:`~trade29.sc.SCBridge.set_chart_study_input` method can be used to set study inputs. Note how this method wants an index for the input to set. This is a 1-based index, and inspecting the study inputs in Sierra Chart shows that the period for the fast MA is input 1 and the period for the slow MA is input 2: .. thumbnail:: media/replay/figure1.png Make sure you use the right ID for the ``Simple MA Crossover`` custom study. .. literalinclude:: examples/replay/version0-0.py :caption: version0.py :linenos: :lineno-match: :start-after: S0 Next we need to set the period of the chart. For this we use the :func:`~trade29.sc.SCBridge.set_intraday_bar_period_parameters` method. We want to use period type ``0``, which is for time based period, and we must note that the period is given in **seconds per bar**, so we need to multiply our minutes per bar by 60. .. literalinclude:: examples/replay/version0-1.py :caption: version0.py :linenos: :lineno-match: :start-after: S0 At this point you should run this script and check to see that the study parameters and timeframe have changed. Starting The Replay ------------------- SC-Py provides a bridge method :func:`~trade29.sc.SCBridge.start_replay` for starting replays. We can run replays really fast, since all of our trading logic is happening in Sierra Chart. For testing, we will simply run a replay starting 2 days from the current time, but obviously in a real testing scenario you would want a constant time so each backtest can start at exactly the same time. .. literalinclude:: examples/replay/version1-0.py :caption: version1.py :linenos: :lineno-match: :start-after: S0 As with before, you should run your script and verify that a replay is being started. Waiting For The Replay To End ----------------------------- There are two methods we could potentially use to get our replay state. We have the async :func:`~trade29.sc.SCBridge.subscribe_to_replay_updates` and the sync :func:`~trade29.sc.SCBridge.get_replay_status`. In this case we could make it work with either, so I decided to use the async version because you can never use too many async examples. First we need to create a subscription for replay updates: .. literalinclude:: examples/replay/version2-0.py :caption: version2.py :linenos: :lineno-match: :start-after: S1 :end-before: S0 There are a few things to note about the :func:`~trade29.sc.SCBridge.subscribe_to_replay_updates`. Firstly, by default, it will only send us the replay status immediately after we make the subscription, and every time the status changes. Thus we need to throw away that first subscription, because we only need updates after we start the replay. Now, since we only get updates when the status changes, after the replay starts we can simply filter the response queue until we get told that the replay has stopped: .. literalinclude:: examples/replay/version2-0.py :caption: version2.py :linenos: :lineno-match: :start-after: S2 Saving The Trades ----------------- Finally we can use the :func:`~trade29.sc.SCBridge.get_flat_to_flat_trade_list` method to get all of the trades we performed. By default, the :func:`~trade29.sc.SCBridge.start_replay` method will clear trade history data, so we dont need to filter the trade list, as it will only include trades since the replay started! Thus we can get our trades, and at the end of the script save them all to files: .. literalinclude:: examples/replay/version3-0.py :caption: version3.py :linenos: :lineno-match: :start-after: S2 Our script is now fully implemented! Conclusion ---------- We did alot this tutorial. We changed subgraph inputs through SC-Py, and changed the chart timeframe. We learned about starting replays, detecting when they end, and getting trade lists. All of this culminated in a script that automates a very tedious testing process, leaving us with a series of CSVs that we can analyse to decide on the optimal parameters to run our strategy with. Next Steps ---------- The next main step would be adding a constant start datetime thats best for your testing scenario. After you do that, you could consider adding further reporting, potentially reporting the average or cumulative P/L of trades.