ARTIFICIAL INTELLIGENCE IN

TRADING

CHAPTER 1 – BACKTESTING

TS. – HUB 16/02/2023 1

CONTENTS
CH 2.1 Why backtesting
AP
TE
R 2.2 Backtesting biases
2

2.3 Backtesting issues

2.4 Backtesting costs

TS. – HUB 16/02/2023 2

2.1 Why Backtesting?

Provide expectations about future performance from past

performance
Each trade (mean here to be a 'round-trip' of two signals) will have

TS. – HUB 16/02/2023 3

2.1 Why Backtesting?

Filter: initial research stage was to set up a strategy pipeline and then filter
out any strategy that did not meet certain criteria →

Model: test new models of certain market phenomena, such as transaction

costs, order routing, latency, liquidity or other market microstructure issues
TS. – HUB 16/02/2023 4
2.1 Why Backtesting?

Optimization: increase the performance of a strategy by modifying the

number of parameters and recalculating its performance.

Verification: strategies are often sourced externally → backtesting ensures

strategies correctly implements

TS. – HUB 16/02/2023 5

2.1 Why Backtesting?
Common Pitfall:
Usually, a backtest would produce a historical performance that is
better than what we would have obtained in actual trading.

TS. – HUB 16/02/2023 6

2.2 Backtesting biases

Optimization Bias (most insidious of all backtest biases):

adjusting trading parameters based on transient noise until the
strategy performance on the backtest data set is very attractive,
however,
Other names: “curver fitting”/ “data-snooping bias”
TS. – HUB 16/02/2023 7
2.2 Backtesting biases
Optimization Bias
+ it’s hard to eliminate as algorithmic strategies often involve
many parameters
+ it can be minimised by keeping parameters to a minimum and
increasing the quantity of data points in the training set.

TS. – HUB 16/02/2023 8

2.2 Backtesting biases
Optimization Bias (Overcome issue)
taking short period data (to avoid many parameter inputs; regime
shifts) → The less independent data you have, the

TS. – HUB 16/02/2023 9

2.2 Backtesting biases
Optimization Bias (Overcome issue)
Sample size: rule of thumb (number of parameters * 252)
Out-of sample testing: Spliting historical data into two parts → one part for
training (at least 70% of total observations, and follow rule of thumb) & one
part for test (more recent data) → build the model based on the training part
and validate the model based on the test part with reasonable performance.
TS. – HUB 16/02/2023 10
2.2 Backtesting biases

Optimization Bias (Overcome issue)

Conducting sensitivity analysis:
Varying parameters incrementally and plotting a "surface" of
performance → smooth surface means good parameters
→ very jumpy performance surfacesmooth surface means good
parameters→ very jumpy performance surface, means that a
TS. – HUB 16/02/2023 11
2.2 Backtesting biases
Optimization Bias (Overcome issue)
Conducting sensitivity analysis. How?
make some small qualitative changes in the features of the model
and see how the performance changes on both the training and the
test sets → If the drop in performance is so drastic

TS. – HUB 16/02/2023 12

2.2 Backtesting biases
Look-ahead Bias
Future data is accidentally included at a point in the simulation
where that data would not have actually been available.
i.e: backtest at reach time point N, then look-ahead bias occurs if
data is included for any point N + k, where k > 0.

TS. – HUB 16/02/2023 13

2.2 Backtesting biases
Look-ahead Bias
Example: if your trade entry rule reads: “Buy when the stock is
within 5% of the day’s low”
→ you could not possibly have known what the day’s low was

TS. – HUB 16/02/2023 14

2.2 Backtesting biases
Look-ahead Bias reasons
+ Technical Bugs: incorrect loops, which include N+k in the test
+ Future data: If the whole data set (including future data) is used
to calculate the regression coecients, then future data is being
incorporated and a look-ahead bias exists.

TS. – HUB 16/02/2023 15

2.2 Backtesting biases
Look-ahead Bias overcomes
+ Lagged historical data: like moving average (previous trading
period only)
+ If not, please make sure series are stationary to eliminate noises
(season, trend…)

TS. – HUB 16/02/2023 16

2.2 Backtesting biases
Survivorship Bias
Strategies are tested on datasets that do not include the full universe,
but only consider those that have "survived" to the current time.
Example: chosing a sample data only including “survied” stocks after
financial crisis. It does not tell an entire historical story (some
companies has been bankrupted) →
TS. – HUB 16/02/2023 17
2.2 Backtesting biases
Survivorship Bias

P&L = -42% Actual return P&L = 388% Fictitious return

TS. – HUB 16/02/2023 18
2.2 Backtesting biases
Survivorship Bias
Overcome issue
1. include delisted entities (bankrupted firms) → costs
2. Collect data for your own use manually at the current time
3. Use More Recent Data: not include shocks, which less likelihood
of overall stock delisting in shorter time periods (3-4 years)
TS. – HUB 16/02/2023 19
2.2 Backtesting biases
Cognitive Bias
Human does not keep discipline due to coginitve decision.
Example: the strategy might possess a maximum drawdown of 25%
and a maximum drawdown duration of 4 months. However, you may
not be unpleasant since you see your money loss → stopped from
trading → significant loss at the end → should follow the strategy
TS. – HUB 16/02/2023 20
2.3 Backtesting issues
Exchange Issues
a. Market orders and Limit orders
Market orders: executes a trade immediately with no price consideration
→
Limit orders: determine the price at which the trade will get executed.
→
TS. – HUB 16/02/2023 21
2.3 Backtesting issues
Exchange Issues
a. Market orders and Limit orders
Discussion: High/Low price vs Close price?

Discussion 2: OHLC vs Candlestick

TS. – HUB 16/02/2023 22

2.3 Backtesting issues
Exchange Issues
b. Shorting Constraints
What is the short sale?
Note: some equities may not have been available (due to the lack of
availability in that stock to borrow) or due to a market constraint
→ be careful for liquidity constraints
TS. – HUB 16/02/2023 23
2.3 Backtesting issues
Data for trading
Try: “free historical intraday futures data”
Trade-off between Cost & Quality

TS. – HUB 16/02/2023 24

2.3 Backtesting issues

Data for trading

TS. – HUB 16/02/2023 25

2.3 Backtesting issues
Data for trading (be careful)
+ Are the Data Split and Dividend Adjusted?
Some stocks been splited/ dividend adjusted in their lives → adjusting prices
a) For Data Split: all the prices before T need to be multiplied by 1/N.
b) For dividend: all the prices before T need to be multiplied by
(Close(T – 1) – d) / Close(T –1), where Close(T – 1) is the closing price of
the trading day before T.
TS. – HUB 16/02/2023 26
2.3 Backtesting issues
Data for trading (be careful)
+ Are the Data Split and Dividend Adjusted?
Example:

TS. – HUB 16/02/2023 27

2.3 Backtesting issues
Data for trading (be careful)
+ Are Data Survivorship Bias Free ?
Check it on the previous section

+ Does Your Strategy Use High and Low Data

Check it on the previous section

TS. – HUB 16/02/2023 28

2.4 Transaction costs
a. Commission and fees
For an exchange (directly or broker); pay and settle trades; taxes…

TS. – HUB 16/02/2023 29

2.4 Transaction costs
b. Slippage
difference in price between the time when a trading system transact
and the time when a transaction is actually carried out. (latency)
→

TS. – HUB 16/02/2023 30

2.5 Summary

Simulated and Realistic Backtesting could be distinguished (due to

overfitting, poor data cleansing, incorrect transaction cost inputs,
market regime change…)
→

TS. – HUB 16/02/2023 31