The analysis of high-frequency financial data (e.g. future prices on a 1 second grid) provides an excellent training ground for the data scientist.
Like in Quantum mechanics, statistical arbitrage strategies acknowledge that it is impossible to predict a specific event, but it is possible to predict the statistical average outcome if a fundamental relation exists between the observed and predicted quantities.
A well-known example of this in finance is pairs trading, which uses the fact that some price series are very highly correlated (like bonds of different maturities, or indices of different European countries). So a move in the DAX might induce a similar move in the EuroStoxx 50 (or the other way around). In the end, the name of the game is understanding multidimensional probability distributions, the natural habitat of the particle physicist.
Finance, where data is sparse (compared to particle physics), can profit from the application of sophisticated analysis methods that were developed in fields were data is more abundant.
On the other hand, performing analysis in finance may sharpen the Data- Scientist’s abilities in two important ways:
- Since signals are very weak buried under a see of randomness a great amount of conviction and mastery of analysis tools is needed until the signal becomes clearly visible.
- Live trading provides a direct feedback and any sort of misjudgment in the analysis process will nearly always result in the los of money immediately (at least when done at high frequency).
The graph below shows the live performance of a strategy I worked on over a period of two and a half years.
The realized Information Ratio was 6.9, and P&L/maximumDrawDown was 62. The main strategy was running on a 1 second grid, while the execution strategy would react as fast as possible to price changes and changes in the order book of correlated instruments.
Latencies are not overly important here, but should be of the order of 10 milliseconds for Euronext for instance (from signal generation to receiving the order execution confirmation).
But it is essential to have a good execution strategy to minimize slippage. The average profit per trade is usually less than the spread, which also means less than one tick in most cases. Thus, it is necessary to have a good broker. The total execution costs (fees + commissions) should not exceed 15 percent of the spread.
The stable long-term performance is mainly due to portfolio diversification effects, resulting from trading a large number of pairs of bond and index futures. I will discuss more details on indicator construction, practical portfolio construction and execution strategies in later posts.