August 22, 2017 · Blog Posts
3 Steps to Successful Trading Performance Analysis
The markets have become ever more sophisticated for professional traders, and yet, many times traders take an unsophisticated approach to measuring their performance. While a trader will spend countless hours studying markets, developing trading strategies, and successfully deploying those same strategies into production, little or no time is spent in examining the factors that contribute to an underperforming strategy or, worse, a once winning strategy that fails to be profitable at all. Based on our experience we believe a thorough and robust performance measurement program that includes a comprehensive set of inputs can have profound results for professional traders.
Three steps in the process
At the outset, establishing a performance measurement framework can seem daunting. While it’s true that rigor and dedication is required to produce results, the essential bones of a successful program boil down to three facets:
- determining what data to capture
- creating the processes to obtain the data and generate performance reports
- establishing a process to analyze and act on the reports
It all starts with the data
The first step in establishing a performance management framework is to figure out what data is crucial for your analysis. While this may seem obvious, there’s more to the question than meets the eye. While it’s critical to capture tick data, P&L, and risk parameters, there are other areas that should be examined as well. For example:
- What are the latencies for your specific trading strategies? Tick-to-trade, fill-to-hedge, and tick-to-cancel latencies are some examples of what could be measured.
- What is going on at the exchange? Capture your queue position, be aware of the overall message volume, and track exchange acknowledgement times. Changes in any of these factors can impact how your strategy performs.
- Trading ratios are extremely important to consider. Besides the order hit ratio, which measures the number of fills compared to orders sent and can result in exchange fines if it is too low, traders can look deeper and monitor an edge hit ratio, which is a reflection of edge captured versus potential or expected edge.
- What events took place that triggered your order/trades? Be sure to take a broader view beyond a single expiration or product in order to avoid market myopia.
- Are you chasing phantom trades? Firing and missing is one thing if no one else gets the trade that you were looking for and quite another if your miss is someone else’s hit.
- Finally, be diligent in tracking any changes that you make to both hardware and software. The interaction of internal changes with external markets are not always intuitively obvious and can have unexpected and unusual impacts on trading.
Automated processes are critical
Once you have determined what you need to look at and why, it is important to establish automated processes to capture and generate performance reports. Reports that are generated on an ad-hoc or as-needed basis simply won’t deliver the same consistent level of analysis and insight that are needed to make the performance management exercise effective. It’s simple human nature: if it isn’t a part of a prescribed and consistent routine then it won’t get done, at least not with the consistency that is needed to compete in today’s hyper-competitive markets. Publishing reports automatically will force people to look at the data.
In terms of the reports themselves, it’s important to include not only day-to-day statistics but also how the data trends over time. It’s easy to miss important factors if the time view is too short and a longer and wider window can reveal events and patterns that may not be obvious. Standardized performance reports will give a good first view at what is going on but it is often necessary to dig deeper to fully understand a situation.
Put in the time if you want to get the results
Collecting all the right data and generating automated and flexible reporting functions are all fine and well but they won’t mean a thing if you don’t take the time to figure out “why” when you see changes in data. The final step in the process is to take the time to consistently examine and analyze your results.
Some of the factors that should be considered are:
- Has there been a software change? It may be that a seemingly innocuous and unrelated change to some step in the software chain has an unintended downstream effect.
- There is a change in latency but where is the change originating from? Whether or not the issue resides within your four walls, at the network level, or with the exchange will have an impact on how you respond. Some of these issues can be dealt with and corrected directly while others are outside of your control and must be adjusted to.
- Have you seen an increase or decrease in message volume? This is frequently overlooked but it can have a significant impact on the performance of a strategy.
- Latency is consistent but hits rates have gone down: why? The first thing to look at is phantom versus real orders: you may not be getting the trade but no one else is either. If that’s not the case, the knee-jerk assumption is that someone else simply got faster but that’s not always the case. It could be someone is triggering off of a different signal which is allowing them to react earlier. Being smarter about which signal you’re using can provide a significant head start to the competition. In any case, it’s important to consider all possible reasons for a change in performance and not to make any quick and unfounded assumptions. This is why it is so important to capture and monitor all sources of data.
- If everything else is accounted for then the last thing to look at is hardware. In some cases, a new piece of hardware or change to the hardware configuration is an easily identified culprit but in others a change in performance may arise from an undetected issue with one of the hardware components. The important lesson here is to consider all elements, including the physical ones.
Performance is too important to ignore
As markets get more complicated it is ever more important to take a holistic look at all aspects of performance. Traditional measure of performance like P&L and risk are important but they’re not enough to compete in today’s markets. To thrive, much less survive, traders must look at all factors that can affect performance, set up the systems to consistently capture and deliver reports, and be diligent in consistently monitoring and analyzing all factors in their performance. At the end of the day, you need to make sure your in-house or third-party system provides you with the you the tools to capture the data. With Rival you have a partner, not just a technology vendor, as we help you bridge the gap between business and technology.