In fact, the results in Table 2.3 confirm our expectations, although we also find other information. Viewing it in the most general sense, the short-term maturities, the short sterling, the European Interbank Offered Rate (Euribor), and Eurodollars are all consistently at the bottom, showing the least noise and the most trend. As the maturities get longer, the prices show increasing noise, although with only a few exceptions the interest rate markets all remain in the lower half of the table.

We see a similar case for equity index markets, cl.u.s.tering near the top of the table. The DAX is the bottom market in each column, indicating the least noise of that group.

As we look from the fastest calculation period, 8 days, to the slowest, we see a shift upward of the longer-term U.S. interest rates. This seems to reflect the trendiness of the past 10 years. The U.S. 30-year bonds spent a number of years moving sideways with relatively low volatility, even when the short-term rates continued lower.

Most other markets scatter throughout the center of the table with the exception of palladium, which is consistently cla.s.sified as very low noise.

We take one last look at the trend performance using both noise and a moving average taken over the same calculation periods, 20, 40, and 80 days. We find that doubling the calculation period gives a better test sample. As before, we scatter the results using the average noise as the bottom scale and the information ratio as the left scale, shown in Figures 2.6a, b, and c. Figure 2.6a seems to have no particular pattern, but that can easily be caused by a mixture of poor performance because 20 days is not long enough to capture macrotrends in most of these futures markets.

FIGURE 2.6 (a) Results of a 20-day trend applied to 25 markets and scattered by average noise and information ratio. (b) Results of a 40-day trend. (c) Results of an 80-day trend.

The 40- and 80-day calculation periods show an increasingly clear pattern where higher values of noise (less noise) produce better returns, indicated by a higher information ratio. Any information ratio below zero indicates a loss. The three diamonds at the far right are the results of the three short-maturity futures markets, Euribor, short sterling, and Eurodollars. The negative ratios are generally the equity index markets, but the 80-day trend shows that 30-year bonds and gold also generated net losses from 2000.

CAPITALIZING ON THE TREND OF NOISE.

We've looked at the long-term average of noise for each of the markets. These averages create an orderly ranking that shows which strategy is most likely to succeed. But over the short, 8-day calculation period, the noise of any market can vary considerably. Figure 2.7 shows the SPX, along with the 8-day efficiency ratio for the first quarter of 2009. During this interval, the SPX drops to its lowest point in March 2009, and the efficiency ratio is seen to peak twice and touch bottom four times. The average of the ratio for the period in the chart is 0.37, although it ranges from near zero, 0.01, to 0.87 in February.

FIGURE 2.7 SPX with an 8-day efficiency ratio, first quarter, 2009.

Observing the patterns on a chart always gives rise to hopeful ideas. In this case, we see that the S&P dropped twice, from about 930 to 800 in January and again from near 850 to under 700 in February and March. At the same time, the ratio shows two insightful patterns. After the ratio peaks in mid-January, it drops quickly and stays low during an interval where the S&P goes sideways. Then, just after February 13, 2009, it starts up sharply, indicating an improved trend, and stays high through the entire period when the S&P drops 150 points. Can the trend of the ratio be used to switch from a trending strategy to a mean-reverting one? Or can it be used to tell us when to stand aside using either strategy?

It's an interesting idea that will be left to the reader to pursue. Once such a pattern is observed, we each approach a solution differently, and this is an exercise worth some investment of time.

1. Prior to this, Mandelbrot published Fractals, Chance and Dimension (W. H. Freeman, 1977).

Chapter 3.

Pairs Trading: Understanding the Process.

Pairs trading is not an industry secret. It is a basic market arbitrage and can be done by any conscientious trader. It may be that greater compet.i.tion in recent years means that you will need to be more selective about the choice of trades, but the opportunities are still there.

The basic stock market pairs trade begins by choosing two companies that are fundamentally related. We can use a simple correlation tool, which we will discuss later, but that's not necessary. By observing the chart of one company against another we can see that the two companies move in much the same way, reacting to news in a similar manner.

The best pairs trades will be between direct compet.i.tors, such as Dell and Hewlett-Packard. There may be periods of time when prices move significantly apart, but even during those intervals, there will be a lot of similarity. We don't want the two stocks to track each other too closely, because that also means there is limited opportunity. We also need to be vigilant about structural changes in one of the companies and how that might affect their relationship to other companies in the same industry. A price jump following an earnings release is not a structural change. Most often the first chip manufacturer to report indicates that other manufacturers in that industry should follow with similar performance.

The trade itself will sell the stronger stock and buy the weaker, looking for their prices to correct, or come back very close to each other. When they do correct, the trade is exited. The number of shares traded for each stock will be calculated to equalize the risk of the long and short. This approach is the basic statistical arbitrage (stat-arb) trade. What differs from trader to trader is the selection of pairs, the size of the distortion being targeted, the size of the positions, and the exit criteria, but not the concept.

THE PROCESS.

This section steps through the process of developing a pairs trading strategy. A similar procedure is used for the other strategies in the chapters that follow, but it is important to understand how it comes about. There are clear rules to be followed; otherwise, you cannot have confidence that the final system will work. Both our computers and development tools are extremely powerful, perhaps too powerful. It is easy to beat the problem into submission. Our challenge is to keep it simple and robust. We must think of the problems that can be encountered in advance and use our data and tools wisely. At the end, we will review the process and comment on critical steps.

An important advantage of a rule-based system is that, once implemented for one pair using a spreadsheet or computer program, it can be easily extended to others. That will allow us to test for robustness. If we are doing it correctly and it worked on Dell and Hewlett-Packard, then it should work on similarly related pairs. Once we have a number of pairs checked out successfully, hopefully in different sectors, you have gained confidence in the method as well as diversification.

All trades will be entered and exited on the daily closing prices, although that won't be necessary for actually trading. You could enter prices into the spreadsheet at any time during the day, perhaps 15 minutes after an economic report when volatility is high, and enter a trade any time there is a distortion that satisfies our minimum threshold.

We will try to approach each problem as we would without any foresight as to what is the correct solution or the best markets. When exploring a new method, there are always false starts. Each time we uncover a problem, we learn more about the process and the solution.

THE BASICS.

We'll start by choosing four U.S. airline stocks, which we intuitively believe react to the same economic fundamentals. If the economy is strong and there is more disposable income, then more people fly for both business and pleasure, and if the economy is weak, they stay put. The candidates are American, Continental, Southwest, and U.S. Airways, all large companies, with Southwest the only domestic carrier. We will use American and Continental as the primary example in the following sections. They were chosen because both had the maximum amount of data and both were major domestic and international carriers. They turn out to be neither a very good example nor a very bad example. By the time you read this, one or the other of the airlines may have a different name, but that won't affect our results.

FIGURE 3.1 The price history of American Airlines (AMR) and Continental Airlines (CAL), 10 years ending July 2010.

If we show American (AMR) and Continental (CAL) on the same chart (Figure 3.1), we can see the similarity in their price movement. Most of the time Continental trades above American, but there are obvious differences that could be turned into either profit opportunities or large trading losses. Most important, we see that on September 11, 2001, both companies took a plunge of more than 50%, based on the public perception that no one was ever going to get onto an airplane again. For those readers who are good with spreadsheets, the correlation of the daily changes over the 10 years is .94, quite high. Remember that correlations always use the price differences and not the actual prices and that the differences are actually returns and are best calculated as a percentage change, It may also seem from the chart that the volatility of Continental is higher than that of American. That would be a good guess because the average price of Continental was higher than that of American during this period. But that's not the case. Over the full 10 years, Continental had an annualized volatility of 69% while American was 81%. Because the volatility is calculated in the standard way used by financial inst.i.tutions, where the daily returns are percent changes, these values are somewhat self-adjusting to the differences in price.

Time Perspective.

We need to decide if our trading should target a few big profits or many small ones. My own preference is for many faster trades, even though a series of small losses could total the same as one large loss generated by a longer-term strategic trade. With faster trading, it is possible to have more consistent performance because the individual trade risk is smaller. If you hold a trade for a long time, then the risk increases proportionally (but not linearly) with the holding period, so it's not possible to have a long-term trade with low risk. More trades also give greater statistical confidence.

If we think of the conclusions given in Chapter 2, "The Importance of Price Noise," we should remember that operating in a shorter time period emphasizes price noise, and taking a longer view emphasizes the trend. Pairs trading a.s.sumes that differences between the two stocks will be corrected; therefore, we are dealing with noise. Using a shorter time horizon will benefit us.

A more intuitive, convincing argument may come from looking at the chart of the two stocks in Figure 3.1. It appears that, about September 2000, Continental moves higher while American moves lower. This is certainly a big opportunity. Within two weeks, prices converge again, and by 9/11 they are virtually the same. Buying American and selling Continental short would have been highly successful. In August 2006, we get another separation, where Continental gains over American, and Continental appears to remain higher until August 2008. In this case, if you consider that the size of the risk increases with time (as do the profits), we have another argument in favor of shorter trades.

The princ.i.p.al argument against frequent trading is the increase in cost. More trades mean acc.u.mulated execution slippage and commissions, and a shorter time frame also means less volatility and smaller profits per trade. Volatility is the main enemy of many of these strategies, which are mean reverting. Prices must move apart by some minimum amount to offer enough profit opportunity.

Note on Data and Costs.

Be sure to check that all the data are in the same format. Even within the same data service, the stocks series that are automatically created may be without any decimal points, so a share price of $22.15 could be shown as 2215. Data downloaded specifically by the user, for example, a stock that is not in the S&P 500, may be shown in decimal form, such as 14.50. For most programs to work correctly, all of the data should be in the same format.

The examples that follow using stocks do not reflect any costs. Instead, the results will always show the returns per share. Costs vary considerably from trader to trader, with professionals paying less than $0.001 per share. You will need to decide whether the potential returns are sufficient given your own costs.

Correlations and Common Sense.

In expectation of adding U.S. Airways (LCC) and Southwest Airlines (LUV) to the mix of pairs when we are done with American and Continental, we will look at data beginning on September 29, 2005, when U.S. Airways began trading on the NYSE. We'll call that the beginning of our test period. When we compare the correlations of the price changes for the four companies (see Table 3.1), we get a wide range, with American, Continental, and U.S. Airways closely tracking one another, but Southwest very different. We can explain this because Southwest is exclusively a domestic carrier, but we don't know if that will make a difference to our trading. It will give us the opportunity to see if the greatest profits come from markets that are more or less correlated while still in the same transportation group. Although we're sure from the 0.766 correlation that American and Continental will correct any differences, we don't yet know if those divergences may turn out to be too small for practical trading, or if the correlations are good but the volatility needs to be watched.

TABLE 3.1 Cross-correlations of American (AMR), Continental (CAL), U.S. Airways (LCC), and Southwest Airlines (LUV) from September 2005 to February 2009. Southwest is clearly different.

If we were fundamental traders, we could explain that Southwest has a different, discount business model. We could remember the news articles that credited Southwest with extensive jet fuel hedging success. But we can't confirm these statements, and we don't know when an average jet-fuel price that's 40% under the market will turn around to become 10% over the market. We do know that Figure 3.2 shows a very different price pattern for Southwest than for the other airlines. Again, we'll let the trading results confirm whether it should be included in the pairs trading mix. For now, we can feel comfortable that three of the four airlines have similar price moves; therefore, they are candidates for pairs trading.

FIGURE 3.2 Comparison of four airlines, September 29, 2005, to January 30, 2009.

Percentage Deviation.

The basic approach to pairs trading is to look at the percentage differences in the daily moves of the two stocks, in this case AMR minus CAL. When the difference is too high, we sell short the stronger (AMR) and buy the weaker (CAL); if too low, we buy AMR and sell short CAL. Based on our observation and the correlations, we can expect this divergence to correct within a few days.

Before you express concern about short selling, it is very common today for actively traded stocks; however, at the end of the airline example, we will show how to use sector ETFs as a simple alternative for the short side.

As an example of using percentage differences, suppose that American gained 1.5% today and Continental gained 0.9%. Then the difference (AMR CAL) would be +0.6%. We now need to know if +0.6% is unusually high. To gain a better understanding of the range of these values, we calculated them for our test period and got the results shown in Figure 3.3.

FIGURE 3.3 Daily percentage difference in American Airlines (AMR) and Continental (CAL).

We can see from this chart that the volatility of the AMR-CAL differences changes drastically over the four years. First, from the beginning in September 2005 until September 2006 volatility was uniform, with daily percentage differences ranging from about +5% to 5%, with a maximum of about 8%. For the next 9 months, volatility dropped. Had we been trading before July 2007, we might have created the following trading rules: Sell short AMR and buy CAL on the close when the daily change in AMR exceeds the change in CAL by 2.5%.

Sell short CAL and buy AMR on the close when the daily change in CAL exceeds the change in AMR by 2.0%.

Selling AMR with a 2.5% threshold and selling CAL with a 2.0% threshold accounts for the upward bias we can see on the chart, where the maximum was 8% and the minimum about 5%.

After May 2007, volatility begins to increase until by mid-2008 when the maximum difference has become 17.5%, far greater than the previous 8%. Had we sold AMR at a 2.5% difference over CAL using our previous rules we would have, theoretically, been holding the same trade when the difference reached 15%. But that would never have happened. When the trade reached 10%, we would have all rushed for the exit, forgetting the opportunities. Without going through the tedious exercise of figuring out all the trades, we can see that the basic method has too much risk and too much variability. This method may have worked during the 1960s, but it is not the one we want now.

It is also good to see that we used the chart to decide buy and sell levels that were not symmetric. These levels, 2.5% and 2.0%, clearly fit the pattern but are not a good way to find a solution. Generally, we see this skew in the prices because the stocks have been moving in one direction, in this case up, or the volatility of one stock is greater than the other. When prices change direction so does the skew. A solution that is going to last will need to be more robust, and the safest approach will always be a symmetric solution.

Changing volatility is also a good lesson in why using more data is better than fewer data for testing. If you were to choose only the last year of data, then you get a narrow, unrealistic idea of market patterns. Whatever program you develop will have a short life span. When you see a structural change in the volatility, you need to consider a more dynamic way to adjust to the market. In Figure 3.3, there are three distinct volatility regimes: moderate, low, and high. By finding one method that adjusts to these situations, you can create a more robust approach.

What is needed is a way to adapt the entry levels to changes in market volatility. We not only are concerned with increased volatility and the a.s.sociated increased risk but also realize that if volatility falls, as it did in 2006, and we're waiting for a 2.5% difference for an entry trigger, then we could wait months before seeing a new trade. The problem needs to be solved for both increasing and decreasing volatility.

Relative Differences.

The solution to adapting to changing volatility is to recognize relative differences. One method of showing relative differences is to use a momentum indicator, such as relative strength (RSI), stochastic, or moving average convergence-divergence (MACD). They all accomplish the same thing in slightly different ways. In our examples, we'll use the stochastic indicator because it is easier to calculate and, interpreted correctly, will give you the same results. In addition, it has less lag than the other indicators. The basic calculation for the stochastic is where Ct is today's close, min(Ln days) is the minimum price (the lows of the day) over the past n days, and max(Hn days) is the maximum price (the highs of the day) over the past n days. The denominator is then the maximum to minimum price range of the past n days. The stochastic indicator actually shows the positioning of today's close within that range, essentially expressed as a percent measured from the low of the n-day range to the high of that range.

The value of the stochastic can vary from 0 to 100. If the past range for AMR was from $22.00 to $18.00 and the close is now at $19.00, then the stochastic will be 25. If today's close was a new high, the stochastic would be 100.

For those familiar with this momentum indicator, our definition is for the raw stochastic. Most trading software show a much slower version, created by taking the 3-day average of the raw stochastic, then again taking the 3-day average of that result, giving essentially a 4.5-day lag (half of 9 days). For our purposes, that creates an indicator that is too slow.

Table 3.2 gives an example of the raw stochastic calculation from the beginning of the data. The calculation period is 10 days; therefore, the first 9 data rows are blank. Beginning in row 11 on February 3, 2000 (row 1 has the headings), column H shows the high of the past 10 days = max(B2:B11) and column I shows the 10-day low = min(C2:C11). Using the highs and lows of the past 10 days, we can calculate the AMR stochastic in column J as = (D11-I11)/(H11-I11). Once both AMR and CAL stochastics are calculated in columns J and M, the differences = J11-M11 are entered into column N.

TABLE 3.2 Stochastic indicators created from AMR and CAL daily prices.

If we calculate the traditional 14-day stochastic (usually the nominal calculation period found on charting services) for AMR and CAL during the second half of 2007, we get the picture shown in Figure 3.4. Both markets move in a similar way between 0 and 100; however, they do not reach highs at the same time, and some of the lows are also out of phase. Without those differences, there would be no opportunity. Based on this view of the divergence in the two stocks, we can devise trading rules that profit from it. Although we think that we can see where the momentum values are farthest apart, the first step is to show those differences more clearly. That can easily be done by plotting the differences between the AMR and CAL stochastics, as shown in Figure 3.5. We'll refer to that as the stochastic difference (SD), in column N on the spreadsheet.

Stochastic Difference.

Figure 3.5 shows both the annualized volatility and the stochastic differences for AMR and CAL. The second half of 2008 was chosen because of its historically high volatility, when everyone thought that the end of the world was coming. It is interesting to see that the stochastic difference did not peak at the same place as the volatility spike. Instead, the bottom line in Figure 3.5, M1-M2, shows that the stochastic differences peaked in May 2008 at a value of about 50 and had numerous lows of about 50 throughout the year. The low levels are more consistent than the peaks, but this is only six months, or 5% of the test period.

Experience teaches us that we should not attempt to fine-tune these levels or bias our trading to expect the extension on the upside to be smaller than the downside. At some point, whatever causes these charts to be asymmetrical will change. Those readers familiar with trend-following systems may have noticed that during the 1990s, when there was a clear bull market in stocks, long positions would have been held longer and shown larger profits than short sales, making the performance noticeably asymmetrical. If we had decided to optimize only the long positions or bias our trading to the long side, the result would have been to hold the longs even longer and possibly not trade any shorts. That would have been a financial disaster after 2000, when prices headed down for more than three years. If we don't know when the next major economic cycle will start, using symmetrical trading rules is the safest approach.

FIGURE 3.4 14-day stochastic momentum of AMR and CAL, second half of 2007.

FIGURE 3.5 The stochastic difference (SD) of the AMR stochastic minus the CAL stochastic, shown with the AMR and CAL prices during the second half of 2008.

Exits.

When planning a trading strategy around these numbers, our nominal exit should always be at a stochastic difference of zero, indicating that the relationship between the two stocks has gone back to equilibrium. For some traders, that might be unnecessarily strict. We should also consider exiting shorts above zero and exiting longs below zero. This would cut profits short but a.s.sure us that we would safely exit more often.

There is always a temptation to hold a short position until the stochastic difference moves from the high entry to the low point, where we would reverse and enter a long position, for example, from a stochastic value of 80 down to 20. Profits would be much bigger and transaction costs less important. But that's not the way the market works. A relative distortion, as we recognize with the momentum indicator, is likely to return to near normal but has no reason to reverse. In our 6-month example, CAL tends to lead AMR, then fall back to normal, and then lead again. We would be exposing ourselves to very high risk unnecessarily if we waited for AMR to lead CAL in order to exit a long position.

Implicit Bias.

As we look at Figure 3.5 and consider the rules, we see that if we sell above 45 and buy below 45, there is only one short trade and four longs. If we choose 25, there would be a lot more trades on both sides, but that would force us to hold those trades with larger unrealized losses. Those are cla.s.sic trade-offs that we will consider later in the development process.

None of the strategy rules will consider asymmetrical parameters. There has always been a bias in the stock market because history has shown a steady increase in the average price of a stock or an index. One example of this bias is the traditional definition of a bull and bear market. A bear market begins when the DJIA turns down by 20%, and a bull market begins after the DJIA turns up by 20%. However, after a decline of 50%, a rally of 20% is actually a recovery of only 10% of the value lost in the downturn. Then the threshold can be twice as large to enter a bear market as a bull market, a definite bias toward the upside.

The 20082009 stock market decline of 50% points out that those upward biases may have provided small improvements during good times and large losses when they go wrong.

The rules in this strategy, and others given later in this book, will all use symmetrical thresholds. A short sale signal will occur when the stochastic difference moves above 40, and a buy on the first day that the stochastic falls below 40. Exits will initially be at retracements to zero.

TABLE 3.3 AMR-CAL trades based on the difference in stochastics, 2008.

Results for Stochastic Difference.

In tracking the performance, there are five trades during this period with a total return of $350.39, or $0.468 per share (as shown in Table 3.3). At that level of return, the method would still be very profitable after costs and slippage. However, this is a single example and not typical of a wide range of performance.

The list of trades also confirms the rules. In the first trade, on January 14, 2008, the stochastic values were both very positive, but CAL was much stronger than AMR, 82.2 compared with 41.8. The difference, AMR CAL, was 40.5, below the 40 threshold to trigger a buy of AMR and short sale of CAL. On February 4, 2008, the stochastic difference moved above zero, and the trade was exited for a loss. Four of the five trades during 2008 were triggered on CAL being stronger than AMR.

It shouldn't be surprising that there are profits because we picked our buy and sell thresholds off the charts and then just verified that those values generated profits. The advantage of this method is that the stochastic should adapt to many different price patterns, including changes in volatility. Remember that the intention is to find a method that would adapt to the more volatile period beginning June 2007. In this first test, the threshold values of 40 generated only five trades in a year. We would prefer to trade more often.

Different Position Sizes.

Notice that the sizes of the CAL positions are different from the nominal AMR position of 100 shares. That is because a volatility adjustment was used. As we go through this process, the size of the two positions will be an important way to control the risk and improve the chance for a profit. In this case, a rolling volatility measurement was used.

To find the volatility-adjusted position size, we begin by a.s.signing a fixed size to one leg. In the previous example, AMR always traded 100 shares. Next, we calculate the average true range (ATR) of each leg, measured over the same n days. For any day, the true range, TR, is the largest of the high minus the low, the high minus the previous low, and the previous close minus the low: The ATR is the average of the past n values of TR. Then, if the ATR of AMR was $0.50 and over the same period the ATR of CAL was $1.00, we would trade twice as many shares of AMR as we would CAL. It is important to remember that this is a critical step in trading two markets simultaneously. They must each have the same risk exposure. If you miss this step, there is no way to correct for it later. This will be discussed in more detail in the section "Alternative Methods for Measuring Volatility," later in this chapter.

Had we not adjusted the position size of each leg, the only alternative would have been to trade 100 shares each, with the results shown in Table 3.4. This method returns a total profit of $178.00 with $0.18 per share compared with the volatility-adjustment method of $350.89 and $0.47 per share. Of course, this is a very small test period, and the results could have greatly favored taking equal positions, especially if it was the CAL leg that was most often profitable. But the performance would have depended on the chance that the leg with higher volatility was the one generating profits, which is not good risk management. There is no subst.i.tute for volatility-adjusting each leg of the trade.

TABLE 3.4 AMR and CAL with equal position size.

Alternate Approach to Position Size.

The idea behind volatility-adjusted position sizes is that every trade should have an equal risk. That way you maximize diversification and are not making the unconscious decision that one trade is better than another.

In the previous process, we started by fixing the position size of one leg at 100 shares and then finding the number of shares in the other leg that caused risk to be equal. An alternative approach is to a.s.sign some arbitrary investment size to each stock, say, $10,000, and then: Calculate the average true range of the stock price over the past 20 days.

Divide the nominal investment size by the average true range.

The result is the number of shares needed to equalize the risk of the two stocks. These can be done for any number of stocks, and all will have been adjusted to the same risk. When you're done testing using these position sizes, the results can be scaled up or down by multiplying all stock positions by the same factor to reach a target volatility or an investment amount.