The deflated Sharpe ratio: what a 2.0 Sharpe is worth after 1,000 tries

A Sharpe ratio of 1.0 sounds like a real strategy. It is, if it came from one strategy tested once. It is not, if it came from the best of a hundred variants, because the best of a hundred draws from a zero-edge distribution is a high number by construction.

The deflated Sharpe ratio (DSR) asks a sharper question: given the observed Sharpe, the sample length, the skew and kurtosis of the returns, and the number of independent trials, what is the probability that the true Sharpe is above zero?

The intuition

Under the null of no skill, the maximum Sharpe across N independent strategies has a known expected value. That expected maximum rises with N. The DSR compares your observed Sharpe against that benchmark and returns the probability that skill, not selection, produced the result.

With N = 1 the benchmark is zero and the DSR reduces to an ordinary significance test. With N = 1,000 the benchmark is high enough that a respectable-looking Sharpe can be entirely unremarkable.

The numbers

Five years of daily returns, normal for now, so skew is zero and kurtosis is three. Each row is the probability that the true Sharpe exceeds zero, at four trial counts.

• Annual Sharpe 0.5 — N=1: 0.868, N=10: 0.324, N=100: 0.079, N=1,000: 0.016.
• Annual Sharpe 1.0 — N=1: 0.987, N=10: 0.745, N=100: 0.383, N=1,000: 0.153.
• Annual Sharpe 1.5 — N=1: 1.000, N=10: 0.962, N=100: 0.792, N=1,000: 0.536.
• Annual Sharpe 2.0 — N=1: 1.000, N=10: 0.998, N=100: 0.973, N=1,000: 0.884.

Read the second row. A Sharpe of 1.0 over five years is a 98.7% probability of real skill if you tested one strategy. Test a hundred and it is 38.3%. Test a thousand and it is 15.3% — worse than a coin flip. The strategy did not change. The search did.

Why 2.0 still survives

A Sharpe of 2.0 is strong enough to clear even a thousand trials, at 88.4%. That is the useful part of the table: it tells you how strong a result has to be before a large search can be forgiven. At five years of daily data, roughly 2.0 is the point where a broad search stops being decisive.

It also explains why a Sharpe of 2.0 on 18 months proves so little. The DSR already accounts for sample length; the shorter the record, the lower the probability, at every trial count.

Fat tails and negative skew make it worse

Real strategy returns are not normal. Trend-following has positive skew; carry and short-volatility strategies have negative skew and fat tails. Those are the ones that blow up, and the DSR penalises them correctly.

• Annual Sharpe 2.0, skew -1, kurtosis 6 — N=1: 1.000, N=10: 0.995, N=100: 0.950, N=1,000: 0.821.

Compare with 0.884 under normality. The negative skew costs about six points of confidence at N = 1,000. That is the right direction: a strategy that makes small amounts often and loses a lot rarely should need more evidence, not less.

Time is the other lever

Same Sharpe of 1.0, same 100 trials, three sample lengths:

• 2 years: 0.132.
• 5 years: 0.383.
• 10 years: 0.735.

The only honest way to beat a large trial count is more independent data. More parameters, more indicators, or more cleverness do not move this number. Ten years of the same strategy does.

The practical rule

Count your trials. Not the ones you showed anyone — all of them, including the ones you deleted. Feed that count, the sample length, and the return moments into the DSR. Treat anything below 0.95 as unproven, and remember that 0.95 is a convention, not a law.

If you cannot state N honestly, the DSR is not computable, and the raw Sharpe is not evidence. That is the same failure the seven checks describe: the number is only as good as the search that produced it.

For the selection question rather than the significance question, CSCV and PBO measure whether the winner is the luckiest candidate. The two methods answer different halves of the same problem, and a serious validation uses both.

The Sharpe ratio measures the strategy. The deflated Sharpe ratio measures the search that produced it.

How these numbers were produced

These figures are closed-form evaluations of the deflated Sharpe ratio at the stated parameters: sample length in trading days, annual Sharpe converted to daily, skew and kurtosis as noted, and N independent trials. No simulation is involved, so the numbers are exact for the inputs shown.

Questions

What is the deflated Sharpe ratio?

A correction to the Sharpe ratio that accounts for the number of strategies tested, the sample length, and the skew and kurtosis of the returns. It returns the probability that the true Sharpe is above zero.

How much does the trial count matter?

Enormously. A Sharpe of 1.0 over five years of daily data gives 0.987 at one trial, 0.383 at 100 trials, and 0.153 at 1,000 trials. The strategy is the same; only the search changed.

What Sharpe survives a thousand trials?

At five years of daily data, about 2.0. It returns 0.884 at N=1,000. Below that, a large search can explain the result without any skill at all.

Why do skew and kurtosis change the answer?

Negative skew and fat tails make the Sharpe less reliable. At Sharpe 2.0 over five years with N=1,000, skew -1 and kurtosis 6 reduce the probability from 0.884 to 0.821.

How do I raise the deflated Sharpe ratio?

Add independent data, not parameters. At Sharpe 1.0 and N=100, two years gives 0.132, five years 0.383, and ten years 0.735.