The Open Championship, aka “The British Open,” is the oldest of golf’s four major tournaments and the final major of the golf season. The Open is played on coastal links courses with no trees but plenty of wind. Over the last 11 years, 11 different golfers have won the Open. That is not because of the rotating courses, capricious wind, deep bunkers, and gorse-bush rough. It is because they are playing golf.

A total of 222 golfers have won at least one of the four majors (Open, Masters, U.S. Open, and PGA Championship). Of these major winners, 140 (63%) never won another major afterward.

Even among the best golfers, luck is endemic. There is considerable happenstance in gusts of wind and in fortunate and unfortunate bounces. Sometimes a ball lands on a bank of grass and sticks; sometimes it rolls into a lake or sand trap. Sometimes a ball whistles through a tree; sometimes it bounces off a branch. Sometimes a branch ricochet puts the ball back on the fairway, where the grass is cut short and the ball can be played cleanly; sometimes the ball bounces into foot-high grass.

In a 2016 tournament, Phil Mickelson hit an errant drive. The ball caromed off the head of a spectator on one side of the fairway and landed in the rough on the other side of the fairway. Mickelson told the spectator, “If your head was a touch softer, I’d be in the fairway.”

The golfer who has the best score in the first round is not necessarily going to win the tournament. A golfer who wins a major is not necessarily the best golfer in the world and is unlikely to win the next major he plays in. Shane Lowry probably won’t win the Masters this April.

In any competition including academic tests, athletic events, and company management where there is an element of luck that causes performances to be an imperfect measure of ability, there is an important difference between competitions among people with high ability and competitions among people of lesser ability. If four work friends play a round of golf and one player is much better than the others, the winner is determined mostly by ability. If four of the top golfers in the world play a round of golf, the winner is determined mostly by luck. This is the paradox of luck and skill: the more skilled the competitors are, the more the outcome is determined by luck.

Let’s look at some data on this. The Open begins with 156 golfers playing two rounds (36 holes) on a Thursday and Friday in mid-July. The top 70 (including any ties) play two more rounds on Saturday and Sunday. Because of ties, 79 players made the cut in 2018, and 73 players made the cut in 2019. Only 33 players made the cut both years—which is, by itself, evidence of the importance of luck in golf. Of those who made the cut in 2018, 58% did not play well enough to make the cut in 2019. What happened? Luck happened.

Figure 1 shows the 2018 and 2019 Open scores for the 33 golfers who made the cut both years. The correlation between the 2018 and 2019 Open scores is 0.07. There is a positive correlation, but it is loose.

Another way to grasp the importance of luck in golf is to compare scores from one round to the next. If performances depended entirely on ability, with luck playing no role, some golfers would get a 68 every round while other golfers would get a 74 every time. If, at the other extreme, performances were all luck, there would be no correlation between scores from one round to the next. A golfer with a 68 in one round would be no more likely to get a 68 in the next round than would a golfer who shot 74 in the earlier round.

To investigate which scenario is closer to the truth, Figure 2 compares each player’s third- and fourth-round scores in the 2019 Open. Those who did well in round 3 tended, if anything, to do slightly worse in round 4:

It’s not just the Open, with its revolving courses and fickle environment. The Masters Tournament is played on the same course (Augusta) every year and luck is just as important there as it is at the Open. For the 40 golfers who made the cut in 2018 and 2019, the correlation between their 2018 and 2019 scores was 0.18. The correlation between the third-round and fourth-round scores in 2019 was 0.05.

When you watch a golf tournament, remember that even the best golfers in the world are subject to the paradox of luck and skill. Wish your favorite golfer good luck because he surely needs it.

Oh, did I mention that the paradox of luck and skill holds for any competition, including academic tests, athletic events, investment performance, and company management?

Also by Gary Smith:

We see the pattern!—but is it real? Patterns are not always a source of information. Often, they are a meaningless coincidence like the 7-11 babies this summer.

Podcasts: Catch Gary Smith discussing with Bradley Center director Robert J. Marks what AI can and can’t do:

Why was IBM Watson a flop in medicine? Last year, the IBM Health Initiative laid off a number of people, seemingly due to market disillusionment with the product.

Why an AI pioneer thinks Watson is a “fraud.” The famous Jeopardy contest in 2011 worked around the fact that Watson could not grasp the meaning of anything.

Can AI combat misleading medical research? No, because AI doesn’t address the “Texas Sharpshooter Fallacies” that produce the bad data.

AI delusions: A statistics expert sets us straight. We learn why Watson’s programmers did not want certain Jeopardy questions asked.

and

The US 2016 election: Why Big Data failed. Economics professor Gary Smith sheds light on the surprise result.