There’s a story about The Beatles’ Yesterday that I love. From Wikipedia (condensed):
[T]he entire melody came to McCartney in a dream one night. Upon waking, he hurried to a piano and played the tune to avoid forgetting it. Initially he was concerned, though, that he had subconsciously plagiarised someone else’s work; as he put it: “For about a month I went round to people in the music business and asked them whether they had ever heard it before. Eventually it became like handing something in to the police. I thought if no one claimed it after a few weeks then I could have it.”
Upon being convinced that he had not copied the melody, McCartney began writing accompanying lyrics. As John Lennon and McCartney were known to do at the time, they used placeholder words as temporary working lyrics until something more suitable could be written. The working title was “Scrambled Eggs.”
I’ve been having my own version of this over the last few weeks, with an idea that is clear and compelling enough to me (though not nearly so lyrically beautiful as Yesterday) that I am sure I must have read it somewhere, though I can’t find it. This note is my version of handing it around.
The idea in question has to do with the Productivity Paradox. In brief, the “paradox” is that productivity didn’t accelerate, and actually decelerated, during the time when information technology (IT) was attracting enormous investment and permeating the economy on the promise that it would boost productivity. As the economist Robert Solow memorably put it, “You can see the computer age everywhere but in the productivity statistics.”
The missing boom
Output per hour — log scale
Growth rate by decade
| Period | Growth per year | Level at period end |
|---|---|---|
| 1890–1900 | 1.2% | $6.60 |
| 1900–1910 | 1.7% | $7.80 |
| 1910–1920 | 1.5% | $9.09 |
| 1920–1930 | 2.8% | $12.04 |
| 1930–1940 | 3.0% | $16.17 |
| 1940–1950 | 2.8% | $21.35 |
| 1950–1960 | 2.9% | $28.40 |
| 1960–1970 | 2.7% | $37.01 |
| 1970–1980 | 1.4% | $42.54 |
| 1980–1990 | 1.5% | $49.15 |
| 1990–2000 | 1.8% | $58.85 |
| 2000–2010 | 2.2% | $73.30 |
| 2010–2020 | 1.1% | $82.11 |
| 2020–2024 | 1.0% | $85.45 |
I’ve thought about this phenomenon often over the last decade. It was relevant to my work in strategy at Microsoft, and sometimes made things difficult. I was asked a few times to weigh in on corporate messaging or speeches executives were making, and sometimes they’d include statements about how important computing had been to economic growth and global prosperity over the past 50 years. I’d always have to write back some version of “that’s not what the data says.”
Now, this question is more than rhetorical: we are investing hundreds of billions each year in developing AI. Investment at that scale needs to generate growth across the economy to pay off, and that in turn will require significant productivity growth. If the past half-century of IT didn’t deliver it, why should we expect that AI will?
There are hypotheses for explaining the paradox: the measurements are wrong, IT helps companies capture more market share but doesn’t actually increase overall productivity, IT is flashy but doesn’t actually improve productivity that much, IT does improve productivity but only if companies make other major adaptations to take advantage of it (and most don’t), there are lags in the impact of IT so the effects show up late, etc., etc.
All of these have some grounding in data; none of them is satisfying. IT is simply such a large portion of the economy and of the budget of every company that it’s implausible that they’ve all kept buying more and more for fifty years without seeing any benefit.
Investment without acceleration
IT investment, share of GDP
Productivity growth by decade
| Period | IT share at period end | Productivity growth per year |
|---|---|---|
| 1950–1960 | 0.9% | 2.9% |
| 1960–1970 | 1.6% | 2.7% |
| 1970–1980 | 2.4% | 1.4% |
| 1980–1990 | 2.9% | 1.5% |
| 1990–2000 | 4.4% | 1.8% |
| 2000–2010 | 3.5% | 2.2% |
| 2010–2020 | 4.1% | 1.1% |
| 2020–2024 | 4.5% | 1.0% |
The mental test is: what would happen to a company today if they wound their computing technology back to what was available in 1980, 2000, or even 2020? They wouldn’t survive the year. There are explanations that account for that—e.g., maybe IT helps competitors take business from one another but doesn’t actually produce new value?—but to me they sound like the epicycles early astronomers used to make the data fit their belief.
Here’s the alternate hypothesis I’ve been wondering about: the gains were real, but swallowed by a countervailing force. So what we see as decline is actually significant gain measured against a declining baseline. And the countervailing force I propose is complexity. Over the last fifty years, the economy has become increasingly global, filled with byzantine financial instruments, tortuous supply chains, sprawling multi-national corporations, and technologies that require deep study to even partially understand. It’s a modern lament that science has advanced to a state where it takes the focused effort of a major portion of a human lifetime to understand any field, let alone make any contribution; it is now beyond human capacity for a polymath like, say, Benjamin Franklin to see across a swathe of human knowledge and add something. We face the same challenge with our economy.
All of that presents challenges to businesses and policymakers:
- Managers of companies have workforces in the tens or hundreds of thousands to manage, spread across dozens of business units in just as many jurisdictions.
- Workers have to collaborate with coworkers across all of that.
- Anyone trying to model the future (e.g., for investing or regulation) of even a single company, let alone an industry, let alone the entire global economy, has to grapple with all of the complex interactions and trends that are at play.
At some point, there are limits to human cognition—what we can remember, what we can conceptualize, what we can reason through—and we can get in over our heads. That happens in a thousand small ways each day, like losing track of an email, or it can happen in a major crash: a defining cause of the financial crisis of 2008 was that the complexity of the financial instruments that caused it obscured an impending crash from regulators, credit rating agencies, ordinary people, and even the bankers creating the risk (poor incentives didn’t help). Minor or major, complexity becomes a drag on productivity as we labor to understand the context surrounding the tasks in front of us.
Over the course of our history, we’ve invented ways to extend our cognitive capacity, beginning with language and writing but extending to libraries, abaci, the post office. When we can store and retrieve information, analyze it more efficiently, or share it with one another more fluidly, we can accomplish more than our individual minds and memories allow.
As we develop these cognition-expanding technologies, the new possibilities they create transform our economy. Merchants develop bookkeeping to keep track of their accounts, but once they have it, they can expand their business to far-flung trade routes. Technology and complexity are in a cycle: we create tools to manage complexity, they give us the space to increase that complexity, and eventually we make better tools, which again open more space. Sometimes the tools are ahead of the complexity (as with the upturn in the late 90s through early 2000s), and sometimes it’s the other way around.
Under this hypothesis, what we see in the data is not that IT produced no productivity gains, or that these have been somehow hidden or concentrated. It’s that over the period of IT expansion, our economy grew in its complexity and IT allowed us to both stay on top of it and drive it forward. To sustain growth in a planetary economy took every spreadsheet, every video call, every database; we couldn’t have gotten to this point with typewriters and filing cabinets. To look at that achievement and see missing productivity growth is a mistake. Instead, we should see an economy that would not have been possible otherwise.
The implication for the emerging AI era is that we need new and better tools to keep going. The first generation of computerized spreadsheets came on the market in the late 1970s and early 1980s, software like VisiCalc (1979), Microsoft Multiplan (1982), Lotus 1-2-3 (1983). They would have been mind-blowing to recordkeepers of just a few decades earlier, who kept their data in physical ledgers. Today, they aren’t nearly up to the task of managing the terabytes of data that modern businesses process regularly. If we want continued economic growth, we need more ways to tame the complexity that growth produces. It may be that AI will lead to a dramatic increase in productivity and growth, and I hope it will. But it may simply be necessary for the growth we have to continue.
To be clear, this is a hypothesis in the most complete sense. There’s no empirical work here and there might be a simple and obvious reason why it’s wrong. But, if that’s out there, I haven’t found it.