Exponential Growth

Exponential growth is fast. That’s why everybody gets so excited about things that increase exponentially. But exponential functions have some weird properties that make them difficult to spot and to predict. This post will shed light on these peculiar functions.

What is an exponential function?

When people talk about exponential growth they often mean runaway growth. Something that not only increases year after year but the increase itself increases and the increase of the increase increases and so on until infinity. The image in their heads will look like the proverbial hockey-stick:

An exponential function f that remains flat for some time until it takes off steeply and a polynomial function g that is steep from the beginning but fails to gain as much as the exponential

But not everything that looks like a hockey-stick is exponential. Curve f (the blue one above) is ($math 1.69^x$) — but curve g (the red one) is not. What’s the difference?

g is a polynomial function. In this case it’s $math x^2$. Polynomial functions still grow fast but the growth is limited at some point. Take $math x^2$ for example. Its growth rate $math \frac{\partial g}{\partial x}$ (get out your calculus) is $math 2x$, the growth of its growth is 2 and the growth of the growth of its growth is 0.

However, the growth of the growth of the growth etc. of an exponential function will never become 0. So even exponential functions that start out very flat will become steeper and steeper until they overtake every single polynomial function if you wait long enough. Wow. [1] [2]

Comparing an exponential function that intersects a polynomial and get steep quickly

Exponential examples

When something doubles every 18 months (Moore’s law) or when you are promised a single grain of wheat on the first field of a chess board; double that on the second field and so on until the 64th field. (Wheat and chessboard problem) Those are all exponential functions.

But also when something grows by 7% per year (Compound interest) – which may not sound like much but it means a 10-fold increase after 35 years.

Exponentially difficult

Exponential functions tend to not look like much for some time until they take off faster than anything else could. That’s what makes them so difficult to spot correctly. Let’s look at an example.

Say your app has a single user and all you wish for is exponential user growth because that’s what the cool kids have. You are audacious, so you imagine a 30% increase each month. You do the math and get 543 users after 2 years ($math 1.3^{24}$). Sounds good!

But 6 months into the thought-experiment you look at the number of users again and it looks like this:

Users after 6 months - at best linear growth

You only have 5 users. At this rate (+4 users in 6 months) you will only get to 17 users after 2 years. That’s disappointing. After another 6 months you look at the statistics again:

Users after 12 months. Maybe linear growth?

Even though growth has picked up a bit this still looks bleak. You’re at 23 users and at this rate after 2 years you can only expect to have 45 users.

The second half of the chessboard

But then something happens. 18 months into the thought-experiment you suddenly hit 100 users. 3 months later it’s 250 and after 2 years you have 543. What happened?

Users after 24 months. Clearly a hockey-stick shaped function

This is typical for exponential functions – and especially for functions that less than double at each timestep. For some time it looks as if nothing was happening – at best a slight linear increase. And then – all of a sudden – things take off.

In the story about the grain of wheat on a chessboard this is the second half of the chessboard. The first half contains a manageable 280 tonnes of wheat – an amount you could buy for about 50,000 USD. The second half holds over 1,600 (one-thousand-six-hundred!) times the global wheat production in 2014. [3]

Another example is the increase in energy density of batteries. Let’s say it’s true that the density increases by 7% per year. For a few years this looks like marginal improvements. But after 10 years the density has doubled and you can drive 1000km with your electric car.

Extrapolation

The lesson to be learned is that even in a perfect example it’s hard to spot an exponential function in the beginning and easy to despair when the performance of an app or a fund or a company looks like it’s flatlining. Even more so when the data is noisy (imagine no constant increase in users but random variation with some ups and downs).

As everything that grows by a certain percentage per year is an exponential function and most things change by a percentage [4] it’s obvious why people are so bad at predicting things. Even when growth looks like it’s flatlining it might be the very beginning of exponential growth. Just hang in there.

 

[1] One limitation though: We’re looking at exponential functions with a base greater than 1. Nothing interesting happens for $math 1^x$ (Wolfram Alpha) and $math a^x$ for $math a < 1$ (Wolfram Alpha) is a whole different subject.

[2] Take a look at this answer on StackExchange for a more mathematical examination.

[3] Things doubling sounds familiar. And so the chessboard story often is used as a metaphor for computing power doubling every 18 months and that we now witness the transition from the first half (about 50-60 years of transistors) to the second half of the chessboard where things move unexpectedly and unimaginably fast.

[4] In another post I’ll look at why most things only grow exponentially until a damping factor kicks in (sigmoid function; in the end often due to physical limitations) which gives people the opportunity to first underestimate the exponential function and then underestimate the damping which pretty much explains hype cycles.

Learn to work with data at databriefing.com

Read more machine learning articles

Want more posts like this one?

I send out emails when I add new posts.

Or follow me on twitter