hero:Tech: The high cost of computation
– 3 min read

Tech: The high cost of computation

Or what we need to understand when discussing AI and what it can do.

Theory of Computation was the most feared course at the University where I studied Computer Science. I’ll happily admit that I had to take it thrice to pass. While I did agree at the time with most of my co-students; that the course felt too hard for “basic studies,”1 the things I learned there have turned out to be essential in my career.

One of the most subtle insights from the course is the big O notation. Or how the amount of computation for a given algorithm grows as the input size grows. To simplify, while the cost to run an algorithm might grow linearly to input size, it most likely grows exponentially or faster2.

Computation happens by moving electrons in physical hardware, which requires energy and – thus – costs money. In business terms, the cost for a given algorithm or solution grows like compound interest. There is always a point where scaling the solution or its capabilities is no longer economically feasible3. A point where adding more processing no longer costs a bit more, as you need a new dedicated nuclear reactor just to run the code. Or 100.

A very high-level illustration of exponential growth and the cut-off point of cost.

A very high-level illustration of exponential growth and the cut-off point of cost.

Typically, that is the point where we’ll still see some minor incremental updates to the solution and its capabilities. But even if the solution provider boasts they can make the solution 80% faster, that gain is quickly eaten by the exponential cost growth.

From theory to practice

Many commentators were disappointed with the Google Gemini pricing model and capabilities. And the disappointment is repeating with the neOpenAI’s and Microsoft’s new models. For some reason, even the giants and unicorns are hitting some invisible barrier for the tech.

At the same time, buying more AI processing from a vendor is nearly impossible after you hit a certain limit.

These hints point towards a conclusion: we have now found the limits of the 4th wave of AI - also known as the generative AI. We’ll still see incremental upgrades like we saw between 2017 and 2021. But it will take some time before we see such transformative revolutions as the ChatGPT was.

If the hypothesis is correct, this suggests that we should adjust our tech roadmaps and strategies to:

  1. Pivot from discovery and understanding the new technology; to build tangible MVPs based on traditional and current generative AI technology
  2. Expect the offerings of the different vendors to stabilize – the advantages of one provider over another will likely stop to exist or be very transient.
  3. Focus on security and risk management of AI solutions as the tech becomes more prevalent. The more generally used tech is, the more attackers will try to misuse it.

Footnotes

  1. Finnish “basic studies” of the time translate roughly to “College level ” or the “bachelor level.” ↩

  2. EDIT: As Antti Rautiainen pointed out in Threads, some neural network algorithms are linear (https://www.threads.net/@anttiraut/post/C2_zuUXt09t). While this is true, we rarely run an algorithm in an isolated and dedicated environment built to run the algorithm. Thus the cost of computation for the whole system is still exponential. In hindsight, I should have mentioned this in the original post, or footnoted it. ↩

  3. Many scientists, including mathematicians, think this means cryptocurrencies are an eventual Ponzi scheme. When the currency hits the point where the computation required for a transaction becomes expensive enough, it becomes a Ponzi. ↩

#tech #AI #theory of computation #big O notation #strategy