Moore’s Law, one of the fundamental laws indicating the exponential progress in the tech industry, especially electronic engineering, has been slowing down lately (since 2005, to be more precise), and has led many in this sector to believe this law to no longer hold true. That was, until Artificial Intelligence joined the arena! Since then, the game changed, and Moore’s law is slowly being revived. Gordon Moore, one of the co-founders of Intel, noticed in 1965 that the number of transistors per square inch on integrated circuits had doubled yearly while, at almost the same speed, costs were halved. He therefore predicted that this trend would carry on into the foreseeable future.
I know, some of us may think “That’s all? Well, how about I notice it’s raining and make my own law saying that it’s gonna rain every year??”. But Moore’s law is not just a vague guess: it also agrees with a technology scaling rule called Dennard Scaling which states that “as transistors are reduced in size, their power density remains constant”. Erm…what? Okay, translated into English, this means: The number of transistors per unit area will increase. Power requirement per unit area of chip (now having more transistors) will remain constant, while voltage and current will scale downward with length. And the predictions did come true: the electronics industry has been booming!
Transistors have not only become smaller, but also faster and more energy-efficient such that a chip now offers at least twice the performance at roughly the same dollar and power budgets.
Moore’s law held true…until it didn’t.
Why is Moore’s Law dying?
As Chien-Ping Lu from NovuMind Inc so concisely put it in his paper:
The turning point happened in 2005, when the transistors, while continuing to double in numbers, were neither faster nor more energy-efficient at the same rates as before.
Here’s why: The 21st century is when the electronics industry was about bringing the tech to the people. Businesses boomed, Samsung and Qualcomm being examples for mobile phones and processors respectively. But in this day and age, competitors in the tech industry are very much real! As such, the hardware provided by a company had to carry its legacy: it had to meet the specs, give better user experience with each new version, and could therefore not be aggressively modified to save energy.
We all noticed the increase in power requirements.
There was a breakdown of the Dennard scaling aspect, so there has from then on been a deviation from Moore’s law’s trend. Enter Artificial Intelligence, and how it fixes the situation.
How A.I. is revamping Moore’s Law
We all know how the media and the film industry are overhyping AI with androids and over-intelligent systems. Some computer pioneers, Alan Turing (you may want to watch The Imitation Game to appreciate the legend he is) at the forefront, did set off on projects with a view to making machines that think. Turing, however, did realise that this would be abysmally difficult, and in 1950 proposed:
Instead of trying to produce a programme to simulate the adult mind, why not rather try to produce one which simulates the child’s? If this were then subjected to an appropriate course of education, one would obtain the adult brain.
This idea grew on to become Deep Learning. Fast forward to 2018: we have, and are still gathering, massive amounts of data. We have and are still developing more and more advanced algorithms (Generative Adversarial Networks and Capsule Networks stand as strong examples.) But do we have the hardware to crunch all those calculations within reasonable time? And if we do, can it be done without having all those GPUs cause another global warming on their own by literally heating up from all the processing?
Right there, Artificial Intelligence is imposing a constraint: keep the power constant or decrease it, but increase performance… doesn’t that sound a bit familiar to some scaling rule we have just seen? Exactly, by forcing the tech industry to come up with new processors which can perform more calculations per unit time, while maintaining power consumption and price, Artificial Intelligence is imposing Dennard Scaling again, and hence forcing Moore’s Law back to life!
If you liked this article, please give it an applause so that others can find it as well. It is always good to help each other in the data science community :)
Hans A. Gunnoo is a Data Scientist, having begun his career in Electronic Engineering and later specialising in Machine Learning. Also makes contributions to Open-Source AI projects and blogs about the latest trends in the data science field in his spare time.
No comments:
Post a Comment