We live in an era where being a tech titan makes you a household name. They’re not only awarded with unimaginable wealth, but also fame and celebrity.
But for every Jobs, Zuckerberg, and Altman, we have Wozniak, Saverin, and Sutskever. The people whose life’s work isn’t rooted in marketing, but progress. The search for something new and the development of trailblazing technologies. The people who achieve greatness through what they create, instead of who they are.
Today, I’ll tell you the story of Alex Krizhevsky. You may not know him but trust me when I say this: he’s one of the most important figures in technology.
A Quiet Life: Alex’s Early Years
I’ve done extensive research. I looked everywhere. From what I can tell, there’s only one thing we know about Alex Krizhevsky’s early life:
He was born in Ukraine but raised in Canada.
After that, absolute silence. The second earliest information we have about him is that, in his undergraduate years, he was already being mentored by Geoffrey Hinton: one of the most respected minds in deep learning and now a thought leader in AI ethics. Keep that name in mind, because he’s a central figure in Alex’s story.
I will say this about him though: the fact that we know so little of his early years is indicative of the life he ended up living. Unlike his peers, who we’ll get to in a minute, all Krizhevsky wants is to contribute and influence the next few years of machine learning.
And that he most certainly did.
Changing Deep Learning Forever: The Development of AlexNet
I can count on one hand the important milestones in the field of deep learning. Off the top of my head, we have Weizenbaum’s ELIZA chatbot; Rumelhart, Hinton, and Williams’ backpropagation, and Vaswani’s Attention Is All You Need.
When it comes to the applications of deep learning in computer vision, I’d argue that the most significant milestone of the past decade came from Krizhevsky, Hinton, and some guy called Ilya Sutskever. This is now known as AlexNet, which can recognize and analyze intricate patterns within large image datasets.
This technology may seem commonplace now, but it’s only so because of AlexNet.
But, where did it actually come from? Who had the idea first? And, most importantly, why was it made in the first place?
Where Did AlexNet Come From?
In 2010, Geoffrey Hinton introduced Alex to one of his other mentees: Sutskever. If you don’t know who that is, you really should. He’s the brains behind OpenAI, the company that made ChatGPT and DALL-E. Together, the trio made the decision to enter the ImageNet competition: an annual competition attended by the best minds in AI to test the accuracy of their systems in identifying objects without errors.
Krizhevsky’s algorithm hinges on a discovery made by Hinton himself a few years back, which was a machine that runs on GPUs rather than CPUs. From there, he figured out that he could significantly increase the processing speed of Hinton’s machine if he layered neural networks on top of each other.
The result was a success, but ImageNet wasn’t on Alex’s mind until Sutskever had an epiphany: the best use-case of the improved machine is by entering the ImageNet competition.
So, they registered. They set up. They waited. And they won — quite handily, in fact.
The reason why AlexNet is seen as a monumental shift in deep learning and a step closer to AGI wasn’t only that they successfully scaled their system to deal with ImageNet, it’s because they beat their closest competitor with a 10.8% margin.
What Happened Next?
It shouldn’t come as a surprise that companies in Silicon Valley lined up to acquire these trio and their system. The winner of the bidding war? Google.
AlexNet became a central part of Google Photos and some of their other technologies. Krizhevsky was particularly invested in the development of their autonomous cars but, as we all know, it’s already 2024 and that tech is yet to pan out.
After a few years, they all left Google to pursue different goals. In particular:
- Alex Krizhevsky left Google because he “lost interest in the work.” His next venture was Dessa, a company aimed at developing business-facing AI.
- Geoffrey Hinton left Google because he deeply regrets his role in the advancement of artificial intelligence. He’s now a thought leader in AI governance.
- Ilya Sutskever left Google because he was recruited by Sam Altman for his little startup called OpenAI. He’s now their Chief Scientist.
Where Did AlexNet Come From?
As for AlexNet, it’s now known as an important milestone in deep learning and AI. It’s far from the best in its segment today, but it serves as the foundation of many of our modern standards in computer vision such as ResNet and YOLO.
More than the results, the value of AlexNet is its insights. Everyone wanted to improve upon the model and, eventually, they did. Because of this, we now have facial recognition in our pockets, advanced medical imaging, and lightning fast GPUs.
Other Important Contributions
- Dropout: A regularization technique for neural networks. It involves randomly dropping some neurons during training to prevent overfitting (capturing random unimportant elements) and enhance its efficiency. It has since become a standard in machine learning.
- CIFAR-100: A large dataset containing 80 million handpicked tiny images.
- Robotic Hand-Eye Coordination: Contributed to the grasping mechanism of current-day robot technology.
- Chauffeurnet: A significant advancement in the field of autonomous driving.
Where Is He Now?
In typical Alex fashion, mentions of him online are so few and far between that it’s hard to pinpoint where he actually is now.
From what I can gather, he’s now a partner at a Silicon Valley-based venture capital firm called Two Bear Capital. Apart from that, he recently released a new scientific paper on robotic grasping, his first in almost two years.
The Bottom Line
I’ll admit, I still had one question left unanswered after reading everything I can about Krizhevsky:
Where is his place in AI history?
As I write this last section, I wonder if anyone will ever remember Alex a few years from now. AlexNet is now obsolete, and he never did join a large company like Ilya Sutskever. Unlike his mentor, he didn’t have enough time to establish himself in the field before fading into obscurity.
That said, I believe that Alex Krizhevsky should be given his laurels — more so than some of his contemporaries.
Our technological landscape wouldn’t be the same without him. No ChatGPT, no accessible triple AAA games, no advanced medical imaging. His work made all those, and so much more, possible.
Alex Krizhevsky never did become an industry leader. Perhaps, he never wanted to be. But he is a visionary, and that’s enough for him.