Technology is nothing if not iterative, evolutionary and continuously advancing.
In the world of fast-changing technology, the startup Neural Magic has raised a $15 million seed round to break a persistent myth: that you need specialized hardware accelerators to run machine learning algorithms requiring heavy-duty computation power.
The round was led by Comcast Ventures, with participation from NEA, Andreessen Horowitz, Pillar VC and Amdocs.
The Somerville startup was founded by MIT researcher Dr. Alex Matveev and Nir Shavit, an MIT professor in the department of electrical engineering and computer science. As Shavit put it, Neural Magic is pioneering a “No-Hardware AI” model with its software-only solution. The software processes deep learning workloads on central processing units (CPU), eliminating the need for the specialized hardware usually required for that kind of computation.
That also means the startup is taking on industry heavyweights—NVIDIA, AMD, Asus and more.
"The new kind of computation requires a new kind of hardware like specialized accelerators made by companies like NVIDIA," Shavit said. "But you don't need these specialized accelerators, because it's a system engineering problem."
Neural Magic sells software that can be installed on a CPU to bring its performance to the same level of graphics processing units (GPU).
"The networks being run required a lot of computational power, because we didn't really know how to write algorithms better," Shavit said. "And as we are learning how to write better algorithms, the computational power needed is going down. That's why we think software on CPUs will take over the [machine learning] world."
But why address this issue now?
As Shavit put it, even though GPUs can run algorithms more quickly than regular CPUs can, their limited memory is a persistent problem. When the model and data become too big to fit in the memory, the process slows down. For example, a picture in 4K resolution would have to be compressed. Or instead of fitting 30 frames per video, a program could only fit in three.
"People are compromising on accuracy inside the limited memory of the accelerators, and the way the accelerator manufacturers plan to solve the problem is to connect a lot of devices together," Shavit said.
But unnecessarily so.
Shavit and his team believe that machine learning will eventually become a software play.
The one-year-old upstart, with $20 million in its coffers, plans to sell software subscriptions—downloadable products priced based on the computational capacity needed. The first use cases? Shavit imagines Neural Magic's "Inference Engine" system being used for real-time recommendation systems and image recognition applications on B2C websites.
"When you want to run machine learning algorithms, instead of needing to buy a GPU or rent it from Amazon Web Services, you download Neural Magic and start running," said chief commercial officer Bryan House. "We are similar to VMWare—we are solving a problem no one thought we could solve, and once it's done, it seems easy."
The money from this round will help the startup build out its team. Shavit plans to hire machine learning engineers, software engineers and sales and marketing professionals.
