Deep learning, a branch of artificial intelligence, curates social media and helps serve Google search results. In the near future, deep learning may be able to check your vital organs or even regulate your thermostat. The researchers at MIT recently developed a system which could introduce deep learning neural networks to newer and smaller devices, like the little computer chip in household appliances and the numerous other objects on the Internet of Things (IoT).
MCUNet, as the system is called, designs portable little neural networks which deliver never seen before speed and precision for deep learning on IoT gadgets, regardless of their processing power and restricted memory. This technology could ease the growth and development of the IoT universe, while also preserving energy and advancing the data security.
The details of this research will be presented at the Conference on Neural Information Processing Systems holding next month. The lead author is Ji Lin, a PhD student in Song Han’s lab in the Department of Electrical Engineering and Computer Science at MIT. Some of the co-authors include Han and Yujun Lin of MIT, Wei-Ming Chen of MIT and National University Taiwan, and John Cohn and Chuang Gan of the MIT-IBM Watson AI Lab.
The Internet of Things
The IoT was created in the early 1980s. Some students at Carnegie Mellon University, including Mike Kazar ‘’78 connected a Coca-Cola machine to the internet. The motivation of the group wasn’t innovation or to invent anything, it was simply laziness. They wanted to confirm if the machine was stocked before making their way to purchase it. It unintentionally became the world’s first internet-connected appliance. Kazar, now an engineer at Microsoft said, “This was pretty much treated as the punchline of a joke. No one expected billions of devices on the internet.”
Since then, common devices have become connected into the ever-growing IoT. This is inclusive of everything ranging from wearable heart monitors to even smart fridges that could say when you’re running out of milk. IoT devices usually run on microcontrollers— simple computer chips that have no operating system, limited processing power and memory. Deep learning which involves pattern recognition tasks are tricky to operate locally on IoT devices. For complicated analysis, data collected from the IoT are usually sent to the cloud, making it susceptible to hacking.
Han said, “How do we deploy neural nets directly on these tiny devices? It’s a new research area that’s getting very hot. Companies like Google and ARM are all working in this direction”, including Han.
Using MCUNet, Han and his group codesigned two components required for “tiny deep learning”, which is the operation of neural networks in microcontrollers. One of these components is TinyEngine, an inference engine that regulates resource management, very similar to an operating system. TinyEngine is optimized to operate a specific neural network structure, which is chosen by the other MCUNet component: TinyNAS, a neural architecture search algorithm.
System-algorithm codesign
The design of a deep network for microcontrollers is not easy at all. Existing neural architecture search methods begin with a large pool of likely network structures based on a predefined template, then they slowly search for the one with high accuracy and a low cost. While this method is effective, it’s not very efficient. Lin says, “It can work pretty well for GPUs or smartphones. But it’s been difficult to directly apply these techniques to tiny microcontrollers, because they are too small.”
So, Lin developed TinyNAS, a neural architecture search technique that makes custom-sized networks. He says, “We have a lot of microcontrollers that come with different power capacities and different memory sizes. So we developed the algorithm to optimize the search space for different microcontrollers”. TinyNAS’s customized nature means that it can create portable neural networks with the best possible execution for a particular microcontroller—with no needless parameters. “Then we deliver the final, efficient model to the microcontroller”, Lin says.
To operate that tiny neural network, a microcontroller also requires a lean inference engine. A typical inference engine carries information it doesn’t need, like instructions for programs it rarely runs. The extra codes are no issue for a smartphone or laptop, but could easily upset a microcontroller. Han says, “It doesn’t have off-chip memory, and it doesn’t have a disk. Everything put together is just one megabyte of flash, so we have to really carefully manage such a small resource.”
The researcher’s inference engine was developed in conjunction with TinyNAS. TinyEngine generates the necessary code to operate TinyNAS’ customized neural network. Any unnecessary code is done away with, which reduces the time for compilation. “We keep only what we need,” says Han. “And since we designed the neural network, we know exactly what we need. That’s the advantage of system-algorithm codesign”. During the group tests of TinyEngine, the compiled binary code’s size was between 1.9 and five times smaller than similar microcontroller engines from Google and ARM. TinyEngine also has innovations that decrease the runtime, including in-place depth-wise convolution, which reduces the peak memory usage by half. Subsequent to codesigning TinyNAS and TinyEngine, Han and his team put MCUNet to the test.
MCUNet’s first problem was image classification. The researchers made use of the ImageNet database to train the system with tagged images, and to also test its capacity to classify novel images. On a commercial microcontroller they tested, MCUNet was able to successfully classify 70.7% of the novel images— the previous advanced neural network and inference engine combo was only 54% accurate. Lin says, “Even a 1% improvement is considered significant. So this is a giant leap for microcontroller settings.”
The team found matching results in ImageNet tests of three other microcontrollers. MCUNET beat them on both speed and accuracy for audio and visual “wake-word” tasks, where a user begins interacting with a computer using vocal cues, (like “Hey, Siri”) or by just entering the room. These experiments illustrates MCUNet’s ability to adapt to numerous applications.
The auspicious test results gives Han hope that it may one day become the new industry standard for microcontrollers. He says, “It has huge potential”.
The advance “extends the frontier of deep neural network design even farther into the computational domain of small energy-efficient microcontrollers”, says Kurt Keutzer, a computer scientist at the University of California at Berkeley, who wasn’t a participant in the research. He also added that MCUNet could “bring intelligent computer-vision capabilities to even the simplest kitchen appliances, or enable more intelligent motion sensors.”
MCUNet could also improve security of IoT devices. “A key advantage is preserving privacy. You don’t need to transfer the data to the cloud.”
Analyzing data locally reduces the chances of personal information being hacked. Hans envisions smartwatches not only for sensing blood pressure and the likes, but for also analyzing and helping people understand that information. MCUNet could also introduce deep learning to IoT devices in rural areas with limited access to the internet and vehicles.
Also, MCUNet’s lean computing footprint translates into a lean carbon footprint. “Our big dream is for green AI”, Han said, adding that the training of a large neural network can burn carbon equal to the lifetime emissions of five cars. MCUNet on a microcontroller would only need a small percentage of that energy. “Our end goal is to enable efficient, tiny AI with less computational resources, less human resources and less data”, Han said.
By Marvellous Iwendi.
Source: MIT News