Deep learning is part of a broader family of machine learning methods based on AI.
Models that are based on deep learning require huge amount of resources, which include servers with lots of memory and clusters of GPUs. This is why deep learning models are only accessible to a limited few who could afford the resources.
And for those who do have the resources, they can provide the models to others through cloud computing.
As a result, neural networks become popular as more devices could run them.
However, these devices need to connect to the internet to access the cloud, so the AI could work. Without connectivity, the AI is rendered useless.
Since the demand for devices using AI is increasing, tech companies try to improvise by reversing the trend.
And that is by bringing machine learning models to work without the internet, meaning that they are bringing AIs to edge devices.
This time, IBM partners with Massachusetts Institute of Technology (MIT) work on tiny machine learning, or 'TinyML', in order to develop a machine-learning model that can run on devices that have limited memory and processing power, and/or have internet connectivity that is either non-present or limited.
As detailed in a paper presented at the NeurIPS 2021 conference, the model is called 'MCUNetV2'.
This model can run CNNs on low-memory and low-power microcontrollers (MCU).
With MCUNetV2, IBM and MIT bring TinyML to the edge and one step further.
The researchers at the two companies are utilizing the fact that microcontrollers are cheap, and they’re everywhere, embedded in consumer and industrial devices.
Microcontrollers don't have the things their generic computing devices have. For example, microcontrollers have small CPUs, which are limited to just few hundred kilobytes of low-power memory (SRAM) and a few megabytes of storage. Due to the low-power resources, these microcontrollers can operate on cell and coin batteries for years.
Whereas standard consumer CPUs consume between 65 watts and 85 watts and standard consumer GPUs consume anywhere between 200 watts to 500 watts, a typical microcontroller consumes power in the order of milliwatts or microwatts.
For these reasons, fitting MCUNetV2 on MCUs can open the doors for many applications and opportunities.
The reason for the development of this MCUNetV2, is because deep learning is so successful, but is not applicable to all situations.
For example, drones in a rescue mission may have its automation limited when internet connectivity isn't available. In other domains, such as in healthcare, privacy requirements and regulations can make it very difficult to send data to the cloud for processing.
Without access to the cloud, applications that require real-time machine learning won't work properly.
With TinyML and MCUNetV2, the tech industry can make on-device machine learning possible, and because of that, it's commercially attractive.
According to a post by MIT, it's "appealing but challenging."
In the past, there have been many efforts to bring deep neural networks to fit into small an low-powered computing devices.
For example, there have been projects to reduce the input, reduce the number of parameters in the deep learning model, like through pruning or other optimization techniques and compression methods.
TinyML takes all previous attempts to the next level, simply because it addresses the memory bottleneck of the neural networks.
And using MCUNetV2, the two companies want to address the peak-memory bottleneck of convolutional neural networks (CNN), a deep learning architecture that is especially critical for computer vision applications.
This is possible because MCUNetV2 only needs to store one patch of neurons at a time, meaning that it can reduce memory peak considerably without having to reduce the resolution or parameters of the model.
