“Although artificial intelligence is already around, people are still exploring how to implement deep learning technology in the “original” microcontroller application scenario. This time, the editor will tell the little friends who are still searching up and down a short story about the successful implementation of “microcontroller + AI”.
When walking in the fields and mountains, enjoying beautiful scenery and breathing fresh air, you can often see motorized wells used for irrigation. You may not pay more attention to them except for being careful. But have you ever imagined that they will also be associated with artificial intelligence and microcontrollers?
Although artificial intelligence is already around, people are still exploring how to implement deep learning technology in the “original” microcontroller application scenario. This time, the editor will tell the little friends who are still searching up and down a short story about the successful implementation of “microcontroller + AI”.
Powerful edge meter reader in deep wells
The protagonist of the story is the “Edge Meter Reader” module jointly developed by Beijing Smart Water Development Research Institute and Beijing Hongcheng Xinding Intelligent Technology Co., Ltd. This module will be the first to be used for smart meter reading of mechanical water meters. Let’s take a nice photo first–
The light blue “hat” worn on the water meter on the left is it, and the picture on the right is a “naked photo” of it. This edge smart meter reading module uses the NXP i.MX RT1020 cross-border microcontroller to read the camera and runs the “SlimSSD” detection algorithm based on deep learning. It can be directly buckled on the water meter dial to take pictures and recognize the readings of the dial.
This module is very powerful and can be used in many occasions. In addition to being installed at home, it can also be “landed” to the main water pipe-
Not only satisfied with “landing”, but also “falling in the well” –
The program was well received and obtained the invention patent
After nearly two years of development and rigorous testing, this module has been affirmed in the appraisal of the results:
“The unblocked communication rate of the edge AI meter reader is more than 95%, and the average recognition accuracy rate is 83.42%. Among them, the recognition accuracy of 8 meters is more than 98%, and it has achieved good performance in terms of performance; automatic auxiliary data correction and manual review The correction rate is 100%, and the data is truly usable; the power consumption is converted according to the power consumption of one piece of data per day, and it can work for an average of 4425.6 days (about 12.1 years), which has greatly exceeded the design life of 8 years.”
What’s more gratifying is that after the improved model, the latest news unblocking rate reaches 96%!
The picture below is the effect of using the edge AI meter reader to detect the recognition area, recognize the reading, and upload the recognition result and the detection area in the original picture together. The amount of data transmitted wirelessly at one time is only a few hundred bytes (while sending the full picture) Tens of thousands of bytes).
Beijing Water Affairs Bureau also plans to conduct pilot applications in Beijing’s East-West Water Transfer Management Office, more than 70 pumped wells in Mentougou District, and Ganjiakou Building. Realize the unattended water metering of rural water wells, farmland and forest area motorized wells, and water supply pipeline network, and realize the function of automatic meter reading without replacing the original water meter.
What’s more gratifying is that the design of this edge AI meter reading device has been repeatedly designed and patterned for many times. The applicability of the edge AI meter reading device has become stronger and stronger, and it has obtained a high-gold invention patent.
The essence of analytical solutions: deep learning
Next, the editor will tell you about the most “essential” part of this program-deep learning.
What makes the editor most admired is that this meter reading module uses Object Detection (OD) technology that is more advanced than image classification, and realizes that it can automatically adapt to new dials without adjusting parameters. And until recently, we have seen that some other manufacturers have just proposed similar deep learning technology, but use a reference model for handwritten digit classification-note that it has just been proposed-and it is still the most basic of deep learning computer vision.” Image classification” technology.
What is the key difference between image classification and object detection? Xiaobian drew a sketch to illustrate (forgive Xiaobian’s art is taught by math teacher).
The most important thing is that the classification model treats the image as a whole and gives a category (precisely predicting the probability of each category and selecting the largest category); while the detection model has to do two things, that is, to find the image For the location of each object, it is also necessary to determine what type of object is found. However, although it seems that detection is much more powerful than classification, the magic is that the technologies they share are highly similar-in particular, among the components of the detection model, the most important one is called the “backbone.” The key part of is from the convolutional neural network part of the classification model, which is used to extract abstract and general image features.
It can be considered that the classification model adds a classifier on the basis of the convolutional neural network (usually a single-layer fully connected layer, also called a perceptron, is enough); and the detection model extracts multiple intermediate results and the final output of the convolutional neural network , And add the related structure of the detection neck and the detection head, but it is much more complicated than the classifier. The training of the backbone network is generally realized with the aid of a classification model.
Back to the application of AI meter reading, if you use the classification model, you must manually set the classification area for each type of dial, one for each number, which is very troublesome; but the detection model can automatically find out where to read and how many to read. The number is obviously more convenient. In this specific patent, an optimized version of a single shot multibox detector (Single Shot Multibox Detector, SSD) model is used-the inventor called it “SlimSSD”. It can be seen from the name that it is a more advanced version. “Slim” SSD-the inventor also used the attention mechanism to more accurately help tailor the model.
Let’s talk about the details of SSD later. Here, let’s first give the “look” of a highly streamlined SSD model.
The three eye-catching vertical bars in the figure are the backbone of this model, which comes from the convolutional sub-network part of a concise classification model. The lower part of the “parallel” structure is used to detect and classify objects of different sizes in the input image. Finally, they are summarized into the left and right branches, which respectively give the detected object coordinate information and object category information.
SSD, like many other deep neural networks, is very flexible. Its composition can be modified and optimized like building blocks according to different application requirements. The computing power requirements can even be optimized by hundreds of times, so that the microcontroller can also be used. Bearer. The editor checked the patent number “CN113255650B” and found that the SlimSSD optimized by the customer reduced the official SSD model to only 0.5% of the original SSD model, while still maintaining 99% accuracy! What is this concept? To put it vividly, a 200-jin strong man can carry 200-jin sacks, but now a 1-jin villain can carry 198-jin sacks! Well, it’s almost the big red baby in the gourd baby.
Don’t let your computing power limit your imagination
This success story also deeply shocked the editor, feeling that my lack of understanding of deep learning and practical applications has limited my imagination.
I feel that many people feel that “you can’t do anything with computing power less than 0.5 TOPS”, and this smart edge meter reading solution is completed on the i.MX RT1020 platform with theoretically effective computing power of only 0.0003 TOPS, which is 1600. How many times! And in an unattended environment, it can be copied once a day and work continuously for more than 12 years with only the battery!
Seeing this, I want to express my inner exclamation in one sentence:
Deep learning => create miracles
Deep learning + NXP microcontroller => another miracle
In addition to the advanced nature of the model itself, what is even more commendable is that the main development team of this module, Beijing Hongcheng Xinding Technology Co., Ltd., started the project two years ago, and NXP’s eIQ machine learning kit for microcontrollers was released only half a year ago. , They have accomplished such a seemingly impossible miracle alone with our technical support alone!
Among them, the one that impressed the editor the most was Mr. Lian Yongkang, the president of Beijing Hongcheng Xinding. Three years ago, the editor met him at an MCU+AI seminar. At that time, the editor thought about the project-yes. “Try it out” using basic image classification, but Mr. Lian Yongkang resolutely started this project and adopted more advanced object detection methods with great courage. You know, let alone basic image classification 3 years ago, even the basic software of deep learning on microcontrollers is almost blank, and Arm CMSIS-NN has only been released for a few months.
Written at the end
The above story comes to an end, but the complete story continues. The editor learned that Beijing Hongcheng Xinding Technology Co., Ltd. did not stop there, but on this basis, further developed a modified version that can be used in addition to the water meter, such as the fire extinguisher pressure gauge and the liquid crystal Display instrument. The combination of artificial intelligence will escort the people to live and work in peace! Among them, NXP’s high-quality long-life microcontrollers will also continue to fulfill the glorious mission of undertaking computing platforms.
Finally, the editor would like to say that the shrinkage of deep learning is much greater than we imagined. As long as the model is reasonably optimized and simplified according to the actual requirements of the application and the characteristics of the hardware platform, there are many applications that are hard to think of. Can become a reality. Especially don’t underestimate the potential of microcontrollers.
Although the computing power of the microcontroller is much weaker than that of the PC or the application processor, the burden on it is much lighter. Coupled with the great scalability of the deep learning model, there are too many “impossible” practicalities. The above is possible, just wait for you to continue the story of the miracle. With more miracles, it becomes ordinary.
▲The author of this article is Song Yan, a system engineer at NXP Semiconductors.