About tinyML Asia

TinyML promises us to teach the smallest of devices to feel, hear and see things – an amazing feat. But Machine Learning used to be the domain of large data centers full of servers, so how do we suddenly run these algorithms on battery-powered devices with kilobytes of RAM? In this talk we’ll look at the technologies that underpins TinyML, how traditional signal processing is still incredibly helpful, and at the state-of-the-art algorithms for processing vibration, biosignal, audio and image data directly on your sensors.

Recent progress in computing hardware, machine learning algorithms and networks and availability of large datasets for model training have created a strong momentum in development and wide deployment of game changing AI applications. Dedicated hardware becomes tiny and very energy efficient (with mW or less power consumption), algorithms and models – smaller (down to 10s of kB of memory requirements), software – lighter down to deployment on deeply embedded platforms. This enormous technology innovative wave and fast growing tinyML ecosystem create enormous opportunities for new applications and business models. This presentation will review the state-of-the-art of tinyML, its applications in various “verticals”, describe some practical examples of technologies and products pioneered at Qualcomm and else, and discusses near-term trends and opportunities, as well as call to actions. As an end result, tinyML will create a better, healthier and more sustainable environment for all.

According to CCID, an industry consulting agency, the China’s AI chip market will keep a growth rate of more than 50% in the next three years, and further increase to 30.57 billion yuan in 2021. Driven by the rapid development of the market, China’s AI chips continue to attract the attention of industry and academia. A large number of start-up companies have sprung up in China, such as SenseTime, Ali Pingtouge, Cambrian and other start-up enterprises to cut into the emerging subdivision fields, overtake in different lanes, and catch up with international giants, which has attracted extensive attention. Domestic academic circles are also promoting the development of new technology of artificial intelligence chip in China. In recent years, the combination of chips based on traditional computing architecture and various hardware and software acceleration schemes has achieved great success in some scenarios, but the general artificial intelligence chip technology which can flexibly adapt to diverse needs has not yet emerged. Reconfigurable computing technology has the characteristics of real-time dynamic configuration, which makes the software definition no longer limited to the functional level, and the calculation accuracy, performance and energy efficiency of the algorithm can be included in the scope of software definition, which makes the intelligent computing chip not only obtain high flexibility, but also has more advantages in computing energy efficiency. Therefore, in order to better adapt to the characteristics and requirements of intelligent computing and to improve the resource utilization and energy efficiency ratio of neural network computing, highly parallel computing architecture with hardware reconfiguration capability will continue to be a research hotspot. In a word, this field is still in the primary stage of “a hundred schools of thought contend”, and there is a huge space for innovation in scientific research and industrial application.

Deep neural networks (DNNs) have achieved great success in many applications of artificial intelligence (AI). To enable always-on and pervasive AI applications in mobile and IoT devices, ultra-low power neural network processors are required. With the progresses of both neural network algorithms and computing architectures, it is possible to design mW-level neural network processors. In this talk, we introduce Thinker processors which have the potential to embed AI in everything.

With the explosion of endpoint/IOT devices, the demand for compute intelligence on the edge continues to increase. However accelerating Machine Learning (ML) /AI on the edge is a complex task. Newly available micro-network processor units (micro-NPUs) enables a significant change in Machine Learning processing on the edge. By enabling low cost and highly efficient AI solutions, the micro-NPUs such as Arm’s Ethos-U unlock new applications/capabilities on embedded devices like never before. In this session, learn more about the latest advancements in Machine Learning solutions for the next generation of embedded devices.

More and more data-driven applications have been widely accepted and deployed in the market to improve life quality, where local intelligence solutions are embedded in the IoT devices to reach better data security control and real-time performances under limited bandwidth constraints. In the first part of this talk, model reduction related to network architecture search (NAS) and adaptive quantization (AQ) will be addressed to see how both storage capacity and computing complexity can be reduced while maintain acceptable accuracy, energy-efficiency, and service quality. In the second part, a new approach related to meta-learning will be introduced to enhance inference accuracy for AIoT devices deployed in different environments. Some examples related to image/video services will be provided to demonstrate the feasibility of our proposal in real-life applications. Finally some topics will be outlined for future AIoT research.

Machine learning on embedded hardware is a new field that’s only beginning to be used in the developed world, but it has some characteristics that may make it particularly appropriate for developing countries. The hardware and software required to train students, advance research, and build products is much cheaper than the equivalents for Cloud ML, and since the devices don’t need reliable network or even mains power connections, they can be deployed almost anywhere. Pete Warden is the Technical Lead of the TensorFlow Micro open source project from Google, and has previously helped projects like Plant Village use on-device deep learning in West Africa. In this talk he’ll introduce some of the possibilities that edge compute opens up for the world outside the West, and will be looking to learn from the audience what problems in their domains it might be applicable to.

Neural network language model (NNLM) has shown to be a fundamental component for speech recognition and natural language processing in the deep learning era. Unfortunately, large memory consumption of prohibits its use in many resource-constrained scenarios. Effective NN LM compression approaches that are independent of NN structures are therefore of great interest. However, most compression approaches usually achieve a high compression ratio at the cost of significant performance loss. We will show that, with advanced structured quantization techniques, it is possible to achieve a very high NNLM compression ratio, 70-100, without losing performance compared to the uncompressed models.

The rapid rising computing power over the past decade has supported the advance of Artificial Intelligence. Still, in the post-Moore era, AI chips with traditional CMOS process and VanNeumann architectures face huge bottlenecks in memory walls and energy efficiency wall. In-memory computing architecture based on emerging memristor technology has become a very competitive computing paradigm to deliver two order-of-magnitude higher energy efficiency. The memristor process has apparent advantages in power consumption, multi-bit, and cost.However, it faces challenges of the low manufacturing scalability and process variation, which lead to the instability of computation and limited capability of accommodate large and complex neural networks. This talk will introduce the algorithm and architecture co-optimization approach to solve the above challenges.

The recent years have witnessed unprecedented advances in machine learning (ML) accelerators on mobile and IoT devices at the edge. The system-on-chips, pairing regular processor cores with ML-capable GPU, DSP, and ASIC accelerators, are dominating the edge landscape. However, the current state-of-the-art is inadequate in the software dimension despite tremendous progress at the hardware level. This talk will put the spotlight on the compiler and runtime software approaches to unleash the full potential of the hardware for ML in mobile and IoT applications. In particular, we will present deep neural network specific optimizations, workload partitioning, and voltage-frequency scaling to orchestrate the different on-chip compute resources in a synergistic manner and achieve low-power, real-time edge ML.

Raspberry Pi, a small, inexpensive computer favored by developers around the world, is equipped with a VideoCore IV/VI GPU, but it is underutilized as a computing resource. We have developed device programming tools, libraries, math kernels and an optimizing graph compiler specialized for machine learning models to use VideoCore for GPGPU. And we have achieved a significant speedup of deep learning inference on Raspberry Pi series without the use of computation-reducing techniques, such as quantization, which may compromise accuracy. This talk will introduce the architectural features of VideoCore and techniques for the acceleration, as well as other our research and development results on ARM CPUs, Intel GPUs, FPGAs, etc.

In this session, we are going to introduce TencentOS Tiny open source project, and how it accelerates end to end AIoT application development on MCUs. We will also present our solutions in retail, smart conferencing and other industrial verticals.