As an important representative of the new round of scientific and technological revolution and an increasingly powerful force in all fields of science and technology, artificial intelligence is being talked about by more and more people and is becoming a new engine for global economic development. During the two sessions, the government work report proposed to strengthen the research, development and application of the new generation of artificial intelligence, develop intelligent industries and expand intelligent life. This is the second time that "artificial intelligence" has appeared in the government work report as a scientific and technological term since it was first written in 2017; PricewaterhouseCoopers previously predicted that by 2030, AI will contribute $15.7 trillion to the world economy, a figure that will exceed the current economic aggregate of China and India.
At the technical level, AI is moving towards the terminal side. On this basis, the close combination of heterogeneous solutions and software is also becoming the mainstream of technology development. As a leader in the field of low-power and high-performance application processing technology, Qualcomm provides one of the largest commercial terminal side mobile platforms for AI. The company has expanded its product solutions to support the partner ecosystem using its platform and use AI to provide an amazing user experience. In view of this, Qualcomm announced the launch of AI Engine during MWC. The AI Engine is composed of multiple hardware and software, which can accelerate the implementation of terminal AI user experience on some Qualcomm Snapdragon mobile platforms.
Why is AI moving towards the terminal side—— Traditionally, much of the training and reasoning of neural networks is done in the cloud or based on servers. In the past few years, with the continuous improvement of mobile processor performance, higher requirements for data privacy protection, and the complete reliability brought by the continuous evolution of connection technology, the entire pattern has changed greatly, and many AI reasoning work, such as pattern matching, modeling detection, classification, recognition, detection, has gradually shifted from the cloud to the terminal side, This is of great benefit to protecting data privacy, improving performance and overall reliability. In other words, in the future, terminals will become an important entrance to artificial intelligence, including smart phones, laptops, head display devices, Internet of Things terminals and automotive systems.
Qualcomm AI Engine "hard and soft"
Qualcomm AI Engine is a collection of hardware and software components that accelerate the realization of AI user experience on the terminal side on the Snapdragon mobile platform. The key components include: hardware: the optimized combination of artificial intelligence of Hexagon vector processor, Adreno GPU and Kryo CPU; And a set of software tools and libraries: Snapdragon Neural Processing SDK, Android NN and Hexagon NN; All of the above work together to support developers and OEMs to easily implement and accelerate AI processing on the Snapdragon core.
To migrate AI reasoning from the cloud to mobile terminals, Qualcomm makes full use of the multi-core heterogeneous computing core Hexagon vector processor, Adreno GPU visual processing subsystem and Kryo CPU's Snapdragon programmable architecture. In fact, the highly efficient terminal AI is a multifaceted problem that cannot be solved in the best way by a single AI core. In contrast, different AI application scenarios require different power consumption and computing resources, so programmable heterogeneous computing is required. Qualcomm's heterogeneous computing solutions provide developers with a wider range of choices. They can support a large number of convolutional or cyclic neural networks for different types of functions, based on different types of data, and at different levels of computing accuracy. Heterogeneous computing cores (such as Snapdragon's CPU, GPU and vector processor) have many advantages required by developers, OEMs and independent software developers (ISVs), allowing them to accelerate their application implementation through precise control or the pre optimized AI library developed by Qualcomm.
Hardware: multi-core heterogeneous computing core
·HEXAGON vector processor
The Qualcomm Hexagon vector processor is an acceleration engine in the vector processor subsystem, which supports the processing of 8-bit fixed-point operations. Compared with GPU or CPU, application workloads involving vector mathematics are generally more suitable to run on Snapdragon's vector processors.
· ADRENO GPU
It has a huge parallel data set, requires a large amount of vector mathematics, especially the application workload that requires floating point precision, and is very suitable for running on the Adreno GPU of Snapdragon. The latest Adreno GPU optimizes the AI workload with 16 bit floating point precision to provide better energy efficiency performance, making it an ideal choice for AI applications such as computer vision object classification, video style conversion, scene detection and games.
· KRYO CPU
Neural network models are usually trained in the cloud with 32-bit floating point (high-precision). It can also achieve similar operation on the CPU on the terminal side, but the cost is too high, requiring the terminal to make some sacrifices in performance and power consumption. Snapdragon 845 optimizes support for 8-bit fixed-point precision on Kryo CPU to help support applications with larger cores, relatively less vector processing, irregular data structures and/or complex process control.
In general, the most efficient solution to accelerate AI on the Snapdragon platform is to allocate AI workload to CPU, GPU and vector processor. This provides developers with a variety of flexible options to achieve AI user experience.
Software
The second component of AI Engine is software and tools. Software and tools are undoubtedly more important when dealing with AI applications. Hardware is an accelerated platform, while developers at the software level can achieve a lot of innovation and create the latest AI applications. The core software architecture of Qualcomm AI Engine includes: Snapdragon Neural Processing SDK, Android NN API released with Google Android Oreo, and Hexagon Neural Network (NN) library. Among them, the Snapdragon neural processing engine can help developers save time and work to optimize the running performance of trained neural networks on the Snapdragon terminal. The Hexagon NN library allows developers to directly run AI algorithms on Hexagon vector processors. It provides optimized deployment for the basic machine learning module and significantly accelerates the operation of artificial intelligence such as convolution, pooling and activation.
Extensive ecosystem support to promote AI application development
In the face of a large number of AI features and applications brought by intelligent terminals today, terminal side AI needs multiple hardware architectures, software tools and frameworks to meet the needs of developers and performance thresholds. At present, many leading global OEMs, developers, cloud providers and other ecosystem manufacturers have adopted or announced support for Qualcomm AI Engine to optimize and accelerate AI applications.
During this year's MWC, Qualcomm and a number of Chinese enterprises, including SenseTime of Shangtang Technology, Face++of Kuangshi Technology and Tencent, demonstrated AI applications realized by AI Engine components of Qualcomm's artificial intelligence engine.
Qualcomm cooperated with Shangtang Technology to demonstrate real-time video style conversion. The application can process the ordinary video captured by the camera into a specific stylized effect video through the model in real time. The advanced AI algorithm of Shangtang Technology and the powerful processing performance supported by Qualcomm AI Engine can ensure that the stylized video has richer colors, more obvious details, and is full of sense of hierarchy and stereoscopy, while ensuring the real-time and smoothness of the converted video.
Real time video style conversion application supported by Shangtang Technology AI technology and Qualcomm AI Engine
Qualcomm and Kuangshi Technology Face++jointly demonstrated the application of artificial intelligence for smart phones - 3D Animoji. Thanks to the support of Qualcomm AI Engine, the algorithm can achieve faster processing speed on the Snapdragon mobile platform, and realize complex scene perception calculation under extremely low power consumption, and complete real-time 3D face modeling and facial expression muscle tracking.
Qualcomm and Kuangshi Technology Face++jointly demonstrated the application of artificial intelligence for smart phones - 3D Animoji
Qualcomm and Tencent demonstrated the interactive feature called "high-energy dance room" in Tencent's mobile QQ application. In mobile QQ on Android side, AI Engine component is adopted to accelerate the frame rate of this feature.
Qualcomm and Tencent show the characteristics of Tencent mobile phone QQ "high-energy dance room"
summary
At present, Qualcomm Snapdragon 845, Snapdragon 835, Snapdragon 820, Snapdragon 660, and the newly launched Snapdragon 700 series mobile platform will support the AI Engine, of which Snapdragon 845 will support the most advanced terminal side AI processing. Xiaolong 845 is Qualcomm's third-generation AI mobile platform. Compared with the previous generation SoC, Snapdragon 845 has brought about nearly three times the overall performance improvement of AI - changing mobile terminals into excellent personal assistants; Simplify the shooting of pictures and videos; Improve VR game experience and make voice interaction more natural.
Now, many smart phone manufacturers have used AI Engine on the Snapdragon mobile platform to accelerate AI applications on their terminals, including Xiaomi, Yijia vivo、OPPO、 Motorola, ASUS, ZTE, Nubia, Hammer and Black Shark, some of which are planning to use AI Engine to optimize AI applications on their flagship Snapdragon smart phones in the future.
In addition to Shangtang Technology and Kuangshi, many other leading AI software development enterprises have also brought exclusive use case optimization to Snapdragon mobile platform. Hongruan provides Xiaolong customers with single shot and dual shot algorithms, and actively develops AI user experience. Chuangtong Lianda provides complete solutions for AI visual use cases and terminals. Uncanny Vision provides optimization models for people, vehicles, and license plate detection and recognition. These enterprises and more will use the AI Engine component of the Qualcomm AI engine to accelerate the performance and user experience of their AI models.
In addition, Qualcomm and Baidu also jointly develop terminal side keyword detection and natural language processing technology based on DuerOS to supplement cloud services. At the same time, Qualcomm and Google have maintained a long-term close cooperation in the AI field. Since the Snapdragon 820 mobile platform, the two sides have worked together to achieve the optimization of TensorFlow for mobile terminals. The Snapdragon mobile platform has thus become the first commercial mobile SoC supporting TensorFlow. At present, Qualcomm is optimizing many Facebook applications, especially those in the AR field, based on Facebook's Caffe2 framework. Qualcomm, Microsoft and Amazon are also cooperating using the ONNX exchange format to jointly explore the combination of terminal AI and cloud AI.
Follow our Weibo @ Love Computer
Follow our WeChat official account: playphone
Of course, we also pay attention to our Bilibili account: love computer
