Transferring Intelligence to the Edge: Modeling, Optimization, and Deployment of Computer Vision Models for Embedded Systems

Overview

Patterns recognized by deep learning models from visual information have reached unprecedented precision levels in diverse fields. However, most of the state-of-the-art computer vision systems require high computing resources, which in turn makes them need high bandwidth for image or video transferring to central servers. Recently, the significant improvements in GPU, TPU, and CPU components on diverse embedded devices have opened opportunities for new computer vision applications with chances to work in real-time. Nevertheless, inference near where the data is collected still requires strategies to cope with constrained resources to function without delays. Moreover, efficient systems at the edge must be designed considering power and memory monitoring and estimation. In this tutorial, new opportunities, tools, strategies, and challenges for the deployment of computer vision systems will be explored. Moreover, concrete experiences using OpenVino, TensoRT, ONNX, NVIDIA Jetson Nano, OA