A Russian university in collaboration with an European company has developed an Automated Deep Learning platform which can automatically selects and optimizes the structure of neural networks to achieve the best possible values of quality (e.g., accuracy) and improves performance of artificial intelligence (AI) solutions for particular hardware platforms.
This development arises from a known challenge when preparing neural networks for commercial product launch. There is a need to optimize a trained neural network for the particular hardware platform it is intended for launch or deployment in.
The technology is implemented as a framework that is flexible for AI developers, and built on top of Neural Architecture Search engine developed by the team. It allows optimizing and improving both accuracy and latency/RAM/model size during training procedure on PyTorch.
The technology can find the best architecture for the neural network (NN) from one billion available options automatically. It accounts for many parameters including input resolution, depth of neural network, operation type, activation type, number of filters/neurons at each layer, bit width for target hardware platform for neural network inference.
The framework allows to speed up/compress neural network by by 3 to 25 times depending on the type of a task and targeting hardware platform. The technology supports and automatically combines all the main methods of deep neural networks optimization such as neural architecture search (NAS), quantization, structural pruning and distillation. The platform is supplied as a software library for PyTorch and is applicable for all types of processors: x 86 /x 64 GPU, ARM, FPGA and inference frameworks.
The applied innovative approach allows to cut down duration of the optimization process from several months to two weeks with significant reduction of requirements to hardware resources within the optimization process at the same time.
The platform is available for installation in the customers’ infrastructure as well as SaaS option as an integrated part of cloud platforms for machine learning development.
The technology is designed for software developers, companies and organizations, that develop AI solutions.
The technology can be used by cloud service providers and can be supplied as an application for cloud platforms to integrate and increase their revenues.
The technology can be licensed to an industry that is exploring integration of AI and Neural Networks solutions within their operations.
The edge AI market is expected to grow as artificial neural network (ANN) technology matures and new opportunities for deployment of such technologies in different hardware platforms are explored.
Development of NN optimization tools may require months of effort and hundreds of experiments, with degrees of uncertainty in whatever a best-fit optimization solution could be identified. The amount of resources spent in terms of manpower, time and cost meant that development of such edge AI tools is an expensive endeavor. An average project of such development might involve 5 to 7 developers working on a runway of 9 to 12 months with budget in the range of USD 400K and above.
The availability of a ready built technology block for potential tech seekers to scale off their development work on, will help those exploring this space to lower resources spent and risk incurred, during the process of creating new products/services in the edge AI space.
The technology helps to standardize the development process and reduce development efforts 2 times and more. It allows to decrease production risks and cut down the time-to-market for development of AI solutions.
It can significantly reduce consumption of hardware resources for AI services by 3 to 25 times.
Major advantage of the technology in comparison with existing open source methods for neural network compression is that NAS methods, pruning, distillation, quantization work together, and solve one big optimization problem instead of a sequence of separate methods.