Fujitsu’s Novel AI Deep Learning Technology Sheds New Light on Real-World Problems

Dr Tsuneo Nakata, CEO at Fujitsu Laboratories of Europe, explains the benefits of this new deep learning technology:
“In recent years, we have seen a massive surge in technological advancements that use hardware accelerators to support the enormous scale of computations required to build Deep Neural Networks (DNN) for AI applications. The continuous increase in the computational costs of DNNs is a major challenge, particularly when the model size of DNNs increases to the point that it cannot fit into the memory of a single accelerator. Wider and deeper Neural Networks are needed, together with finer classification of categories, to address emerging AI challenges. Our solution addresses this directly, distributing DNN memory requirements onto multiple machines. With our technology, it is possible to expand the size of neural networks that can be learned on multiple machines, enabling the development of more accurate and large-scale DNN models.“

The new solution achieves this new memory distribution process by transforming the layers of arbitrarily designed neural networks into equivalent networks in which some, or all, of its layers are replaced by a number of smaller sub-layer parts. These sub-layer parts are designed to be functionally equivalent to the original layers, but are computationally much more efficient to execute. Importantly, since the original and new layers stem from the same profile, the training process of the now transformed and distributed DNN converges to that of the original DNN at no added cost.

Fujitsu Laboratories of Europe evaluated the new technology extensively, including applying the new mechanism to Caffe, an open source deep learning framework widely used by R&D communities around the world. The solution achieved more than 90 percent efficiencies in memory distribution when transforming the fully connected layers of AlexNet onto several NVIDIA GPUs. As a hardware independent technology, it has the capabillity of exploiting the computational power of both conventional processing units as well as current and emerging hardware accelerators including, for example, NVIDIA GPUs, Intel Xeon Phi, FPGAs, ASICs etc. or any other alternative hardware chips specifically tailored for increasing the computational efficiencies in Deep Learning.

Example applications for the new solution include healthcare analysis (e.g. diabetic retinopathy detection); satellite image classification and analysis; natural language processing, where largescale deep learning models are required to model and learn the full complexity of the human language; largescale graph-based data involving IoT devices, financial transactions, social network services etc.

Fujitsu Laboratories of Europe is a centre of excellence for Fujitsu’s advanced research into machine learning and deep learning, as part of the digital solutions and services being developed under Fujitsu's human-centric AI initiative called Zinrai. Its activities include extensive collaboration and co-creation with Fujitsu customers and research organisations across EMEIA, including San Carlos Clinical Hospital in Madrid (with the HIKARI AI healthcare solution), the University of Seville (data analytics for tourism applications), and the 5G Innovation Centre in the UK.