Fujitsu Develops Automatic DNN Construction System

Beijing,China, July 19, 2017

Fujitsu R&D Center Co., Ltd. and Fujitsu Laboratories Ltd. (collectively "Fujitsu"). today announced an advanced deep learning technology for helping non-professional users to employ DNN models to solve their problems. By using this system, well-balanced DNN models between the accuracy and time consumption will be automatically generated based on the given task of the user. Furthermore, the system is able to provide the DNN model to the user within couple of hours while the conventional process may take several days. We hope this technology can make further contribution to enlarge the DNN application field and realize the general AI platform.

Details are being announced at the 18th ICME meeting, an International conference on multimedia & expro in Hong Kong, China, from July 10 to 14.

Background

Deep learning technology has been widely used for AI related tasks with promising performance. However, construction and parameter adjustment of DNN models need specific knowledge and experience, which are difficult for non-professional users. Even for experts of deep learning, it takes much time and effort to design proper model for certain task. All of these reasons have hindered the expansion of DNN application fields.

Topics

In order to make easy and quick usage of DNN models, especially for non-professional users, we have considered the automatic DNN construction and proposed the corresponding system. This system consists of three steps, data analysis, model recommendation and accuracy curve fitting.

Technology

1. Complexity score analysis for user’s task

   We evaluate the given task of the user and give a specific “complexity score” for it. The complexity score represents the difficulty level of the task. For example, for the classification task, the image data of different classes are analyzed. The average inter-class and inner-class difference are calculated. Finally, the difficulty of this task is given based on the average difference. With the complexity score, the system is able to search the optimal DNN model and parameters to match user’s task perfectly.

2. DNN models ability score evaluation

   We use the “ability score” to describe the classification ability of the CNN model. The ability score is evaluated by considering the total calculation times, the shape (depth and width) of the model and the gradient vanishing problem. We can give a specific ability score for any DNN model.

3. Model recommendation and accuracy curve fitting

   Based on the complexity score, the system will recommend three DNN models according to their ability scores. The three models are small, middle and large model. Users can choose one from them according to their specific requirement. For example, small model for higher speed and large model for higher accuracy. We will train all of the three models and fit a curve to show the trend between model complexity and classification accuracy.

   

   Figure 1: The framework about automatic DNN construction system

   
   

Effect

Our automatic DNN construction system can deal with most of classification tasks and has an excellent performance. Fig 2 shows a recommendation example on dataset of handwritten Chinese released in 2010 by the Institute of Automation, Chinese Academy of Sciences (CASIA), which is used as a standard by academic societies. The best performance by far is 97.30%[1]. Three models are recommended by our system. The accuracy-ability score curve is fitted from the recognition accuracies of the three recommended models. The classification accuracies of the middle and large model are near the-state-of-the-art level. The total recommendation time, including complexity score calculation and the training of the three models, is 30 hours with GeForce GTX TITAN X and Intel® Xeon® CPU E5-2640 v3 @2.60GHz. Our system can recommend good performance models to the non-professional users full automatically. Thus, huge time for study deep learning is saved.

Future Plan

Fujitsu will aim to bring this technology to Zinrai in 2017, Fujitsu's AI technology platform, and apply it to various image recognition tasks.

[1] Xuefeng Xiao, et al, “Building fast and compact convolutional neural networks for offline handwritten Chinese character recognition”, Pattern Recognition, Volume 72, December 2017, Page 72-81.