Presenting Fujitsu’s AI Technologies to the world at ICLR 2023

About ICLR 2023

• Conference：ICLR 2023（International Conference on Learning Representations）
• Date：May 1, - May 5, 2023
• Venue：Kigali Rwanda
  • ①
    Title：Multi-Rate VAE: Train Once, Get the Full Rate-Distortion Curve

    • Authors
      (Fujitsu) So Hasegawa
      (University of Toronto) Juhan Bae, Michael Zhang, Michael Ruan,
      Eric Wang, Jimmy Ba, Roger Baker Grosse
  • ②
    Title：Automated Data Augmentations for Graph Classification

    • Authors
      (Fujitsu) Michael Curtis McThrow, Wing Yee Au,
      Tao Komikado, Kanji Uchino, Koji Maruhashi
      (Texas A&M University) Youzhi Luo, Shuiwang Ji

① Training Stabilization Techniques for Generative Models

Multi-Rate VAE: Train Once, Get the Full Rate-Distortion Curve

Fujitsu and the University of Toronto have developed a technology to adjust properties of features encoded from high-dimensional data according to the purposes.

Demands to analyze enormous and diverse data have skyrocketed in various business areas recently. A tremendous amount (or volume) of data is basically imperative to train AI models. In the case of high-dimensional data, deep learning-based methods to reduce the dimensions of input data have been utilized because capturing the features directly from high-dimensional data involves huge computational cost. However, too much reduction leads to the loss of important features, and practitioners are required to adjust the level of reduction depending on the objectives. Therefore, practitioners meticulously iterate the process of training models with the picked value of a parameter to coordinate the level of reduction, by evaluating the reduction level and the loss of important features with the use of trained models, they can eventually determine the appropriate parameter. The arduous iteration requires a vast amount of time and computational resources.

By combining the knowledge of proven learning techniques (e.g. Adam*) from the University of Toronto and Fujitsu’s AI technology expertise based on an information theory (e.g. DeepTwin*), we have developed a new technology that can freely adjust the level of reduction as well as the loss of important features when utilizing the trained model. As a result, the model is acquired after just a single training run. A key point is that the technology enables the appropriate properties of any features to be utilized depending on the application.

In the future, we plan to work towards using the technology for manufacturing and healthcare applications.

• *Adam
  : The most popular training technique in deep learning
• *DeepTwin
  : AI technology capturing characteristics of data correctly, developed by Fujitsu

• Technical Details：“Multi-Rate VAE: Train Once, Get the Full Rate-Distortion Curve” accepted at ICLR 2023 as notable-top-5% （Fujitsu Research TECH BLOG）

Developer Comments

In our project, we are developing technology to extract relevant information from complex real-world data and implementing AI to achieve our goals. This technology is a significant achievement that enables the practical and efficient handling of high-dimensional and complex data. We will contribute to society by further developing this technology and applying it to a variety of fields, including manufacturing and healthcare.

② Data Augmentation Technology for Graph Data

Automated Data Augmentations for Graph Classification

Fujitsu and Texas A&M University have developed a new technology to augment training data and improve accuracy for AI that classifies graph data *.

Expectations are growing for AI that uses graph data to enable the reasoning and interpretation of predictions in various fields. In the field of drug discovery, chemical compounds are represented as graph data that shows the connections between atoms and are attempting to predict molecules with high potential from a large number of drug candidates, which is said to be 10 to the 60th power. In the financial sector, with the accumulation of vast amounts of data, it is expected that transaction histories will be used to detect fraud and forecast growth by viewing them as graph data representing relationships between people and companies.

However, since there is little data on compounds with high drug potential or transactions involving fraud, the lack of training data required for AI has become a major problem that results in insufficient practical accuracy.
There is an existing technique that increases the amount of training data by adding fluctuation to the data so that the prediction result does not change, thereby improving the prediction accuracy. However, unlike images and text, in the case of graph data, important features may change significantly by adding or removing only one vertex or connection. Therefore, it was necessary to devise ways to add fluctuation without changing the prediction results by applying expertise drawn from each field.

Leveraging a machine learning technique called reinforcement learning *, we have developed a technology that automatically selects operations such as adding and deleting connections so as not ultimately to damage important features of graph data. This has enabled us to improve the accuracy of graph data prediction in a wide range of areas without requiring specialized knowledge.

We will continue to expand the range of AI applications in areas such as healthcare and finance by utilizing this technology.

• *Graph data
  : A data format that represents connections between people and things
• *Reinforcement Learning
  : A machine learning method to learn how to choose the best strategy to achieve an objective

• Technical Details：“Automated Data Augmentation for Graph Classification” presented on ICLR 2023 （Fujitsu Research TECH BLOG）

Developer Comments

In our project, we are developing technology that makes the process of human discovery possible with AI. This technology is expected to improve the predictability of explainable AI on graph data developed by Fujitsu and dramatically increase the potential for discovery. In the future, we will contribute to society by applying this technology to a wide range of areas and developing new services that have the potential to lead to numerous discoveries through AI.

Related links

・Presenting Fujitsu’s AI Technologies to the world at NeurIPS 2022（January 30, 2023 Technical Topics）

For more information on this topic

contact-iclrpaper2023@cs.labs.fujitsu.com