Fujitsu AI Technology Dynamically Automates Complex Data Integration for Knowledge Processing
Fujitsu Laboratories of Europe Ltd.
- Fujitsu Laboratories of Europe has developed an innovative AI-based technology that transforms the process of data integration for knowledge processing.
- Dynamic Data Loading (DDL) technology solves complex data reconciliation challenges across multiple time-critical sectors, including banking and healthcare applications.
- Analyzes company data, linking relevant information from open and private sources to inside knowledge, and automates the reconciliation process
- DDL technology is able to remember user decisions and provide comprehensive decision-making support
Fujitsu has focused on solving the problems associated with analyzing huge volumes of data reliably, accurately and rapidly in the financial domain. Fujitsu’s reconciliation technology is unique in the combination of advanced features used for record linkage. The main innovation lies in the entity type and domain recognition methods that use the system’s knowledge base to recommend the candidates. Domain recognition provides considerable context to the user and to the system itself to facilitate the reconciliation process. Another innovation stems from the use of AI to automate tasks, with the system learning from user decisions and feedback, resulting in a progressively more customized user experience, matched by a high level of automation for simple, repetitive tasks. The technology is based on Fujitsu’s cutting edge entity reconciliation and microservice orchestration techniques, transforming high volume data reconciliation practice for any application.
Dr Adel Rouz, CEO of Fujitsu Laboratories of Europe, explains: ‘‘We have extensive co-creation experience, particularly resulting from deep learning and AI projects in the financial services sector, and have applied this to develop an innovative new approach to complex dynamic data loading and integration challenges. Our AI-based approach breaks new ground in terms of the scalable, accurate and intelligent analysis of huge volumes of data. We have combined three essential ingredients : a powerful data reconciliation mechanism with advanced and flexibile analysis technologies, supported by an interactive virtual assistant. Our initial application involves a platform for regulatory authorities, with future applications involving any industry handling massive data volumes, such as the finance sector, healthcare, retail and manufacturing. It is an exciting development, which we are confident has wide-ranging future potential - for example being applied to identify cross-border relationships, by integrating open data sets from around the world.’’
Figure 1: Key benefits of Dynamic Data Loading technology
Fujitsu’s new approach applies a novel methodology for record linkage, using the system’s knowledge base for enhanced entity type and domain recognition. For example, it uses linkage types such as company and company, and company and person. In most cases, the incoming dataset lacks data property descriptions or any reference to standard vocabularies/ontologies. Therefore, the person in charge of the data integration has to guess relevant meta-information (e.g. the meaning of property names) or to ask the data provider. With Fujitsu’s technology, the dataset is contextualized, with minor input from the user. Additionally, the system’s capability to learn from user actions is an important reconciliation feature. In Fujitsu’s reconciliation technology, learning from user decisions and feedback results in a more customized user experience and a high level of automation for simple and repetitive tasks.
Figure 2: Dynamic Data Loading technology is implemented as a microservice workflow
The figure above details the reconciliation workflow components of Dynamic Data Loading technology, together with an example of reconciled company data from the finance sector. The workflow is divided into four main blocks:
- Data Property Reconciliation module
- Entity Type and Domain Reconciliation module
- Entity Disambiguation module
- Knowledge Base Storage
During a one-month trial period, data reconcilation times and data loading times were significantly reduced, typically taking one week compared to one month previously. Importantly, the process of reconciling a financial dataset, such as the one used in the example, produces a knowledge graph that can be used for future data reconciliation tasks.
Fujitsu Laboratories of Europe is a Center of Excellence for Fujitsu’s advanced research into machine learning and deep learning, as part of the digital solutions and services being developed under the Fujitsu’s Human Centric AI approach Zinrai. Fujitsu Laboratories of Europe’s activities include extensive collaboration and co-creation with Fujitsu customers and research organizations across Europe, including San Carlos Clinical Hospital in Madrid (with the HIKARI AI intelligent healthcare solution), the University of Seville (data analytics for tourism applications), and the 5G Innovation Centre in the UK.
Notes to Editors
Fujitsu Laboratories of Europe’s new AI solution is part of Fujitsu’s digital solutions and services being developed under the Human Centric AI approach called Zinrai, which comprises a comprehensive framework of component technology, such as machine learning, deep learning and visual recognition.
Detailed Information relating to Fujitsu’s Dynamic Data Loading technology
- Data Property Reconciliation module: Standardized data properties are matched against existing properties in the Knowledge Base. If the labels are similar (string distance) and/or a high percentage of new and stored values coincide, the new property is mapped to the existing one in the Knowledge Base. For instance, in the figure "id" is mapped to "Ticker". In the second phase of this block, the Potential Identifiers Finder looks for identifier candidates within the dataset by looking at the Knowledge Base mappings and at the uniqueness of data values. As a result, the property "Ticker" is the proposed candidate
- Entity Type and Domain Reconciliation module: First, given a set of standardised data properties and candidate identifiers, the system recommends the top ranked entity types, e.g. "Bank". The recommendation is based on the existing knowledge, the user preferences, and the preferences of the rest of the users. In the second step, the system considers the previous inputs for this module plus the entity type selected by the user to suggest a domain, such as "Finance", also based on existing knowledge and user preferences
- Entity Disambiguation module: The goal of this module is to disambiguate and recognize the real world entity which the data refers to, e.g. "Bank Example Ltd.". The inputs are the data properties, the entity potential identifiers and the recommended entity type and domain
- Knowledge Base Storage: This module provides a unique interface for the data and knowledge stored in the system. The main advantage of this unified Knowledge Base is that it manages the storage based on the data features, e.g. a corporate network dataset would be stored in a property graph datastore such as Neo4j
Fujitsu is the leading Japanese information and communication technology (ICT) company, offering a full range of technology products, solutions, and services. Approximately 140,000 Fujitsu people support customers in more than 100 countries. We use our experience and the power of ICT to shape the future of society with our customers. Fujitsu Limited (TSE: 6702) reported consolidated revenues of 4.1 trillion yen (US $39 billion) for the fiscal year ended March 31, 2018. For more information, please see http://www.fujitsu.com/global/.
About Fujitsu Laboratories of Europe
Established in 2001 and with an active presence in Europe since 1990, Fujitsu Laboratories of Europe Limited represents Fujitsu Laboratories across EMEIA, focusing on regional initiatives that reflect the diverse mix of countries and ideologies. Fujitsu Laboratories of Europe is focused on the creation of cutting-edge solutions that benefit society, adopting a co-creation strategy and working with customers, collaboration partners and society as a whole to pioneer a new generation of user-centric applications and services underpinned by creative information analytics. As one of Fujitsu’s global centers of excellence for AI, its work encompasses security, social innovation, manufacturing, ethics, and high performance computing applications. For more information, please seehttp://www.fujitsu.com/uk/fle/.
Garrett Axford Ltd (on behalf of Fujitsu Laboratories of Europe Ltd)
Phone: +44 1903 854900
Company:Garrett Axford Ltd
Georgina Garrett, Director
All company or product names mentioned herein are trademarks or registered trademarks of their respective owners. Information provided in this press release is accurate at time of publication and is subject to change without advance notice.
Date: 12 June, 2018
Fujitsu Laboratories of Europe