Fujitsu Develops Technology that Quickly Tracks Changes Made to Large-Scale Open Source Software

Kawasaki, Japan, April 27, 2016

Fujitsu Laboratories Ltd. today announced the development of technology to quickly identify changed locations in source code, such as added functions, to enable timely development using large scale open source software.

Open source software (OSS) can be easily used by anyone around the world, and its source code - what might be called its blueprints - can be repeatedly changed, adding and deleting functions, without detailed records of what was changed. In this situation, software developers working on OSS will often independently add their own changes along the way, as they see fit. Understanding where changes have been made in the OSS source code is a necessary, yet time-consuming process.

Fujitsu Laboratories has now developed a technology that can quickly compare changes in the source code by analyzing functions' call paths from the source code. This technology was able to identify approximately 400 changed locations in a program with 3.8 million lines of code in about 1/4th of the time of existing source code text comparison methods.

Going forward, Fujitsu Laboratories expects that by quickly identifying changes in large-scale OSS, this technology will be useful in speeding up OSS development.

Development Background

OSS is developed through collaborative work among developers around the world, and can be easily used by anyone. This means it is often actively used in developing software when quickly building a system to meet the needs of customers.

Because the source code is often changed by many developers, such as when adding features or fixing bugs, developers using OSS have a need to understand what parts have been changed as quickly as possible within their limited development period.

Issues

In development that uses OSS, even while changes are being made to the source code by a particular user, there are frequently many changes being made in parallel, such as additional functions and bug fixes, by numerous other open source developers. This means that a user needs to confirm the source code changes, before and after OSS changes, as relates to the source code that a user is working on. Because these changes would have to be checked across tens of thousands of files of source code by text differences, or by visually checking the differences in the structure of related function calls, the process would require an enormous amount of work.

In addition, during manual, detailed checks of the source code, such as after identifying differences, an effective method is specifying part of the name of a function, while interactively narrowing down parts to check using a partial search function that displays candidates that match that part on a case-by-case basis. In large scale OSS, however, there are a huge number of functions that need to be checked, and there are many functions which share similar names, in particular method (function) names in classes, so the time taken for search processing was a problem.

Newly Developed Technology

Fujitsu Laboratories developed the following two technologies in order to efficiently check changes when large-scale OSS is updated.

1. Technology that quickly compares function call paths and displays them graphically

When tracking changes in source code between different versions of the same software, there is a tendency to focus on the call path from a specific function. Focusing on this issue, Fujitsu Laboratories developed a technology that, by comparing the differences between the two source code versions at each point on a function call path that starts from the specific function, greatly refines the targets for comparison and displays them graphically in a format that can be understood at a glance. This resulted in a 44% reduction of time required to extract differences, compared with existing combined comparisons.

2. Technology to efficiently find similar names

Fujitsu Laboratories focused on the fact that in large-scale OSS, there are many functions with similar names. In input for function name search, by newly developing a data structure that enumerates and efficiently stores, in advance, the corresponding search results for various combinations of input characters, it was possible to search matches for those parts that had good responsiveness.

Results

Using these newly developed technologies, it has become possible to repeatedly search for the changed locations in OSS around relevant function call paths. As a result, in a test case where approximately 400 locations were changed in a program with 3.8 million lines of code, the time taken to find the locations that had an impact was massively reduced to one-fourth of previous times, from about 26 hours with existing text comparison methods to about 6.5 hours. Fujitsu Laboratories expects that this will not only reduce development costs, but will also promote the timely use of OSS.

Future Plans

Fujitsu Laboratories will continue to expand supported languages, such as scripting languages, and will aim for commercialization during fiscal 2016.