Fujitsu Develops Technology for Instantaneous Searches of a Target Image from a Massive Volume of Images

Kawasaki, Japan, February 02, 2016

Fujitsu Laboratories Ltd. today announced that it has developed technology that is installed on a server to specialize features with a specific functionality. In this way it is possible to search at high speed through an enormous archive to find even partial matches of a designated still image. It has confirmed that, compared to searches on general-purpose servers, the new approach resulted in an over fifty-fold improvement in processing time.

Companies and other organizations routinely accumulate presentations and other materials. To thoroughly search through these materials, photos and illustrations to find a desired image, the problem has been that it requires time to search enormous volumes.

Now, Fujitsu Laboratories has developed technology that dramatically improves server performance by configuring an accelerator that performs the necessary processing of image searches on an FPGA⁽¹⁾ and efficiently linking it with software running on the CPU, creating a high-speed image search function. Using a prototype server, Fujitsu Laboratories confirmed that it could search through a database of over 10,000 images and retrieve matches of any desired portion in about one second.

With this technology, companies and other organizations will be able to search through the large volume of image data they have stored and instantaneously find and reuse target images, enabling on-site work to be carried out more efficiently.

A server equipped with this technology will be exhibited at the Fujitsu North America Technology Forum 2016, which is scheduled to open on February 16 in Santa Clara, California. In addition, details of the technology will be announced at the IEEE Winter Conference on Applications of Computer Vision 2016, starting March 7 in Lake Placid, New York.

Development Background

Companies and other organizations produce and store numerous documents that include illustrations and photos for use in presentations and other materials as part of their everyday operations. If a target image could effectively be found out of the large volume of data stored, it could be expected to improve work efficiency, so image search technology that can quickly retrieve matching images is attracting attention.

Issues

When a person searches for an image, it is often prompted by a vague recollection or an associated image. There is a technology for finding images based on matches with a part of the query image itself, called partial-image search. This can be used to find similar or related images, or with new hints to spark the user's memory through associations, to more intuitively search for the target image. But partial-image searching requires massive computing resources, as comparisons of partial images need to be made between all the images. On a general purpose server, finding a target image from among 10,000 images can take over one minute. Search speed can be improved by using multiple servers in parallel or with distributed processing, but there are drawbacks in the form of great costs for energy consumption and equipment.

About the Technology

Fujitsu Laboratories has developed technology that dramatically accelerates search speeds by offloading the computing elements able to perform massive parallel processing as part of the processes that conventionally are done on a CPU to an FPGA, and furthermore, by optimizing the process scheduling, with a focus on data transfers, so that these processing units could be run in parallel at high utilization rates.

This technology is based on the "domain-specific server" approach, which dramatically improves server performance using a hardware configuration optimized for this particular application domain along with software optimized for key functions.

To achieve high performance using the FPGA, the keys are to densely pack processing units on the FPGA that efficiently executes complex processes, and to operate the processing units so that they are kept supplied with data and not left idle.

Key features of the technology are as follows.

1. Intensively conducting primary parts of the processing of the partial-image search algorithm on the FPGA

Fujitsu Laboratories analyzed the partial-image search algorithm in detail and shifted the feature extraction process and matching process from the overall search flow, and relocated these processes to the FPGA. Because the FPGA has a flexible design in which its processing units and parallelization can be organized to fit the process characteristics in a way that CPUs and GPUs cannot, compact and efficient processing units were designed for each of these two processes and incorporated at high density and with a high degree of parallelization into the FPGA (Figure 1). Efficient coordination between the high-performance FPGA processing units and the software running on the CPU results in high-speed partial-image searches.

2. Scheduling technology for process-sequence reordering to maintain high run rates

In image processing, especially when it involves large volumes of image data, it takes time to read data from memory, where it is temporarily stored. So it is possible to shorten read latency by reading in advance the next required data currently being processed. But the next piece of data to work on may change depending on the results of the current process, negating the benefit of reading ahead. With a high degree of parallelization, and the simultaneous use of a large volume of data, the penalty for a wasted read-ahead is considerable. Fujitsu Laboratories has developed a scheduling technology in which, while read-aheads are being performed, the order of processing can be switched, depending on the circumstances, and thereby avoid wasting the read operation data. This resulted in good processing performance (Figure 2).

Results

A prototype server using this newly developed technology was demonstrated to be able to search a database of over 10,000 images and finish retrieving matches of any desired portion of an image in roughly one second, performing partial-image searches over 50 times faster than on general-purpose servers (Figure 3a). Compared to general-purpose servers with comparable performance, the prototype required less than 1/30th the electrical power and less than 1/50th the cubic volume of space as multiple general-purpose servers.

It is anticipated that by using this technology, a company or other organization could, for example, instantaneously discover target images from a large set of documents including the images it has, and then use those results to refine the search accordingly. This would allow it to efficiently reuse the enormous volume of images it has accumulated to date, and enable quick searches from target documents and more efficient production work (Figure 3b).

Future Plans

Fujitsu Laboratories plans to move ahead with development of a system using this high-speed image-search technology, with the goal of a practical implementation during fiscal 2016. In addition, outside of just image processing, it also plans to move forward with R&D on domain-specific servers for other application domains where there is a desire for improvements in server processing performance.