Fujitsu Develops Traffic-Video-Analysis Technology Based on Image Recognition and Machine Learning

Kawasaki, Japan, and Beijing, October 18, 2016

Fujitsu Laboratories Ltd. and Fujitsu Research and Development Center Co., Ltd. today announced the joint development of a technology that utilizes image processing and machine-learning to analyze surveillance camera images of traffic, with high accuracy and in real time, to recognize traffic conditions such as congestion and accidents, as well as violations.

The technology achieves high-precision traffic-video analysis by combining two technologies. The first is a technology that analyzes the images from surveillance cameras installed along highways and streets, automatically grouping characteristics that can lead to recognition errors, such as changes in lighting and environmental factors including night and fog, and images from cameras that have been similarly positioned. This leads to efficient machine learning, increasing recognition accuracy. The second is a technology that analyzes moving objects, such as vehicles and people, and efficiently identifies complex incidents such as accidents, while minimizing computational demands.

In field trials of this technology conducted in cities around China in collaboration with the Tsingha University Suzhou Automobile Research Institute (TSARI), it was found that 11 types of incidents of interest, such as traffic accidents and violations, were recognized with accuracy levels of 90-95%.

Even when used with existing cameras that do not have advanced image-correction features, this technology can be used to deliver a highly accurate, low-cost monitoring system that can automatically assess traffic conditions, apply traffic-flow controls and analysis to reduce congestion, and take quick action in response to accidents and traffic violations.

Background

With populations concentrating in cities, traffic has become congested, leading to chronic traffic jams, noise, air pollution, and other social issues. In China, especially, the rapid increase in the rate of car ownership has intensified security and safety problems, such as traffic violations and fatal accidents. As a result, in recent years there are high expectations that initiatives involving remote, centralized management of large-scale urban deployments of surveillance cameras can be used for objectives such as improved traffic safety, reduced pollution and reduced congestion.

Issues

When monitoring surveillance cameras installed over a large area, it is important to quickly and correctly extract the information needed from a huge volume of imagery, and convey that to the relevant people. The issue with traffic-monitoring technologies that use conventional image recognition is that they are highly susceptible to the influence of a variety of environmental factors such as light sources, including headlights, sunlight, and shadows. As a result, there were limits to how much recognition accuracy could be improved when using existing cameras for analysis using video recognition. This is because it is difficult to adjust the cameras, their position, and direction in accordance with constantly changing environmental variations. In addition, it was also difficult to efficiently and accurately recognize such varied and complex incidents as traffic accidents and violations.

About the Technology

Fujitsu Laboratories and Fujitsu Research and Development Center have developed highly accurate vehicle-recognition technology that tolerates changes in the surrounding environment, including changes in light, time of day, and fog, and another technology for efficiently identifying complex incidents, such as traffic accidents.

Key features of the technology are as follows:

1. Highly accurate recognition technology that learns from images and classifies environment conditions

This technology automatically classifies similar data based on each environmental-factor-related characteristic, such as shadows cast by sunlight, night, fog, and camera placement, groups the data, learns from, and identifies the images of each group to improve the recognition accuracy of target vehicles.

2. Efficient technology for identifying complex traffic events based on motion analyzing

Analyzing complex events such as traffic accidents require that the combined motions of multiple related moving objects be analyzed, but performing this kind of analysis continuously requires enormous computations. This technology quantifies a scene's anomaly score based on changes in vehicle speeds and directions, which could indicate the occurrence of an anomalous incident, and only when this number exceeds a certain amount does it analyze multiple nearby moving bodies. This has the effect of keeping computational load to a minimum (Figure 3).

Together with the newly developed technology, and combining the propriety technology⁽¹⁾ developed by Fujitsu Research and Development Center that clarifies images from polluted environments, it becomes possible to offer a low-cost, high-precision visual monitoring system.

Results

In a field trial that applied this technology to software designed with the ability to recognize the 11 events described below, it achieved recognition accuracy of 90-95%, an improvement of 5-10% compared to before the new technology was applied. With this newly developed software, by reducing computationally intensive image processing with proprietary high-speed algorithms, a single commercially available PC is able to take input from four surveillance cameras, and is able to simultaneously perform recognition processing on seven events from each camera.

1 Traffic jam detection; 2 Anomaly detection (traffic accident, etc.); 3 Driving in reserved emergency lane detection; 4 Illegal parking detection; 5 Traffic-flow statistics summarization; 6 Wrong way driving detection; 7 Vehicle on road detection; 8 Street occupancy detection; 9 Smoke detection; 10 Fog/haze detection; 11 Congestion index summarization

Combining these technologies enables automatic monitoring of traffic images using imagery from surveillance cameras already installed over a large area, which would be difficult to achieve with human visual monitoring. This holds the prospect for bringing about a safer and more secure automobile society at low cost.

Future Plans

Fujitsu Laboratories and Fujitsu Research and Development Center are working to increase the accuracy and incidents recognizable with this technology, and plans to continue field trials jointly with TSARI. Also, by combining this technology with FUJITSU Intelligent Society Solution SPATIOWL, Fujitsu Limited's cloud service that utilizes location information, after fiscal 2016 the companies aim to roll out a service in China that recognizes incidents occurring over a wide area in real time and instantly plot them on a map. Following that, they aim to extend to other regions, including Japan.