Fujitsu’s unique camera position estimation technology overcomes challenging conditions for highly accurate human identification

At Fujitsu, we have developed a new camera position estimation technology (hereinafter, referred to as our technology), which uses an individual’s unique characteristic, namely their height, in order to estimate the camera position and orientation. This is achieved irrespective of whether a reference object of known definitive length is captured as well. Our technology also can estimate a person’s height and accurately identify the person, even if dark conditions or poor lighting make that person look very different. Using our technology, it is now possible automatically to identify the same person with high accuracy, succeeding with images that defeated identification when using existing methods.

We recently presented our results at ICEET 2021, one of the leading international conferences involving image processing and other engineering topics.

“Body height” as the key to unlock accurate identification

When identifying a fugitive criminal from video footage, investigators generally look to extract the criminal’s visual characteristics such as hairstyle, clothing and belongings based on eyewitness testimony, looking through the huge amount of video footage from surveillance cameras installed in multiple locations. By associating these features with the videos, investigators can check the criminal’s path. However, crimes often occur at night, and even if the criminal wears exactly the same clothes, the color may appear to be different due to the lighting. This makes it much harder for experts to identify the candidate criminal, involving mapping each identified characteristics of the criminal to each person shown in the videos. This requires considerable effort to identify the criminal from the huge amount of video data.

Our breakthrough was to focus on one element that doesn’t change, namely a person’s height.
There are a number of conventional methods used to estimate the height of a person in video images. These methods rely on knowing certain specific camera settings in order to estimate a person’s height accurately such as its position and angles. To ascertain the camera settings, conventional methods capture markers such as the patterns of a pedestrian crossing or safety fences as a known reference of objects’ length. In order to simplify these preliminary tasks, a combination of special cameras and sensors can be used, but the need for extremely precise camera alignment and expert set-up has proved to be a major barrier to their practical application.

Fujitsu’s camera positioning estimation technology eliminates the needs for length reference markers

We identified that using someone’s height as a reference of unchanging information could provide the key to calculate camera positions, and therefore accurately estimate a person’s height. In the real world, a person’s height never changes, even when it is distorted for example in a video image based on the positional relationship between the camera and the person. By concentrating on this particular relationship, we have successfully developed a method to calculate both the position and installation angle of a camera from a geometric relationship. In addition, we use a person’s skeletal data as identified by our Actlyzer behavioral analysis AI technology so that an individual’s unique height will always be confirmed in the video footage. By using our technology, the camera positions and angle can be accurately estimated even when the camera is already mounted. Once the camera position and angle are estimated, even if the installation environment changes, the height of the person can be estimated with high accuracy based on the positional relationship between the camera and the person.

In the past, when the same person is captured on multiple cameras at different times, changes in the color of clothing or lighting can negatively impact conventional technology’s ability to determine if it is indeed the same person simply by comparing their appearance and features. However, thanks to our technique of comparing this data with the estimated height of the person, it is now possible to identify an individual from multiple cameras with high accuracy, even if there is a change in clothing or lighting.

Schematic images of our developed technology

Field tests to evaluate our technology confirmed that the same person was identified even if they changed their clothing, and the accuracy of the matching was the same as when the clothes were the same.
As our technology can be applied to already installed surveillance cameras, we believe that it can be rapidly introduced into the field, and we plan to use this technology to realize a safe, secure and resilient Trusted Society.

About ICEET 2021

Details of this technology was presented at ICEET 2021, an international conference in the field of engineering including image processing.

・Conference：International Conference on Engineering and Emerging Technologies (ICEET )
・Date：October 27 – 28, 2021
・Title：Improving Person Re-Identification Based on Human Height Information
・Authors：Takahiro Yoshioka (Fujitsu), Shuji Awai (Fujitsu), and Takeshi Konno (Fujitsu)

The Inside View from our Development Team

・Fujitsu Research Advanced Converging Technologies Laboratories – Takahiro Yoshioka, Researcher

Over the years, many methods for estimating camera positions have been proposed, but to date it has proved difficult to identify each camera position across the huge variety of images from for example surveillance cameras. We believe that our technology can solve this problem and is an important opportunity to make use of the inherent information about people contained in images.
There are many obstacles to overcome in this discipline, such as the issue of privacy. We are working to refine this technology and apply it to a variety of applications, while continuously contributing to the creation of a safe and secure society.

・Fujitsu Research Advanced Converging Technologies Laboratories – Takeshi Konno, Project Manager

In order to realize a safe and secure society as quickly as possible, it is necessary to develop technologies that can be used with existing surveillance cameras, which are already installed. As our technology can be used with general surveillance cameras, it can be applied to many situations such as detecting lost children in stores or suspicious behavior. We will continue to contribute to the realization of Fujitsu’s Trusted Society through our ongoing research into AI image analysis technology.

Looking Ahead

We plan to use this technology in FY2022 as part of a new person identification technology that supports Fujitsu’s family of “FUJITSU Human Centric AI Zinrai” AI technologies.

Related Links

・Fujitsu Develops New "Actlyzer" AI Technology for Video-Based Behavioral Analysis[Fujitsu Global Site] （November 25, 2019 Press Released）

・Introduction of GREENAGES Citywide Surveillance, where Fujitsu's behavioral analysis technology Actlyzer was introduced[Fujitsu Global Site]

For more information on this topic

Fujitsu Research Converging Technology Laboratory
fj-ctfund-contact@dl.jp.fujitsu.com