Outstanding Performance for Fujitsu’s Behavior Recognition Technology on the World Stage
August 26, 2021
JapaneseBehavior recognition is playing an increasingly important role in modern society. This technology is able to recognize human actions and behaviors from videos, and has many useful applications, including helping improve customer service in retail environments and identifying suspicious behavior in public places.
Fujitsu is one of the leading developers of behavior recognition, developing the Actlyzer
AI technology for video-based behavioral analysis. Actlyzer can recognize complex behaviors based on a combination of multiple actions, without having to rely on large amounts of training data – making it extremely efficient and fast to deploy.
Development Team: Zhiming, Tan (upper left), Xiantan, Zhu (upper right), Yuya Obinata (bottom left), Takuma Yamamoto (bottom right)
Our Actlyzer technology was recently put to the test during the annual AVA Challenge in June, winning an impressive second place in the AVA-Kinetics task. This is an action recognition competition held as part of the CVPR conference, one of the world’s foremost AI events.
The AVA Challenge is recognized internationally as the premier behavior recognition competition, and since 2018, it has attracted the world's leading companies and universities in the AI field. This year, Fujitsu secured a second place ranking out of 11 teams in total, moving up from last year's 3rd place.
Background to the Behavior Recognition Competition
- Challenge Name: AVA Challenge 2021 (AVA-Kinetics Task)
- Date
- June 19, 2021
- Location
- Online
- Results:https://research.google.com/ava/challenge.html
The AVA-Kinetics task involves recognizing various actions in a video, using a large data set containing more than 230,000 video data published by Google AI and DeepMind. This task is all about actions that are important for identifying the relationships between people and things, including more complex elements such as “talking to people” and “touching things” as well as just “walking” and “standing”.
AlphAction is a conventional method that captures these relationships. AlphAction recognizes human behavior by focusing on a person or object extracted from a video as a feature of the relationship. Importantly, however, AlphAction does not use take into account the relationship derived from any background information in the video (namely the area that does not include people or objects), and this limits its performance quite significantly. For example, in the case of someone swimming, AlphAction can only recognize the behavior as standing rather than swimming.
Fujitsu’s approach involves extracting specific context features from the video material, and then fusing this with AlphAction’s features. This approach allows us to recognize behavior that is based on the relationship between the person and the background information. For this year’s competition, we also thoroughly explored the optimal AI model structure for human and object detection, as a novel way of improving accuracy.
Figure 1 An overview of the behavior recognition method proposed by Fujitsu.
The Inside View from our Development Team
・Fujitsu Research and Development Center (FRDC) – Zhiming Tan, Project Manager
Our aim was to benchmark our behavior analysis technology using the AVA-Kinetics Crossover task in the AVA Challenge 2021. The task of the challenge is to locate the action of each person in the video. It has been a valuable experience, and we will be applying the lessons and skills learned in our future research and developments.
・Fujitsu Research and Development Center (FRDC) - Xiantan Zhu, Researcher
Our method involves fusing context-related features extracted from the video sequences with other relevant features, such as those relating to people and objects. For some categories, such as “drinking”, this relies on the objects around the person (for example, a cup). In order to do this, we extend the detection area around a person so that any related objects are included. We then apply what is called the “model ensemble” method, which is an approach to combine multiple models in a prediction process.
This novel combination is why, I believe, we won second place in the AVA-Kinetics task.
・Fujitsu Research Advanced Converging Technologies Laboratories - Yuya Obinata, Researcher
We were determined to improve our third place ranking from last year. We investigated the latest trends and techniques on behavior recognition technology from recently-published papers, and worked creatively together as a team to come up with exciting new ideas.
I believe that we achieved our goal as a result of this approach, combining all of our team members’ knowledge to improve our techniques. We are fired up to take on more ambitious challenges and to achieve even higher goals in the next round.
・Fujitsu Research Advanced Converging Technologies Laboratories - Takuma Yamamoto, Researcher
As in the previous year, we had a joint team from Japan and China but there was a big difference in how we were able to work together this year. Unlike in 2020 when we could have face-to-face meetings at our respective offices, we had to rely on online discussions. Despite these difficulties, we were able to build an even stronger collaboration and find new ways of combining our ideas – culminating in this year’s success. Next time, we'll aim even higher – going for first place!
For more information on this topic
fj-actlylzer-contact@dl.jp.fujitsu.com
Please note that we would like to ask the people who reside in EEA (European Economic Area) to contact us at the following address.
Ask Fujitsu
Tel: +44-12-3579-7711
http://www.fujitsu.com/uk/contact/index.html
Fujitsu, London Office
Address :22 Baker Street
London United Kingdom
W1U 3BW
Share
Recommend
Connect with Fujitsu Research
