GTM-MQNC2Z4
Skip to main content
  1. Home >
  2. About Fujitsu >
  3. Resource Center >
  4. News >
  5. Press releases >
  6. 2013 >
  7. Fujitsu Develops 3D Gesture Recognition Technology Capable of Complex Tasks with a Single PC or Tablet Camera

Fujitsu Develops 3D Gesture Recognition Technology Capable of Complex Tasks with a Single PC or Tablet Camera

Improved functionality via detection of "push and pull" hand movement, in addition to vertical and horizontal motion

Fujitsu Laboratories Ltd.,Fujitsu Research and Development Center Co., Ltd.

Kawasaki, Japan, and Beijing, China, May 20, 2013

Fujitsu Laboratories Limited and Fujitsu Research and Development Center Co., Ltd. of China today announced the development of 3D hand gesture recognition technology that, using video captured on a single PC or tablet camera, can detect not only vertical and horizontal hand motion, but also the forward and backward movements of increasing or decreasing distance of a hand from the display.

With existing hand gesture recognition technology it has been possible to detect hand motions along the vertical and horizontal axes, which represent operations similar to moving a mouse cursor. However, detecting forward and backward hand motions—changing distance to perform the same function as a mouse click—has proven difficult. Fujitsu Laboratories and Fujitsu R&D Center have developed technology for accurately and quickly detecting the area of a hand based on models of palms, as well as a technology that detects forward and backward hand motions. This has made it possible for users to perform click operations through intuitive "push" gestures.

The new technology makes it possible to use hand gestures to perform a host of complex operations. For instance, a user can wave his or her hand up and down to make a menu selection on a PC and then confirm the selection with a "push" gesture, even from a distance. Maps and other images can also be expanded or shrunk with "push" and "pull" gestures.

Details of the new technology will be announced at the International Conference on Machine Vision Applications (MVA) 2013, to be held starting May 20 at Ritsumeikan University in Kyoto, Japan.

Development Background

In recent years, attention has been gathering around hand gesture recognition technologies, which enable devices to be controlled through natural human movements. This approach to operating a device through hand motions that are detected with a camera is typically employed in situations when a user is physically separated from a PC or tablet device, when the user's hands are wet or dirty, or when the user does not want to directly touch a publically accessible device due to hygiene concerns. When attempting to substitute hand gestures for mouse operations, it has been possible to detect vertical and horizontal hand motions that are similar to how a mouse cursor moves, but detecting depth, as in forward and backward hand motions, which are equivalent to mouse clicks, has been difficult.

Technological Issues

Detecting forward and backward hand motions with a single camera relies on changes in the area of the user's palm. When the user is wearing short-sleeved clothing, however, or when the background is a color similar to the user's skin tone, it is difficult to distinguish between the user's palm and the background. As a result, it has been impossible to accurately extract the area of the user's hand from the background. Until now, methods using distance sensors that measure distance or multiple cameras have been used to detect forward and backward motion, but the equipment required for these approaches has made them quite costly.

About the New Technology

Fujitsu Laboratories and Fujitsu R&D Center have developed technology that can accurately extract the area of the user's hand from video captured on a single camera, thereby enabling constantly changing differences in the user's hand size to be reliably tracked. Key features of the new technology are as follows.

1. Hand area detection technology based on palm models

The hand is initially detected in a captured image, from which it is clipped out using block units of set sizes. Calculations are then made to determine whether or not the hand's characteristics (a palm model) exist as pre-registered data. The actual position and size of the hand is determined by performing this operation against the backdrop of the overall image while changing the position and size of the blocks. Next, the hand's area is extracted by taking the determined hand position and size and simultaneously changing the color threshold of certain parts to express the hand's original image. These operations ensure stable detection of the hand area.

Figure 1: Comparison with Conventional MethodsFigure 1: Comparison with Conventional Methods

2. Depth detection technology based on changes in hand size

This technology is able to accurately estimate depth based on the continuity of the user's hand motions, including constantly changing features such as hand size, angle and central location. This allows "push" and "pull" motions to be detected with roughly 90% accuracy.

Figure 2: An operation that makes use of depth due to this technologyFigure 2: An operation that makes use of depth due to this technology

Results

There are many possible uses for this newly developed technology. For instance, a user can wave his or her hand up and down to make a menu selection on a device and then confirm the selection with a "push" gesture. Movies, music, sports, and other kinds of content can be selected by waving a hand left and right, and then confirmed with a "push" gesture. Maps and other images can also be expanded or shrunk with a "push" or "pull," thereby making possible even more complex operations via hand gestures. In addition, since motion can be detected with a single camera, equipment can be produced at a low cost.

Figure 3: Example Usage ScenariosFigure 3: Example Usage Scenarios

Future Development

With the aim of commercialization in fiscal 2014, Fujitsu Laboratories plans to evaluate the technology's usability while working to improve the accuracy of the hand detection and recognition technology.

About Fujitsu

Fujitsu is the leading Japanese information and communication technology (ICT) company offering a full range of technology products, solutions and services. Approximately 170,000 Fujitsu people support customers in more than 100 countries. We use our experience and the power of ICT to shape the future of society with our customers. Fujitsu Limited (TSE:6702) reported consolidated revenues of 4.4 trillion yen (US$47 billion) for the fiscal year ended March 31, 2013. For more information, please see http://www.fujitsu.com.

About Fujitsu Laboratories

Founded in 1968 as a wholly owned subsidiary of Fujitsu Limited, Fujitsu Laboratories Limited is one of the premier research centers in the world. With a global network of laboratories in Japan, China, the United States and Europe, the organization conducts a wide range of basic and applied research in the areas of Next-generation Services, Computer Servers, Networks, Electronic Devices and Advanced Materials. For more information, please see: http://jp.fujitsu.com/labs/en.

About Fujitsu Research and Development Center

Established in 1998, Fujitsu Research and Development Center Co., Ltd. is a wholly owned R&D center of Fujitsu Limited, located in Beijing. The center's research areas cover the major business fields of the Fujitsu Group, including information processing, telecommunications, semiconductors, and software and services.For more information, please visit: http://cn.fujitsu.com/frdc/.

Press Contacts

Public and Investor Relations Division
Inquiries

Company:Fujitsu Limited

Technical Contacts

Media Processing Systems Laboratories
Image Computing Lab.

E-mail: E-mail: depth-gesture@ml.labs.fujitsu.com
Company:Fujitsu Laboratories Ltd.


All company or product names mentioned herein are trademarks or registered trademarks of their respective owners. Information provided in this press release is accurate at time of publication and is subject to change without advance notice.

This press release has been revised as of December 17, 2018.

Date: 20 May, 2013
City: Kawasaki, Japan, and Beijing, China
Company: Fujitsu Laboratories Ltd., Fujitsu Research and Development Center Co., Ltd., , , , , , , , ,

Services & Products

Corporate Information

Country Selector

Global

Change

World Map