At the MM Research Institute Award 2021, Fujitsu won the Judges’ Award for our Judging Support System. MM Resarch Institute Award was created to promote the development of market and industries of ICT field. Judges Award is a category which is under the MM Research Institute Award 2021. Head of Sports Business Division Hidenori Fujiwara (“Fujiwara” below) talked about what the award meant and future prospects.
- Contents
-
- A Focus on Digitalization of Human Movement Beginning with Artistic Gymnastics
- Judging Support System Repeatedly Validated through On-site Testing
- Award Win Based on ICT Evaluation of Sport Fairness and Equality
- The Desire to Make People Healthy through Sports
- Generating Future Social Change with Diversity in Fields and Partners
A Focus on Digitalization of Human Movement Beginning with Artistic Gymnastics
Why rework the judging system for Artistic Gymnastics with AI?
Fujiwara: Six years ago, I had a conversation with International Gymnastics Federation President Watanabe (secretary general of Japan Gymnastics Association at the time) who told me, “Robots will be doing the judging in the 21st century.” It was a shock at first, but that was what kickstarted our efforts.
People may wonder why we chose gymnastics, but there are no other competitive sports that are as popular or as varied in physical movement as gymnastics. We thought that if we could achieve the digitalization of detailed human movement, then perhaps we could apply that technology to several other fields. For that, our 3D sensor and AI technology were essential. We set up several 3D sensors as thick as magazines, collected detailed data of the human body, and digitalized the skeleton and its movements through AI in real time.
This digitized human movement is matched with a dictionary of gymnastics elements created by quantifying ambiguous parts of the gymnastics rules and incorporating the performances of top gymnasts.This system was developed in conjunction with the International Gymnastics Federation.
Judging Support System Repeatedly Validated through On-site Testing
What kind of hurdles did you face before you could use the Judging Support System in international competition?
Fujiwara: We actually faced many difficulties before it could be used in formal competition.
First, there was the laser sensor. Right when we were going to test the 3D sensor at the competition venue, the sensor’s data became disordered. When we examined it, we discovered that the powdery chalk that gymnasts use as a grip to prevent themselves from slipping had been sucked into the fan. We wouldn’t have discovered that had we not performed on-site testing.
Also, AI was supposed to measure the framework for the basis of the sensor’s data, but we faced an obstacle there, too. At first, we were trying to pick out human movements from supposedly fixed equipment, but actually, there’s also moving equipment such as the still rings typically used in men’s gymnastics. It was necessary to distinguish whether the movements were resulting from humans or the equipment, so it forced us to study more meticulously. Furthermore, because the transitions from technique to technique were seamless, it was also tough to discern the points where each technique started and ended. It was especially difficult to make such distinctions for the pommel horse, to the point where people would say, “Why would you start with the most difficult event?”
However, Fujitsu has a culture of starting from the most difficult spots to challenge ourselves, and we were resolved to make full use of the various technologies available to us. We couldn’t become complacent with ICT, so we listened to the voices of the organizations, judges, gymnasts, and others involved in the sport to continuously improve until it could be used in international competition.
Award Win Based on ICT Evaluation of Sport Fairness and Equality
We heard that the Judging Support System was awarded based upon its ability to realize fairness and equality in the sport as well as its expected applicability in other fields. On that point, what do you make of the win?
Fujiwara: As you know, gymnasts compete on 0.01 denominations. There cannot be any misjudgments by the Judging Support System. To secure its level of quality, we worked on it with international organizations. I think our win can be attributed to the high evaluations received even from people overseas regarding the accuracy and credibility of judgements and our contribution to making fairness and equality more feasible in sports.
In the future, events could be held simultaneously by making judgements online through this technology that are based upon unified standards in several regions. In this way, people from countries unable to participate for various reasons at venues in person will gain the opportunity to participate equally.
Right photo, in order from left: Technology leader Kazuo Sasaki, Yuko Yamabayashi, Yuko Oiwa, project leader Hidenori Fujiwara
The Desire to Make People Healthy through Sports
If we can capitalize on this ability for human movement to be measured in real time thanks to the Judging Support System acknowledged by global gymnastics associations, can we expect this technology to be further applied to other fields in the future?
Fujiwara: Now, as gymnastics equipment gradually evolves, performances can reach even greater heights, and gymnasts are attempting techniques with higher degrees of difficulty. But that also means more burden is put on the body that allows for injuries to come more easily, thus shortening the athlete’s life as a gymnast. With the use of this system, techniques that strain the body can be reexamined, reasonable movements can be sought out, and more scientific coaching can be provided. Consequently, the gymnast’s career span can be lengthened and athletes can continue competing at an older age. I think it would be great if there were senior gymnasts who could compete based on beauty rather than in pursuit of more advanced techniques. It will start with gymnastics, but there have been demands from various sports outside of gymnastics as well, and it could also be helpful in training as well. We have already provided form visualization technology that applies this technology to our joint development partner AIGIA and are commercializing solutions for scientifically self-checking golf swings.
Through the digitalization of human movement with our 3D sensing/AI technology, training can be done scientifically, and spectators can be provided with fresh and rich content. Additionally, the digitalized data of human movement could also be applied to anime or games. I believe our AI is not taking jobs away from humans, but rather enhancing human abilities and further enriching experiences. Applications in a diversity of fields will progress and become commonplace before we know it, and our world will become one of digital transformation.
Generating Future Social Change with Diversity in Fields and Partners
Could you tell us more about future prospects?
Fujiwara: Fujitsu is driven by our purpose to solve social issues with digital technology. In addition to our main efforts concentrated on “Healthy Living (supporting the well-being of people),” we are also being propelled to develop this system by our goal to realize our SDG3 to provide all people with “good health and well-being.”
For instance, those of us in the working generation will be able to perform self-checks online by objectively observing the movements of the body. Videos do not provide feedback on the way we move our bodies to match what we see on screen, but this system would give solid feedback and offer countermeasures to problematic lower back pain caused by working at home. The elderly could have their movements sensed at hospitals. The way they walk could be observed and places of poor mobility could be quickly identified and treated. Additionally, the younger generations who wish to be taught by teachers overseas could still receive such education remotely without actually going to the site. Simply traveling to and from countries has become difficult due to the coronavirus crisis, but correct training provided remotely can still be possible through the application of this system.
This system can be applied even in the cultural sphere, starting with the movements of traditional Japanese Noh plays and the production of action films. In regard to film production, motion capture is currently used to record movements, but applying such a system is a major undertaking. Our 3D sensing/AI technology can easily track movements without markers needing to be placed on the body. It would also be helpful for the passing down of traditional culture through the creation of an archive of movements. By developing the technology gained through the judging system for Artistic Gymnastics and applying it to treatment and aid for the disabled, advancement of health, and all fields including industrial and cultural, we are closing in on achieving our SDG3 to provide all people with “good health and well-being.”
Going forward, we will accelerate our actions to resolve social issues through digital technology. We encourage those who have read this article and feel that this system could be applied to their field to reach out to us. Together, through the power of digitalization, we aim to realize a world where people can live healthily and happily.
Future developments of 3D sensing/AI technology
