Protecting Logistics Drivers Through AI Detection

With online shopping now commonplace, the logistics industry has been experiencing an increase in the volume of transactions year by year. This has led to an increase in the number of occupational injuries while handling cargo, especially accidents caused by forklifts such as collisions and damage to cargo. Suntory Logistics, which serves as the backbone of Suntory Group's logistics operations, has worked together with Fujitsu to develop and implement the industry's first AI detection system for forklift operations. This technology enables Suntory Logistics to promote safe and secure operations by facilitating the verification and evaluation of forklift operations. How does this AI technology work, and how did it solve problems on site? Fujitsu Evangelist Koji Takeda headed to a Suntory Logistics warehouse to interview frontline logistics workers and explore this innovative technology.

Contents

1. Challenges in the Logistics Industry
2. The Quest for an Objective Evaluation System To Ensure Safety and Security
3. Improving Safe Operation Assessments Using AI
4. Digital Transformation is Reshaping the Logistics Industry

Challenges in the Logistics Industry

We visited the Saitama branch of Suntory Logistics, which is the company's largest base and mainly covers the Tokyo metropolitan area and eastern Japan. This branch has one of the highest handling volumes in Japan, transporting 41.8 million cases per year (as of 2019).

First, we asked Mitsuji Kodama, Senior General Manager at the Suntory Logistics the Safety Promotion Department, about the role of the Safety Promotion Department and the issues faced by the logistics industry.

Takeda: What is the role of the Safety Promotion Department?

Kodama: This department is responsible for safety management within our company. Our mission is to embody the General Manager’s policy of “Safety First, One Team, and Speed,” and to instill this into our approximately 530 employees at 13 bases nationwide and applying the SDCA cycle (*1) to guarantee the safety of the entire company. Our department consists of eight members. We visit each base around 200 times a year to provide input to management on what is happening on site.

Takeda: Thank you. So this is a department that manages safety.
I understand that you are always on the front line of logistics. What do you think are the issues that the logistics industry faces today?

Kodama: Across the overall logistics industry, many companies are experiencing an increase in demand as people shelter in place due to the spread of COVID-19. As a result, I have heard that the number of occupational injuries due to the mishandling of forklifts has increased (Fig. A). Fortunately, no occupational injuries have occurred at our company, even during COVID-19. However, since warehouses are places where forklifts and people share the same space, all companies in the industry are always at risk of danger; in the worst case scenario, this can lead to fatal accidents. Despite this, as a general rule, occupational injuries do not occur at random. The occurrence of injuries is influenced by the way we regularly conduct our work. Therefore, our company has established our own work standards for safe and secure operations. Our external contractors are also expected to adhere to these work quality standards at their warehouses. I believe that the most important task is to ensure that employees adhere to these standards in the warehouses.

Takeda: I see, so forklift-related accidents are on the rise. Has your handling volume increased due to the demand created by shelter-in-place and COVID-19?

Kodama: Overall, our company hasn't seen any major changes in volume. But we've noticed a major shift in the item components, and partly due to the state of emergency it has become difficult to predict supply and demand. Also, since we can't conduct on-site management in person, in order to ensure safety, we've implemented mobile cameras with live feeds to supplement our on-site patrols.

Takeda: I see. By the way, I notice that there are huge trucks and forklifts zipping around this branch at a dizzying pace. How do you manage the flow of cargo to maintain a safe environment?

Kodama: Since the warehouse is constantly in motion, and no two days have the same layout, the forklift operators look at the shipping list and decides which cargo to put in storage and where to transport it on a case by case basis. So, in order to work efficiently and safely, crews need to be experienced and adhere to basic operation rules.

――Before developing this technology, Suntory Logistics already had one of the strictest safety management standards in the industry. What prompted the company to implement this AI detection system? To find out, we spoke with Makoto Ukawa, Deputy Senior General Manager at the Safety Promotion Department. He frequently travels between the company's bases throughout Japan to manage each site.

Takeda: Please tell us about the circumstances that led up to the implementation of this system.

Ukawa: In order to prevent occupational injuries and ensure safe and secure forklift operations, we first installed drive recorders on all 200+ of our forklifts in 2018 to check whether they are being operated according to the work standards that our company has established. Initially, our team's eight members were checking the footage manually, but it took us about 500 hours to go through the footage of several hundred forklift operators.

Takeda: That's a lot of work.

Ukawa: Yes. We had technical issues with this approach as well. Conventional drive recorders weren't able to capture movements specific to forklifts, such as those of the forks (where products are loaded). On the other hand, the forklift operators being evaluated often had misgivings about whether the feedback from evaluators inspecting the footage was really correct, since the evaluators would have a different perspective than the operators.

Takeda: I can understand that. No matter how thoroughly evaluators check the footage, there will always be oversights and slight differences in perspective.

Ukawa: We needed a system that could standardize those ambiguous issues and provide objective evaluations.

The Quest for an Objective Evaluation System To Ensure Safety and Security

――To solve these issues, Suntory Logistics worked with Fujitsu to develop an AI detection system. We spoke with Yuka Fukamachi, a data scientist at Fujitsu Cloud Technologies who was in charge of this project.

Fukamachi: This system uses footage taken on 360° drive recorders mounted on forklifts and detects dangerous operation situations. Of the 23 dangerous operation situations that Suntory Logistics had originally established, it detects three that were most strictly penalized: operating the fork while the forklift is still moving, not stopping to confirm that the cargo is stabilized when loading cargo onto the fork, and not stopping the forklift at designated stop lines. It detects dangerous operation situations by considering a combination of three factors: whether the forklift is moving, turning, or stopped, whether the operator is on board, and whether the forks are being operated.

Takeda: How are dangerous operation situations visualized?

Fukamachi: The video displays whether a dangerous operation situation is detected, the state of motion of the forklift (which is the basis for detecting a dangerous operation situation), and a safety coefficient calculated as a percentage of the dangerous operation situations in the entire video. To shorten the time required for evaluation, we have designed the system to automatically shorten the video by fast-forwarding through scenes that don't impact safe operations and don't need to be checked.

Takeda: I see. So the system is specialized in avoiding dangerous operation. What aspects required special consideration in the development process?

Fukamachi: There were precedents for analyzing drive recorder footage from cars, but analyzing drive recorder footage from forklifts had never been done before. Figuring out a way to capture footage of the vertical movement of the forks was technically challenging, so we had Ukawa create a number of data patterns on site and developed a method through repeated trial and error.

Takeda: I would also suppose that there were some situations in which video analysis was difficult due to low lighting conditions on site.

Fukamachi: Yes, it was challenging to design a versatile system that could adapt to various changes in the environment. These changes included not only lighting, but also constant variations in the warehouse environment, the color of the forklifts and forks, and the color of the crew's clothes and helmets. For example, under certain lighting conditions, I really wasn't able to get clear footage of the forks moving vertically. So, I went to a home improvement store, bought several dozen rolls of colored tape, attached that to different parts of the fork, and repeatedly experimented with different lighting and camera flash settings to try to improve accuracy.

Takeda: It sounds like it was important to work closely together with the on-site staff and repeat the trial and error process to make the system usable. How long did this take you?

Fukamachi: Developing this system took us about a year. Suntory Logistics had clear work standards within their company, so we were able to collect enough data. We adapted this system for their business based on those work processes and through repeated trial and error. I believe that Suntory Logistics' relentless pursuit of safety came to fruition in the shape of this system.

Takeda: I see, so the system is packed with Suntory Logistics' on-site expertise.

Fukamachi: Exactly. In addition, instead of expecting a perfect fit at the time we first designed the system, we gradually improved it by making small, incremental modifications and getting feedback from the on-site staff after each adjustment. This increased the accuracy of the AI training data, and we were able to gradually customize it to their business processes.

Improving Safe Operation Assessments Using AI

――The AI detection system was finally implemented in June 2021.

Ukawa: Since the AI detection system clearly analyzes and evaluates movements, we’ve greatly benefited from being able to present clear criteria when giving operators feedback. We've also been able to reduce the time spent on evaluation duties by about 50%, which has had a huge impact on our business operations.

――Meanwhile, how do the forklift operators feel about this system? We talked to Daisuke Muraoka at the Saitama branch to get their side of the story. He has maintained an S rank for two consecutive years as one of the top operators among Suntory Logistics' 200+ crew members and is in charge of mentoring junior staff.

Takeda: In the past, your evaluations used radar charts and reports. Now that the AI detection system has been implemented, what do operators think about it?

Muraoka: Forklift operators, myself included, pick up different habits. Those habits are subconscious, so the operators themselves don't realize whether those are good habits that lead to safe operations, or bad habits that are tied to dangerous operations. The AI detection system clearly shows us how safe we are through safety coefficients, so we feel that we are being assessed equally. It has also made it easier for me to train other operators, since I can share both good and bad aspects of their work.

Takeda: It's a great way to improve safety management on site.

Muraoka: Definitely. It also improves quality. At our company, we value quality nearly as much as safety. For example, boxes containing cases of 2-liter bottles have handles on the sides meant solely for the customer's use. In the unlikely event that the handles are damaged while a box is being transported by a forklift, our company considers it a damaged good. We are not only transporting our own products, but also cargo belonging to our customers, such as wholesalers and retailers. That's why we treat them with the utmost care and make sure they don't get even a single scratch. I believe that if we can achieve a high level of both quality and safety in an industry where perfection is the norm, that will lead to a better sense of security.

Takeda: So, it's only when everyone on site has a high level of professionalism that you can deliver peace of mind to consumers. I've been drinking your products regularly without thinking too much about it, but I now have a new awareness of the support that you provide consumers such as myself. Thank you very much.

Digital Transformation is Reshaping the Logistics Industry

――Finally, we spoke with Kodama and Fukamachi, along with Tomoaki Tamura, Executive Officer and Senior General Manager of the East Japan region at Suntory Logistics, about their outlook for the future.

Takeda: Today, we have discussed this topic from various perspectives. Now, could you tell us about your future plans?

Kodama: For this project, we reached out to several IT vendors about implementing this unprecedented AI detection system. Of those vendors, from the very beginning, Fujitsu was the most sincere about addressing the essence of the problem. After implementing the system, I felt once again that people are at the heart of this process. I realize that the judgment of AI is not absolute, but its insights are very useful if we regard the information it provides as advice from a trusted partner or consultant. Now that 5G is becoming more ubiquitous, allowing us to send video faster, I believe that we are moving toward a future in which we can monitor live footage to alert and train operators in real time. Most recently, since drive recorders have advanced to the point where they can send data over Wi-Fi, I believe they are highly feasible as a remote safety guidance tool during COVID-19.

Tamura: We certainly cannot ignore the importance of human labor, but automation is also necessary as the workforce in Japan declines in the coming years. Rather than using AI for absolutely everything, we should use AI to support people. I also want to implement this AI detection system to drive recorders in trucks and incorporate it into driver education. This will not only improve safety and quality at our company, but I believe it will also help solve social issues such as by making truck operation safer.

Fukamachi: I am glad to hear that. First of all, we are planning to update the AI detection system to detect more detailed items and add a function to provide reports along with the video analyses. We also want to turn this system into a general service that we can offer to other companies. I believe that all logistics companies have the same desire for safety. We would like to help solve the problems that the logistics industry faces by providing a service that is designed to fit a wider range of companies, in terms of labor and cost.

Takeda: Finally, I'd like to ask you all to say a few words to those who are considering undertaking some kind of digital transformation.

Tamura: There are many aspects to consider, but one is AI and automation. In the logistics industry, safety management and work quality are becoming increasingly sophisticated. I believe that we need to make a clear distinction between human labor, which is central to work processes, and the latest technology. Given those changes in our industry, I believe companies should proactively adopt DX; they will not be able to survive unless they do so.

Kodama: I agree. The logistics industry is expected to transform its business model through the use of data. That will optimize the entire process. In this regard, we are seeing a future where using AI will become the norm. On the other hand, humans will remain central to transportation for many years to come. In this context, I believe that ensuring a safe and secure work environment and improving the quality of our logistics are of utmost importance.

Fukamachi: DX certainly does not refer only to adopting IT. In the case of Suntory Logistics, the on-site and safety management staff first identified the essence of the issue: “How can we ensure safe forklift operation?” Then, to solve this issue, they introduced work standards, implemented drive recorders, and sometimes even re-evaluated how they operate. I think that DX is a holistic approach to safety, where technology such as the AI detection system is brought in to support a specific element of safety.

Takeda: When I visited the warehouse for this article, I was impressed by the uncompromisingly high regard for safety, the strong will to deliver products safely and securely to consumers, and the professional skills of the people who operate the forklifts and other equipment there. I felt that the essence of the project was using the latest technologies (such as the AI detection system, video analysis, and deep learning) to support operations. This produces a very high level of effectiveness. I firmly believe that Suntory Logistics' efforts will lead to improved safety management in not only the entire logistics industry but also in all fields, and that they have the potential to solve a wide range of social issues. Thank you very much.

• *1
  SDCA cycle
  One of the improvement methods in production and logistics settings. It is an acronym that stands for Standardize, Do, Check, and Act, the four stages of an improvement cycle. The cycle progresses in the order S→D→C→A.