At the forefront of ocean digitization: tackling blue carbon, port inspections, and biodiversity conservation challenges

Fujitsu embarked on the development of its Ocean Digital Twin in March 2024, aiming to support the planning of carbon neutrality initiatives (achieving net-zero by balancing greenhouse gas emissions with removals) and biodiversity conservation measures in the ocean. By digitally replicating various aspects of the marine environment, the Ocean Digital Twin enables the examination of solutions to ocean-related challenges and facilitates easy post-implementation verification. Specifically, it creates a 3D representation of the underwater world based on marine data collected and measured using underwater drones and other devices. This article delves into the characteristics of the three core technologies behind the Ocean Digital Twin - automatic control, ocean measurement, and seaweed bed AI modeling - and explores the potential future applications of these technologies.

A conceptual video of the ocean digital twin can be viewed here:

Kouichi: The underwater environment is truly a "dark cosmos." Visibility is limited, radio waves don't penetrate, and GPS is unavailable. Moreover, disturbances such as ocean currents and waves have a significant impact, making it extremely difficult to maneuver underwater drones with precision. Even for experienced pilots, accurately reaching a target location is no easy task. Our automated control technology, leveraging control technologies honed through the development of hardware such as robots and hard disk drives, enables autonomous underwater navigation even without GPS, allowing for stable collection of underwater data. The underwater drone itself constantly tracks its position by monitoring changes in speed and direction obtained from a compass and other sensors, accumulating this data over time. This enables it to follow pre-defined routes autonomously. In addition, attitude control maintains a stable orientation, preventing the drone from being tilted by wave action.

Kouichi: Seaweed sways with the currents, and the underwater drone itself is also in motion. To rapidly acquire 3D data of objects in dynamic conditions, we leverage technology developed through our previous work on a gymnastic Judging Support System. This system employs a MEMS (Micro-Electro-Mechanical Systems) mirror, a tiny mirror created using MEMS technology, rapidly to scan and project laser beams that measure the distance to the target. This enables the measurement of the gymnast's quick movements. By applying this same technology, we can rapidly acquire 3D shape data even for moving objects underwater.

Kouichi: While the Judging Support System uses infrared lasers, infrared light hardly penetrates seawater. Therefore, we use visible light lasers underwater. Specifically, we utilize three laser colors - red, green, and blue - and select the optimal color depending on the sea conditions, such as turbidity. For example, red laser light tends to penetrate relatively well in highly turbid water, while green and blue are more effective in clearer water. This ability to switch laser colors based on sea conditions, allowing us to stably and accurately reconstruct and measure the shape of objects, is a key strength of our technology.

Kouichi: Yes, it is possible. Ordinary camera images can appear greenish or lose clarity due to turbidity and other factors. To address this, we use AI technology (deep learning) to sharpen the images of the target objects, restore their contours and reproduce their colors. This is particularly useful for identifying seaweed species. In some cases, seaweed species are difficult to distinguish by shape alone, making color information a crucial factor for identification. By reproducing color information through deep learning-based image processing, we can accurately identify different types of seaweed.

Kouichi: Traditionally, seaweed species classification has relied on manual methods, such as divers taking photographs and visually confirming the species or physically collecting seaweed samples for examination. Our seaweed bed AI modeling technology accurately identifies seaweed species based on marine data collected autonomously by underwater drones. High-accuracy seaweed species classification typically requires a vast amount of training data for machine learning and faces challenges with reduced classification performance in turbid underwater conditions. We addressed these challenges by enabling highly accurate automatic classification of seaweed species using limited reference data. This technology leverages our image enhancement technology to mitigate the effects of underwater turbidity in images automatically collected by underwater drones. Moreover, by focusing on inherent seaweed characteristics that are less susceptible to turbidity and visibility issues, we achieve high-accuracy classification even with limited data. For more details, please refer to our TechBlog.

Yosuke: Marine specialists with expertise in numerical analysis, such as marine biology researchers, can fully grasp the growth status of seaweed just from numerical data. However, it can be difficult for others, like local government officials responsible for biodiversity conservation, to visualize this. Therefore, we create 3D representations of the data to make it easily understandable, even for those without specialized knowledge. The 3D visualizations clearly show the distribution of seaweed at a glance. We can even use color-coding to represent density. By utilizing these 3D visualizations, we can focus on areas with low seaweed density, implement measures to promote growth and verify their effectiveness.

Yosuke: This technology can also be applied to address a wide range of business and societal challenges. Here are a few examples of applications.

Kouichi: We are currently exploring ways to expand the information we can acquire underwater. Real-world environmental data, such as ocean currents and water quality, are essential for realizing a more precise Ocean Digital Twin. To expand the acquired data, we will advance the development of more accurate measurement and sensing technologies and conduct simulations incorporating knowledge from fields like biology and environmental science. Imagining the potential impact this comprehensive representation of the ocean could have on our lives is a major driving force behind our technological development.

Yosuke: We aim to enhance the functionality of the digital twin further to enable simulations of solutions to ocean-related challenges based on real-world sensor data, allowing for preemptive verification of the effectiveness of these solutions.

• If you are interested in participating in a proof-of-concept trial, please contact us.
• To explore other advanced technologies from Fujitsu, please visit the Fujitsu Research Portal.