Skip to main content



Archived content

NOTE: this is an archived page and the content is likely to be out of date.

Image Processing LSIs

FSTJ 2013-1 Cover Image

2013-1 (Vol.49, No.1)

This special issue introduces key technologies developed by Fujitsu Semiconductor for achieving a variety of system LSIs for image processing. These technologies are not limited to the LSI products themselves-they also include software technologies for achieving a variety of solutions and design and development support technologies for consolidating diverse functions in LSIs.

2013-1 (Vol.49, No.1) Contents

1. Preface (553 KB)
With an extensive background in advanced image-processing technology, application-support technology, and design and development support systems, Fujitsu Semiconductor is well positioned to help its customers grow their businesses through unified support, from the planning and development of LSI solutions to the manufacturing and sales of advanced products. ---[Haruyoshi Yagi, Member of the Board and Corporate Senior Executive Vice President, Fujitsu Semiconductor Ltd.]
2. Fujitsu's New Direction for Image Processing LSIs: Integration into a Visual Application Platform (880 KB)
Under the catchphrase of "Fujitsu for Image Processing," Fujitsu has developed image-processing technologies that engage human emotions and has developed LSIs that apply those technologies. Many of these technologies have been adopted in imaging-related devices and systems in a wide variety of fields, and they have been highly praised for their proven track record in image quality and expressiveness. Today, digital images are finding widespread use in society, and the demands placed on imaging devices by the market are changing. In addition to the demand for high-performance, compact, and low-power devices, the mutual dependency between device performance and content and associated services is growing. This trend is driving a change from imaging devices of the single-function, standalone type to those of a multi-function, networked type. In the face of these market changes, Fujitsu is constructing a visual (visually appealing) application platform that integrates LSIs incorporating Fujitsu's proprietary image-processing technologies with a development support environment that has a proven track record in system development projects with customers. This paper introduces visual solutions based on this platform. ---[Tom Miyake, Yukio Otobe]
3. Milbeaut ISP for Digital Cameras (1.21 MB )
The Milbeaut image signal processor (ISP) enables a digital camera system to be implemented on a chip. It can process signals from a variety of image sensors, perform enhanced-quality image processing, and store 20-megapixel-plus still images and full high-definition (Full HD) video. This paper introduces the MB91696AM ISP, the sixth generation of the Milbeaut series. It features high-speed continuous shooting and Full HD video capture thanks to a newly developed high-speed signal-processing platform. It provides enhanced noise-reduction performance and optical-correction functions that have been improved with every Milbeaut generation and incorporates general-purpose image-processing macros to satisfy market demands. As a processor that provides high-performance and high-quality processing of still images to Full HD video as desired by today's camera users, the MB91696AM has come to be used in many digital cameras that require a high pixel count, high-speed processing, and multi-functionality. ---[Kentaro Endo]
4. Milbeaut Image Signal Processing LSI Chip for Mobile Phones (1.01 MB )
The camera function of mobile phones has progressed in the last few years, as typified by the rapid spread of high pixel count, high resolution, and full high-definition video recording and playback features in smartphones. Although it is possible to implement camera functions such as rapid imaging and browsing from personal computers and mobile terminals into the application processor in a mobile phone, many customers prefer dedicated image signal processors (ISPs) as they offer image quality and performance comparable to those of compact high-end digital cameras. This paper describes the development and performance of the Milbeaut Mobile image processing LSI chip developed by Fujitsu for mobile phones. ---[Satoru Komatsu, Mitsumaro Kimura, Akira Okawa, Hideaki Miyashita]
5. Graphics Display Controller for Automotive Applications (1,021 KB)
Fujitsu has had graphics display controller (GDC) products on the market for years as system chips for use in in-vehicle information devices. More automobile instrument cluster and center display systems are now using full-color liquid crystal displays, and there are often multiple displays in a single vehicle. Keeping pace with this evolution of devices, Fujitsu developed the MB86R1x series of systems-on-a-chip (SoCs), which integrate a high-performance microprocessor with a proprietary graphics processing unit (GPU) and various in-vehicle network functions. Here, we briefly describe the latest trends in automobile information devices and systems and present an overview of the MB86R1x series of SoCs, our most recent automotive GDC product line. We also explain the technologies developed for these systems, including the GPU, image processor, and in-vehicle network. ---[Eisuke Miura, Makoto Nakahara, Hidefumi Nishi]
6. Superior Error Concealment and Robust Audio-Video Synchronization in Set-Top Box Chip (1.09 MB )
Fujitsu's MB86H06 set-top box (STB) chip is a high-density chip featuring high-performance, superior error concealment, robust audio-video (AV) synchronization, and low cost. It comprises a QFP256 package with DDR2 memory and supports a two-layer printed circuit board (PCB) layout for a low bill of material (BOM) cost. It is widely used in the China terrestrial television broadcasting (CTTB) market. In this paper, we discuss the practical realization of error concealment and AV synchronization under weak signal conditions. ---[Jiang Li, Yan Yingrui, Ni Xin]
7. H.264 Transcoder LSI Chip (918 KB)
Extension of video recording times and further reduction in network traffic loads require that video data compressed with the MPEG-2 format used for digital terrestrial broadcasting and other types of broadcasting be converted to the H.264 format, which has a higher data compression rate. It is also desirable that H.264 video data itself be converted to an even lower bit rate. This paper introduces a transcoder implemented on an LSI chip for executing in real time the transcoding processing required for these applications. The transcoding method used is advantageous in terms of image quality. It first decompresses compressed video data and then recompresses it so as to minimize the image deterioration that normally accompanies cross-format conversion. This LSI has low power consumption and a small form factor, enabling it to operate with USB power (under 2.5 W) and be mounted in devices the size of a PCI Express Mini Card (30 × 50.95 mm). It is now being used mainly in PC-TVs and personal video recorders (PVRs) to achieve long recording times of digital-terrestrial, BS digital, and CS digital broadcast programs in Japan. However, with the recent proliferation of mobile terminals like smartphones and tablets and the increasing demand for video viewing and recording via the network, Fujitsu expects this transcoder to also be used for converting video to a compression format that can reduce network traffic loads and enable the playback of HD video on today's mobile terminals. Fujitsu Semiconductor is committed to creating new solutions to support these applications. ---[Kazuyuki Tanaka]
8. Development of Tuner Module for Mobile Multimedia Broadcasting (1.18 MB )
Fujitsu Semiconductor has developed a Mobacas (mobile multimedia broadcasting)-compatible tuner module conforming to the Integrated Services Digital Broadcasting for Terrestrial Multi-Media Broadcasting (ISDB-Tmm) standard. This tuner module incorporates proprietary algorithms developed jointly with Fujitsu Laboratories to significantly improve reception sensitivity and enhance reception performance in a single-frequency network (SFN), thereby helping to expand the reception area. The module comes in two types to meet the design needs of terminal developers. The MB86A35V module makes it easy to achieve a required level of functionality and performance by simply connecting the module to an antenna, and the MB86A35C module enables a terminal developer to add an external filter for removing interference. This paper presents the key issues in the development of this Mobacas-compatible tuner module, describes the measures taken to solve them, summarizes the module specifications, and introduces the module's product lineup. ---[Junichiro Okayama, Yoshitaka Mizuno]
9. Imaging Application Platform Using MB86S62 Application Processor (1.61 MB )
Imaging solutions using application processors means providing customers with an application platform that includes software and evaluation boards based on Fujitsu's extensive image-processing technologies. Customers who manufacture assembled products (set makers) can use this platform to develop and market multi-function products in a shorter period of time. In combination with Fujitsu's Milbeaut image processor, this platform provides a high-performance image processing system that can provide customers with a competitive edge over other makers. This paper presents a specific example of this imaging application platform using Fujitsu's MB86S62 application processor. ---[Atsuhiro Suga, Hidetoshi Nakahara, Masato Hyodo, Shinichi Takimoto]
10. H.264/AVC Video Codec Technology for High-Image-Quality and Low-Power Applications (854 KB)
To make use of the high coding performance of the international standard H.264/AVC to achieve high image quality, a massive amount of computation and accompanying reference to many pieces of image data are required. These processes involve a trade-off with reducing power consumption. Fujitsu has developed H.264/AVC codec technology capable of achieving industry-leading levels of high-image-quality and low-power-consumption performance based on its proprietary image analysis technology, and applied it to many LSIs for imaging devices. This paper outlines the technology to reduce power consumption with the focus on data transfer with external memory, which occurs when there is internal computation processing. First, it describes a lossless image compression technique optimized for video codec processing. Then, it gives an explanation about technology called "lossless compression prefetch memory," which is applied to read ahead and retain the data likely to be used repeatedly, thereby significantly reducing the amount of data transfer. The technology presented in this paper, and the H.264/AVC codec technology, which achieves both high image quality and lower power consumption, can be applied to devices for which there is a high demand for power consumption reduction. ---[Kiyonori Morioka]
11. Development Environment of 3D Graphics Systems (1.21 MB )
Recently, more and more in-vehicle devices have been using 3D graphics in applications such as car navigation systems and digital meters. These devices adopt user interfaces that are more graphical, and they have come to offer visual effects that were previously impossible to produce. Fujitsu Semiconductor is focusing on augmented reality, in which actual images taken by cameras are combined with 3D graphics for integration with the real world to enhance the vision of users, and is moving ahead with the development of graphics systems. It requires considerable expertise to build and develop these systems and applications that use 3D graphics. Fujitsu Semiconductor is developing and offering a development environment that can reduce the burden on application developers using 3D graphics and software to realize products capable of delivering optimum functionality and performance. This paper describes the software configuration and architecture needed for a system that uses 3D graphics and gives examples of their application. ---[Yuya Tagami, Makoto Watanabe, Yuko Yamaguchi]
12. Platform to Support Video Encoding (884 KB)
Since the days of analog TV broadcasting, Fujitsu has been engaged in the development of LSIs for compressing video images compatible with MPEG-2. In recent BS/terrestrial digital broadcasting, the broadcasts themselves are data compressed with MPEG-2, but more efficient recording can be achieved by reconverting them into H.264, which features even higher compression efficiency than MPEG-2. In this way, the fields in which LSIs for compressing video images can be applied are expanding. These video image compression processes, including audio processing and the processing of data streams generated by compression, need to match the various types of data to be processed. In addition, video image compression, which characteristically uses a massive amount of memory and handles a large amount of data, essentially requires an external DRAM connection. Accordingly, another important factor in such compression is the configuration of a memory system. In relation to these development requirements, this paper presents the LSIs that Fujitsu has developed from the perspective of a platform. The building of a platform has allowed Fujitsu to accumulate design assets, the surrounding environment and know-how, and allowed it to efficiently develop new LSIs. ---[Tatsushi Otsuka]
13. 3D Graphics Authoring System: CGI Studio (1.43 MB )
Systems that apply 3D graphics are beginning to become common in automobiles, as in instrument clusters integrating full-color LCDs. Fujitsu Semiconductor Embedded Solutions Austria GmbH, a group company of Fujitsu Semiconductor, has developed CGI Studio as a tool for developing these systems. As a framework of authoring tools to develop 3D graphics content for automotive systems, CGI Studio provides a workflow for embodying a designer's ideas in an actual target system (such as an actual product and prototype for product development). CGI Studio can also be used in combination with other widespread graphics authoring tools, and thus consolidate the existing tools used by customers. CGI Studio integrates multiple pieces of software and includes analysis tools for identifying performance bottlenecks in the execution of applications on the hardware used. They include software for 2D and 3D composition, authoring tools, scene generation, the Candera software graphics engine and scene player. This paper outlines the trends in automotive application of graphics and describes the features of CGI Studio and the collaborative workflow between designers and embedded system developers realized by CGI Studio. ---[Gerhard Roos, Gernot Reisinger, Roland Winkler]
14. Inspirium HMI-Studio: Authoring System for Creating HMI Scenarios (1.83 MB )
Performance enhancement, multifunctionalization and diversification of devices that use imaging LSIs are in progress and the scale of software installed is increasing. Hence, a major issue for embedded device manufacturers now is how to develop software while keeping the development costs down. Above all, in devices with many screen pages a large part of the software consists of the user interface (UI), and reducing software development costs especially for this UI portion is a significant challenge. In conventional software development for a UI, first the UI specifications are established, and then the UI is implemented and the design is built; the operability is verified only after it has been installed in an actual device. Accordingly, reworking tends to occur frequently in the following production processes. To reduce this amount of reworking it is necessary to brush up the UI specifications in the UI specification establishment phase. Meanwhile, software development tools for UI implementation are commercially available in large numbers but no UI development tool capable of brushing up UI specifications in the UI specification establishment phase has been prepared. This paper describes the excellent features of Inspirium HMI-Studio, which covers the entire UI development process from the UI specification establishment phase to the following production processes and reduces the total UI development costs. ---[Shunichi Hirabayashi]
15. Automotive Graphics SoC for 360° Wraparound View System (1.74 MB )
Recently, camera systems have been evolving to assist the vision of automobile drivers and reduce their blind spots around a vehicle. The 360° wraparound view system developed by Fujitsu Semiconductor allows the driver to freely change the point of view to zoom in on a desired area by projecting an image on a 3D curved surface. This makes it possible for the driver to identify vehicles and pedestrians in the surrounding area, which was difficult with the existing overhead camera systems. The 360° wraparound view system has been developed by using graphics systems-on-a-chip (SoCs) for in-vehicle use (MB86R11/MB86R12). These SoCs combine ARM's CortexTM-A9, integrate an OpenGL ES2.0-compliant graphic engine and various peripheral interfaces on one chip, and achieve a low delay of 30 ms from camera image capture to composite image output and high image quality with a processing function to improve visibility. In addition, the middleware and authoring tools for creating and editing full-perimeter 3D monitoring images have realized a 360° seamless view around a vehicle. This paper gives an outline of the 360° wraparound view system and an explanation about its development platform. ---[Suehiro Kawanishi]
16. Embedded Software: Inspirium (1.43 MB )
Inspirium is a generic name for embedded software developed by Fujitsu. It is intended to increase the added value and quality of embedded devices and comes in a wide range of products including standalone middleware embedded in image processing LSIs and server linkage middleware. In addition to describing a screen displayed with imaging LSIs, this paper presents human-centric interfaces including a handwriting recognition library capable of easily realizing handwriting input from the touchscreens used in smartphones and tablet PCs, and a speech synthesis library that provides an audio read-out in correct Japanese of text containing a mixture of kana (the syllabic Japanese scripts) and kanji (Chinese characters). The handwriting recognition library captures the characteristics of a person's handwriting on paper with a pen and is good at recognizing simplified characters and scribble. The speech synthesis library can read out text containing a mixture of kana and kanji in correct and listener-friendly Japanese. A predefined dictionary of words is used for adjusting the readings and stresses of words in terms of intonation and rhythm by prosodic processing to synthesize listener-friendly Japanese sounds. This paper describes the specifications and configuration of these two libraries and gives application examples. ---[Ichirou Tokuda, Shigeki Moride]
17. Hardware—Software Codesign for Graphic LSIs (805 KB)
The field of image processing LSIs that are mounted in devices such as digital cameras is one in which the improvement in performance is very remarkable. It is important to have an architecture design that is suitable for the application so as to lower the cost of developing chips for LSIs, and reduce their power consumption and the chip area. Semiconductor and EDA vendors have proposed hardware-software (HW/SW) codesign via electronic system level (ESL) as a way to develop chip architecture and designs. However, it has not been used much in actual development sites because of various issues such as the cost of developing models. Under these circumstances, at Fujitsu Semiconductor we have steadily applied architecture design technology to LSI development based on HW-SW coverification in the ESL methodology in Cedar service, which is a design service for customers to develop ASSPs and ASICs. In this way, we have found that the most important point is to optimize the quality of service (QoS) of on-chip buses and access to external memory such as DDR memory. And, as a result of focusing on this point of optimization and investigating ways to tackle the issues with ESL, we have created a new HW/SW codesign solution that uses ESL. It has come to be used in design sites more than ever before. This paper introduces the technical aspects of this new approach and its effect, and also describes future developments. ---[Hirohisa Kotegawa, Naonobu Hasumi]
18. LSI Design Flow Development for Advanced Technology (979 KB)
LSIs that adopt advanced technologies, as represented by imaging LSIs, now contain 30 million or more logic gates and the scale is beginning to approach the level of 100 million gates. As compared with the 90 nm process generation, this is three to ten times the number of gates. With advanced technologies such as 40 nm and 28 nm, the process characteristics become remarkably complex and wiring resistance rapidly increases, which unavoidably means that more sign-off corners and more accurate sign-off and layout tools are required. In addition, mask design rules, which are process requirements, are constantly increasing in complexity and the 28 nm process requires twice as many mask design rules as the 90 nm one. To address these changing needs in the areas of design and technology, Fujitsu Semiconductor is developing new features for the LSI design environment called Reference Design Flow for introduction into LSI design. This has reduced the turnaround time (TAT) of LSI development adopting advanced technologies. This paper describes the characteristics and effects of Reference Design Flow newly enhanced for advanced technologies. ---[Atsushi Tsuchiya]
19. Low-Power Technology for Image-Processing LSIs (930 KB)
The conventional LSI design assumed power would be supplied uniformly to all parts of an LSI. For a design with multiple supply voltages and a power gating design, the inside of an LSI must be divided into multiple supply regions, and this complicates the LSI design. This paper introduces low-power technology applied to image-processing LSIs of Fujitsu Semiconductor, which designs such complicated LSIs with high reliability. Fujitsu Semiconductor engages in intensive development with CoolAdjust, a set of technologies to reduce power consumption by controlling power supply. Accordingly, this paper mainly focuses on describing technologies for a design that can handle multiple supply voltages and effectively reduce the dynamic power, and a power gating design that can effectively reduce the amount of leakage power. In a power gating design, when the power switch is turned on to supply power to a circuit, a large current called an inrush current runs so as to charge the parasitic capacitance of the circuit. It is known that this inrush current affects the circuits that are already powered on. This problem has been solved by controlling two types of power switches with a time interval, and this is also presented here. ---[Yoshimi Asada]
20. Verification of System LSIs for Image Processing (1.15 MB )
In recent years, system LSIs have become more complicated in terms of their hardware structure along with their increase in size and greater functions, and this is making it difficult to verify their logic quality. Image-processing LSIs, in particular, have a higher complexity of software processing that they perform with their multi-core structures and increasingly complicated structure of internal buses. This makes it essential to ensure that their functions, performance and power consumption satisfy the requirements by conducting system verification at the pre-silicon stage (LSI design phase). However, product development with a short turnaround time (TAT) is called for because of the recent shorter model change cycles, and efficiently conducting system verification while ensuring quality is very important. Fujitsu Semiconductor is making positive use of a hardware emulator (a device capable of mapping the circuit to be verified to dedicated hardware for running it at high speed) to establish verification technology that ensures quality and improves verification efficiency at the same time, and applying it to product models. This paper presents hardware/software co-verification, performance verification and power consumption estimation by using a hardware emulator, which we believe is very effective for verifying image-processing LSIs. ---[Yoshihiko Hayashi, Noriyuki Ikuma, Taku Kawamura, Takashi Tokue]
21. CHIP-PKG-PCB Co-Design Methodology (1.70 MB )
For digital devices integrating an image-processing LSI, performance improvement, cost cutting and reduction of the time to market are essential conditions for surviving in the increasingly competitive global market. At the same time, more and more image-processing LSIs are taking the form of large-scale system-on-a-chip (SoC). It is becoming harder to design them as the degree of integration increases, and signal and power integrity issues are appearing due to their processing speed increase and voltage reduction. As a design methodology to address four challenges in recent SoC design (signal and power integrity issues, high-density design, reduction of design turnaround time [TAT], and cost cutting), Fujitsu Semiconductor has established chip-package-printed circuit board (CHIP-PKG-PCB) co-design methodology and made achievements contributing to first-shot full operation of SoCs and digital devices that integrate them. This paper presents our approach to the CHIP-PKG-PCB co-design for dealing with the four challenges above by giving case examples. ---[Atsushi Sato, Yoshiyuki Kimura, Motoaki Matsumura]
22. Packaging Technology for Image-Processing LSI (1.25 MB )
The main function of a semiconductor package is to reliably transmit electric signals from minute electrode pads formed on an LSI chip (semiconductor chip) to the mounted terminals of the semiconductor package. To ensure a good LSI performance, it is becoming important to optimize the package structure according to the characteristics of the LSI and product use environment. Image-processing LSIs must process a large amount of image data at high speeds, which makes it necessary to have high-speed, high-quality data communication with a high-capacity external buffer memory. Semiconductor package structures are becoming more elaborate so that such LSIs can reliably perform these operations. Image-processing LSIs are used in mobile devices such as digital cameras, mobile phones, and smartphones, and they adopt high-density mounting technologies including packaging technology to miniaturize them and give them a system in package (SiP) structure with mixed mounting of multiple LSIs. This paper presents advanced semiconductor packaging technologies used in semiconductor packages for image-processing LSIs. ---[Yoshiyuki Yoneda, Kouichi Nakamura]