Abstracts of Magazine FUJITSU 2012-7 (VOL. 63, NO. 4)

Special Issue: Image Processing LSI

• Fujitsu's New Direction for Image Processing LSIs: Integration into Visual Application Platform

With the catchphrase "Fujitsu for Image Processing," we have developed image processing techniques that appeal to human sensitivity and developed LSIs that apply them. Fujitsu's LSIs are used in image-related devices and systems in various applications, and they have achieved high commendation and good results in terms of image quality and visual creativity. Digital image technologies and content have become widespread among our society recently and the user's requirements for these image-related systems are also changing. Reflecting deep interdependence between digital content and connection services, conventional single-function standalone devices are being replaced by networked multifunctional systems, in addition to improved performance such as low-power, smaller size and high picture quality. Considering these changes in market demand, we have started to establish "visual (visually appealing) application platform." This platform consists of the LSI that integrates Fujitsu's superior image processing techniques and the user-support environment that has been built and improved through system development with customers. This paper presents visual solutions based on this platform.

• Image Signal Processor: Milbeaut

Milbeaut is an image signal processor (ISP) that realizes a digital camera system on one chip. It can capture signals from various types of image sensors and then enhance the image quality and store the data as still images of over 20 megapixels or full high-definition (HD) moving images. This paper presents MB91696AM, the sixth generation of the Milbeaut series. MB91696AM offers high-speed continuous shooting and full HD moving image shooting that have been enabled by the new high-speed signal processing platform developed, noise reduction performance and optical correction functionality enhanced generation by generation, and the general-purpose image processing macro that accurately meets the market's requirements. Camera users want digital cameras to have a higher pixel density, be faster and be multifunctional. This ISP product, which has realized high performance and high image quality that fulfill a range of camera users' demand for still images and full HD moving images, is used in many digital cameras.

• Image Signal Processing LSI for Mobile Phones: Milbeaut Mobile

Camera functions of mobile phones represented by smartphones have been rapidly evolving in recent years and now offer features such as a large number of pixels, high image quality, and full high-definition video recording and replay. So that the user can quickly take photographs or videos and view them on PCs and mobile devices, these features can be offered with the camera functions integrated into an application processor of a mobile phone handset. However, aiming to give high-end models an image quality and performance comparable to those of compact digital cameras, many customers have adopted an image signal processor (ISP), an LSI exclusively for processing images. This paper presents the history and performance of Milbeaut Mobile, an image Signal processing LSI developed by Fujitsu for mobile phones.

• Graphics Display Controller for Automotive Application

Fujitsu has long been commercially offering graphics display controllers (GDCs) as system LSIs for information devices mounted in automobiles. At first, they were intended mainly for car navigation systems as LSIs for drawing and display control. Recently, however, increasing numbers of automotive systems such as instrument clusters and center displays have been using full-color LCD displays and more than one display is often installed in one automobile. Along with such evolution of devices, Fujitsu has commercialized the MB86R1x series, a system on a chip (SoC) combining a high-performance microprocessor, proprietary graphics processing unit (GPU) and various automotive network functions. This paper outlines the recent trends in automotive information devices and systems. It also gives an overview of the MB86R1x series developed as the latest automotive GDC product and a description about the technologies developed for this LSI including the GPU, image processing unit and network circuit integrated.

• STB for CTTB Application

In this paper, we will introduce one SD STB decoder which has been widely used in CTTB (China Terrestrial TV Broadcasting) market. This decoder features high-performance and low cost. It has superior error concealment and robust AV synchronization; It is QFP256 package with DDR2, and support 2-layer PCB layout for low BOM cost. We will discuss some of practical realization of error concealment and AV synchronization in weak signal condition in this paper.

• H.264 Transcoder LSI

Video data are compressed in MPEG-2 format for use in terrestrial digital broadcasting and such like. To extend the recording time and reduce network traffic, it is necessary to convert such data into H.264 format, which features an even higher compression rate, and also convert data in H.264 format into data of a lower bit rate. This paper introduces an LSI capable of transcoding, which is necessary for these conversions in real time. This LSI uses a transcoding method that is regarded as being the most advantageous in terms of image quality. With this method, encoded video data are decoded once and then re-encoded to minimize the degradation of video quality caused by the conversion processing. The LSI, which has also been given a small size and low power consumption to allow it to operate with USB bus power (up to 2.5 W) and be used in devices with the size of a PCI Express Mini Card (30×50.95 mm), is mainly used at present for PC-TVs and PVRs with a view to achieving long time recording of broadcasting including terrestrial digital, BS digital and CS digital broadcasting. As mobile devices such as smartphones and tablets have become widespread recently, the demand for viewing and recording videos via a network is increasing and it is hoped that the adoption of this LSI will reduce network traffic and convert data into video formats that allow them to be decoded on mobile devices. Fujitsu Semiconductor is working on coming up with new solutions to support these applications.

• Development of Tuner Module for ISDB-Tmm

Fujitsu Semiconductor has developed a mobacas-enabled tuner module compatible with Integrated Services Digital Broadcasting for Terrestrial Multi-Media Broadcasting (ISDB-Tmm), a standard of multimedia broadcasting for mobile devices. This tuner module uses an original algorithm jointly developed with Fujitsu Laboratories to significantly enhance reception sensitivity and improve reception performance in a single-frequency network (SFN) environment, thus helping to expand the reception area. The tuner module comes in two types for use in different design applications of terminal development users: MB86A35V, which is capable of easily being functional and offering a good performance simply by directly connecting it to an antenna, and MB86A35C, which allows terminal development users to add a filter externally to minimize disturbance. This paper describes issues in the development of the mobacas-enabled tuner module and measures for improvement and presents the product specifications and line-up.

• Imaging Solutions Using Application Processors

The imaging solutions using application processors is a form of providing customers with an application platform including software and evaluation boards by taking advantage of image processing technology at which Fujitsu excels. By using this platform, customers who are assembly manufacturers can develop multifunctional and new products in a short time and bring them to market. This platform realizes a high-performance image processing system when combined with the Milbeaut imaging processor, and manufacturers can use it to emphasize their superiority over other assembly manufacturers. This paper presents examples of imaging application platforms that use application processor MB86S62.

• H.264/AVC Video Codec IP for High Image Quality and Low Power Applications

To make use of the high coding performance of the international standard H.264/AVC to achieve high image quality, a massive amount of computation and accompanying reference to many pieces of image data are required. These processes involve a trade-off with reducing power consumption. Fujitsu has developed H.264/AVC codec technology capable of achieving industry-leading levels of high image quality and low power consumption performance based on its proprietary image analysis technology, and applied it to many LSIs for imaging devices. This paper outlines the technology to reduce power consumption with the focus on data transfer with external memory, which is generated along with the internal computation processing. First, it describes a lossless image compression technique optimized for video codec processing. Then, it gives an explanation about technology called "lossless compression prefetch memory," which is applied to read ahead and retain the data likely to be used repeatedly, thereby significantly reducing the amount of data transfer. The technology presented in this paper, and the H.264/AVC codec technology, which achieves both high image quality and lower power consumption, can be applied to devices for which there is a high demand for power consumption reduction.

• Development Environment of 3D Graphics Systems

Recently, more and more in-vehicle devices have been using 3D graphics in applications such as car navigation systems and digital meters. These devices adopt user interfaces that are more graphical, and they have come to offer visual effects that were previously impossible to produce. Fujitsu Semiconductor is focusing on augmented reality, in which actual images taken by cameras are combined with 3D graphics for integration with the real world to enhance the vision of users, and is moving ahead with the development of graphics systems. The building and development of these systems and applications that use 3D graphics require considerable expertise. Fujitsu Semiconductor is developing and offering a development environment that can reduce the burden on developers of applications using 3D graphics and software so as to realize products capable of delivering optimum functionality and performance. This paper describes the software configuration and architecture needed for a system that uses 3D graphics and gives examples of their application.

• Platform to Support Video Encoding

Since the days of analog TV broadcasting, Fujitsu has been engaged in the development of LSIs for compressing video images compatible with MPEG-2. In recent BS/terrestrial digital broadcasting, the broadcasts themselves are data compressed with MPEG-2, but more efficient recording can be achieved by reconverting them into H.264, which features even higher compression efficiency than MPEG-2. In this way, the fields in which LSIs for compressing video images can be applied are expanding. These video image compression processes, including audio processing and the processing of data streams generated by compression, need to match the various types of data to be processed. In addition, video image compression, which characteristically uses a massive amount of memory and handles a large amount of data, essentially requires an external DRAM connection. Accordingly, another important factor in such compression is the configuration of a memory system. In relation to these development requirements, this paper presents the LSIs that Fujitsu has developed from the perspective of a platform. The building of a platform has allowed Fujitsu to accumulate design assets, the surrounding environment and know-how and allowed it to efficiently develop new LSIs.

• 3D Graphics Authoring System: CGI Studio

Systems that apply 3D graphics are beginning to be common in automobiles, as in instrument clusters integrating full-color LCDs. Fujitsu Semiconductor Embedded Solutions Austria GmbH, a group company of Fujitsu Semiconductor, has developed CGI Studio as a tool for developing these systems. As a framework of authoring tools to develop 3D graphics content for automotive systems, CGI Studio provides a workflow for embodying a designer's ideas in an actual target system (such as an actual product and prototype for product development). CGI Studio can also be used in combination with other widespread graphics authoring tools, and thus consolidate the existing tools used by customers. CGI Studio integrates multiple pieces of software and includes analysis tools for identifying performance bottlenecks in the execution of applications on the hardware used. They include software for 2D and 3D composition, authoring tools, scene generation, the Candera software graphics engine and scene player. This paper outlines the trends in automotive application of graphics and describes the features of CGI Studio and the collaborative workflow between designers and embedded system developers realized by CGI Studio.

• Inspirium HMI-Studio: Authoring System for Creating HMI Scenarios

Performance enhancement, multifunctionalization and diversification of devices that use imaging LSIs are in progress among embedded device manufacturers and the scale of software installed is increasing. Hence, a major issue now is how to develop software while keeping the development costs down. Above all, in devices with many screen pages a large part of the software consists of the user interface (UI), and reducing software development costs especially for this UI portion is a significant challenge. In conventional software development for a UI, first the UI specifications are established, and then the UI is implemented and the design is built; the operability is verified only after it has been installed in an actual device. Accordingly, rework tends to occur frequently in the post-production processes. To reduce this amount of reworking it is necessary to brush up the UI specifications in the UI specification establishment phase. Meanwhile, software development tools for UI implementation are commercially available in large numbers but no UI development tool capable of brushing up UI specifications in the UI specification establishment phase has been prepared. This paper describes the excellent features of Inspirium HMI-Studio, which covers the entire UI development process from the UI specification establishment phase to the post-production process and reduces the total UI development costs.

• Automotive Graphics SoC for 360˚ Wraparound View System

Recently, camera systems have been evolving to assist the vision of automobile drivers and reduce their blind spots around a vehicle. The 360˚ wraparound view system developed by Fujitsu Semiconductor allows the driver to freely change the point of view to zoom in on a desired area by projecting an image on a 3D curved surface. This makes it possible for the driver to identify vehicles and pedestrians in the surrounding area, which was difficult with the existing overhead camera systems. The 360˚ wraparound view system has been developed by using graphics systems on a chip (SoCs) for in-vehicle use (MB86R11/MB86R12). These SoCs combine ARM's Cortex^TM-A9, integrate an OpenGL ES2.0-compliant graphic engine and various peripheral interfaces on one chip, and achieve a low delay of 30 ms from camera image capture to composite image output and high image quality with a processing function to improve visibility. In addition, the middleware and authoring tools for creating and editing full-perimeter 3D monitoring images has realized a 360˚ seamless view around a vehicle. This paper gives an outline of the 360˚ wraparound view system and an explanation about its development platform.

• Embedded Software: Inspirium

Inspirium is a generic name for embedded software developed by Fujitsu. It is intended to increase the added value and quality of embedded devices and comes in a wide range of products including standalone middleware embedded in imaging chips (large scale integration or LSIs) and server linkage middleware. In addition to describing a screen displayed with imaging LSIs, this paper presents human-centric interfaces including a handwriting recognition library capable of easily realizing handwriting input from the touch panels used in smartphones and tablet PCs, and a speech synthesis library that provides an audio read-out in correct Japanese of text containing a mixture of kana (the syllabic Japanese scripts) and kanji (Chinese characters). The handwriting recognition library captures the characteristics of a person's handwriting on paper with a pen and is good at recognizing simplified characters and scribble. The speech synthesis library can read out text containing a mixture of kana and kanji in correct and listener-friendly Japanese. A predefined dictionary of words is used for adjusting the readings and stresses of words in terms of intonation and rhythm by prosodic processing to synthesize listener-friendly Japanese sounds. This paper describes the specifications and configuration of these two libraries and gives application examples.

• Hardware–Software Codesign for Graphic LSIs

The field of image processing LSIs that are mounted in devices such as digital cameras is one in which the increase in performance is very remarkable. It is important to have an architecture and design that is suitable for the application so as to lower the cost of developing chips for LSIs, and reduce their power consumption and the chip area. Semiconductor and EDA vendors have proposed hardware–software (HW/SW) codesign via Electronic System Level (ESL) as a way to develop chip architecture and designs. However, it has not been used much in actual development sites because of various issues such as the cost of developing models. Under these circumstances, at Fujitsu Semiconductor we found that in Cedar service, which is a design service for customers to develop ASIC, ASSP, and such like, the most important point is to steadily apply architecture and design technology to LSI development. This can be done by conducting HW/SW coverification in ESL to optimize access to QoS of On Chip Bus and external memory such as DDR. And, as a result of focusing on this point of optimization and investigating ways to tackle the issues with ESL, we have created a new HW/SW codesign solution that uses ESL. It has come to be used in design sites more than ever before. This paper introduces the technical aspects of this new approach and its effect, and also describes future developments.

• LSI Design Flow Development for Advanced Technology

LSIs that adopt advanced technologies, as represented by imaging LSIs, now contain 30 million or more logic gates and the scale is beginning to approach the level of 100 million gates. As compared with the 90 nm process generation, this is three to 10 times the number of gates. With advanced technologies such as 40 nm and 28 nm, the process characteristics become remarkably complex and wiring resistance rapidly increases, which unavoidably means that more sign-off corners and more accurate sign-off and layout tools are required. In addition, mask design rules, which are process requirements, are constantly increasing in complexity and the 28-nm process requires twice as many mask design rules as the 90-nm process. To address these changing needs in the areas of design and technology, Fujitsu Semiconductor is developing new features for the LSI design environment called Reference Design Flow for introduction into LSI design. This has reduced the turnaround time (TAT) of large-scale LSI development adopting advanced technologies. This paper describes the characteristics and effects of Reference Design Flow newly enhanced for advanced technologies.

• Low-Power Technology for Imaging LSIs

The conventional LSI design assumed power would be supplied uniformly to all parts of an LSI. For a design with multiple power supply voltages and a power gating design, the inside of an LSI must be divided into multiple power supply regions, which complicates the LSI design. This paper introduces the low-power technology applied to imaging LSIs of Fujitsu Semiconductor, which designs such complicated LSIs with high reliability. Fujitsu Semiconductor engages in intensive development with CoolAdjust, a set of technologies to reduce power consumption by implementing power supply control. Accordingly, this paper mainly focuses on describing technologies for a design with multiple power supply voltages that can effectively reduce the dynamic power and a power gating design that can effectively reduce the amount of leakage power. In a power gating design, when the power switch is turned on to supply power to a circuit, a large current called an inrush current runs so as to charge the parasitic capacitance of the circuit. It is known that this inrush current affects the circuits that are already powered on. This problem has been solved by controlling two types of power switches with a time interval, which is also presented here.

• Verification of System LSIs for Image Processing

In recent years, system LSIs have become more complicated in terms of their hardware structure along with their increase in size and greater functions, and this is making it difficult to verify their logic quality. Large-scale imaging LSIs, in particular, have a higher complexity of software processing that they perform with their multi-core structures and increasingly complicated structure of internal buses. This makes it essential to ensure that their functions, performance and power consumption satisfy the requirements by conducting system verification at the pre-silicon stage. However, product development with a short turnaround time (TAT) is called for because of the recent shorter model change cycles, and efficiently conducting system verification while ensuring quality is very important. Fujitsu Semiconductor is making positive use of a hardware emulator (a device capable of mapping the circuit to be verified to dedicated hardware for running it at high speed) to establish verification technology that ensures quality and improves verification efficiency at the same time, and applying it to product models. This paper presents hardware/software co-verification, performance verification and power consumption estimation by using a hardware emulator, which we believe is very effective for verifying large-scale imaging LSIs.

• CHIP-PKG-PCB Co-Design Methodology

For digital devices integrating an imaging LSI, performance improvement, cost cutting and reduction of the time to market are essential conditions for surviving in the increasingly competitive global market. At the same time, more and more imaging LSIs are taking the form of large-scale system on a chip (SoC). It is becoming harder to design them as the degree of integration increases, and signal and power integrity issues are appearing due to their speed increase and voltage reduction. As a design methodology to address four challenges in recent SoC design (signal and power integrity issues, high-density design, reduction of design turnaround time [TAT] and cost cutting), Fujitsu Semiconductor has established chip-package-printed circuit board (CHIP-PKG-PCB) co-design methodology and made achievements contributing to first-shot full operation of SoCs and digital devices that integrate them. This paper presents our approach to the CHIP-PKG-PCB co-design for dealing with the four challenges above by giving case examples.

• Packaging Technology for Imaging LSI

The main function of a semiconductor package is to reliably transmit electric signals from minute electrode pads formed on an LSI chip (semiconductor chip) to the mounted terminals of the semiconductor package. To ensure a good LSI performance, it is becoming important to optimize the package structure according to the characteristics of the LSI and product use environment. Imaging LSIs must process a large amount of image data at high speeds, which makes it necessary to have a high-capacity external buffer memory and high-speed, high-quality data communication. Semiconductor package structures are becoming more elaborate so that such LSIs can reliably perform these operations. Imaging LSIs are used in mobile devices such as digital cameras, mobile phones and smartphones, and they adopt high-density mounting technologies including packaging technology to miniaturize them and give them a system in package (SiP) structure with mixed mounting of multiple LSIs. This paper presents advanced semiconductor packaging technologies used in semiconductor packages for imaging LSIs.