Archived content
NOTE: this is an archived page and the content is likely to be out of date.
Why Invest in UNIX Server Infrastructure Now?
Commoditization of system platforms continues, but tangible business value is achieved through the differentiation and specialized features found in modern UNIX servers.
Fujitsu has and continues to provide world-leading capabilities in the design and development of UNIX servers, from processors to server chassis. Fujitsu server developers create and implement cutting edge technologies for mainframes, supercomputers, and UNIX servers. Turning technology into value for customers is a challenging but rewarding task for Fujitsu engineers across the development teams. Let's meet some of the engineers behind the Fujitsu M10 server.
Three Innovative Goals of Fujitsu M10
Hideyuki Unno
Hideyuki Unno, with a background in UNIX server processor circuit design, was the chief developer of the entire Fujitsu M10 system. He speaks here about the innovative goals Fujitsu has held for the Fujitsu M10 server line.
The current Fujitsu M10 UNIX server achieved the high reliability required for mission-critical systems and was developed with a focus on flexibility to handling future changes, including the explosive growth in compute requirements from Big Data. In preceding UNIX server generations we put a lot of effort into compatibility, and that may have resulted in an image of UNIX servers as being "obsolete". But, in developing the Fujitsu M10, we have put so many state-of-the-art designs and technologies into the system to improve performance and functionality that almost no part was left untouched.
In the development of Fujitsu M10, we placed particular importance on three areas:
Extreme Scalability. The new Building Block Architecture enabled us to achieve scalability from just 16 cores on a single CPU all the way up to 16 Building Blocks with 1024 cores on 64 CPUs. Fujitsu M10 CPU Activation is also used to flexibly add resources one core at a time. Together, Building Blocks and CPU Activation make it possible to expand the system according to customers business needs without large upfront costs.
Reliability. This is often considered the DNA of Fujitsu. In order to achieve high reliability and availability of server systems, thoroughly establishing data redundancy and recovery mechanisms at the circuit design phase is required. When failures occur we must have the ability to limit the impact and ensure business continuity through dynamic downgrading and hot-swap features. This requires a robust LSI circuit design and an overall system design that includes the operating system, system monitoring facility (XSCF), and maintenance procedures to ensure disruptions are minimized. Fujitsu M10 exemplifies the technologies and experience that Fujitsu engineers have honed over decades of mainframe design experience, and has led Fujitsu M10 to achieve industry-leading reliability.
High-Density Packaging. While achieving scalability with Building Block architecture, Fujitsu M10 also takes on the challenge of saving space with a high-density chassis. The system performance is improved by integrating memory controllers into the CPU and reducing the physical distance between CPUs and memory. As the density increases, cooling becomes more challenging. Accordingly, we have developed our own innovative Liquid Loop Cooling system. With the cooperation between cooling system, chassis structure, and printed circuit board engineers and designers, we were able to pack high performance in a compact chassis.
Extreme Scalability and Flexible Resource Expansion Capabilities that Reflect the Voice of the Customer
Hiromi Fukumura
Hiromi Fukumura, who has experience in the development of supercomputers and UNIX server processors and was in charge of the Fujitsu M10 system specification, here talks about scalability and flexibility.
Development of new products requires an evolution over previous system designs in order to improve functionality and performance. However, we must avoid losing any value that is important to customers while making such changes. In developing the Fujitsu M10, we have listened to the voice of customers through sales and marketing, system engineers, and maintenance personnel to design a system that meets and exceeds customer expectations.
For instance, Fujitsu M10 employs a 16 core per socket CPU. If we made 16 cores per socket the minimum configuration, users who have been using the previous generation Fujitsu SPARC servers in smaller configurations would have to pay increased software license fees. A lot of thought and discussion went into ways to balance the very large processing power of a 16 core CPU with the needs of many different types of customers. This led to the development of "CPU Activation".
With CPU Activation, customers can initially deploy the system in a small-scale configuration and then purchase additional resources and add to the system as needed.
With Fujitsu M10, we also made drastic changes to how memory is controlled and shared between processors. In our previous generation servers, to enable all processors to access the same memory globally we had adopted a method that broadcasts memory access requests to the entire system. With this method the broadcast performance determines the overall system performance. To improve the performance of this broadcast method, we incorporated creative ideas such as chip partitioning and faster transmission speeds. In order to further improve performance, we had to switch from the broadcast method to a new way.
We, therefore, made a drastic change in the shared memory architecture, from broadcast to the directory method. As the name implies, the directory method controls the consistency of cache memory by storing and maintaining usage information such as if the memory area is shared or not in a directory. This eliminates the need to broadcast each access request and enables the system to scale to a higher level. In addition, access to local memory directly attached to a processor is also made faster.
Broadcast Method:
(1) Reference to cache memory
(2) In case of cache miss, request for data is made to the system controller
(3) Check other processors for the existence of requested data through broadcast to all system controllers
(4) If the requested data is not in other processor's cache memory, data is transferred from memory
Directory Method:
(1) Reference to cache memory
(2) In case of cache miss, a reference is made to the directory to determine if the requested data is in the cache memory of other processors.
(3) Data is transferred from memory
Transferring data from local memory can be completed very quickly without communicating with other processors.
At the same time we had to prevent any negative impacts from the switch to the directory method. As access to local memory is made faster, resource management becomes more important. The processors on which programs run and the memory that these programs use should be located closely together to provide the best performance. To accomplish this and prevent users from having to consider processor and memory locality, the operating system was enhanced to automatically handle resource scheduling for Fujitsu M10 servers. Also, a star-type interconnect is used between chassis in large Fujitsu M10-4S configurations with more than 4 chassis. This increases the transfer speed between nodes and provides maintainability allowing Building Blocks to be replaced without stopping the entire system.
The Fujitsu M10 succeeds with both improved performance and new functionality, and with an evolution of the key customer values acquired over our long history of processor and system development.
A Quest for Unsurpassed Reliability
Yoshiki Mizusawa
Yoshiaki Mizusawa, who has been responsible for UNIX server reliability, availability, and serviceability (RAS) specifications, speaks about the reliability built in to Fujitsu M10.
Fujitsu UNIX servers are used for customers' mission-critical applications and are expected to operate 24 hours a day, 365 days a year. Fujitsu M10 is equipped with high reliability technologies honed from decades of Fujitsu mainframe experience. This extreme reliability is often referred to as "mainframe-class RAS" and is implemented at LSI (Large Scale Integration) chip, component, and system levels. Stacking up reliability at each level improves the overall availability of the system.
- Stacking Up Reliability
- System Level Reliability
Cluster System
Storage/Network Redundancy
Component Level Reliability
Redundancy, Hot-Swap, Status Monitoring, Failure Notification
LSI Chip Level Reliability
Error Checking Mechanisms (Parity, ECC) - Uninterrupted Business
Fault Tolerance
Data Integrity
How did we do it? First, circuits are designed with detailed error checking and recovery functions at the LSI chip level. In the Fujitsu M10 systems, every data path and storage area, such as registers and memory, is protected by redundant coding. Arithmetic results are protected by parity prediction and residue checking. These thorough data protection mechanisms as well as detailed error checking by a number of error checkers, recovery processes, and downgrade functions are at the heart of Fujitsu hardware design.
Next, enabling redundancy, hot-swap, and downgrade at the component level such as disk, power supply, and fan units assures business can continue even in the case of component failure.
Finally, high reliability is achieved at the system level with hardware, firmware, operating system, and middleware, including clusterware, all integrated together. It is, of course, desirable for a large-scale complex computing system to never fail. But once it happens, quickly identifying the cause and restoring the system becomes very important. For Fujitsu M10, the behavior of the entire system, including operating system and middleware, is designed and verified for end-to-end RAS. In case of a failure, applications can continue to operate on the system with the minimum component(s) downgraded while the cause of the failure is accurately identified and replaced.
A major factor in this achievement is that hardware, firmware, and operating system designers as well as the quality assurance department all worked together in the development. Fujitsu M10 is built on technologies and experience cultivated over decades and highlights the strength and philosophy of Fujitsu.
High Performance Packed into a Compact Chassis through Brainstorming and Teamwork
Kayako Kawano
Kayako Kawano, who has significant experience in the development of mainframes and now leads the development of printed circuit boards for Fujitsu UNIX servers, recalls the challenges in high-density packaging.
Achieving high-density packaging was a huge challenge. Fujitsu M10-4 and M10-4S Building Blocks have 4 CPUs (with a max of 64 cores) and up to 64 memory modules in a 4U chassis. For this, we adopted the unique idea of stacking two system boards, each fitted with two CPUs and 32 memory modules, one on top of the other. If you look inside the chassis, you will see the three-dimensional structure and how densely it is packed.
High-density packaging not only reduces footprint for datacenter savings, but also plays a significant role in improving system performance.
For Fujitsu M10, we have developed our own hybrid cooling system called Liquid Loop Cooling, which combines liquid cooling to circulate coolant inside the chassis and fan-based air cooling to remove heat from the chassis. This eliminates the need for large heat sinks on the processors, and processors no longer need to be placed near fans because the coolant can transfer heat effectively across larger distances. This greatly enhances the flexibility in laying out the system.
- Mounting Distance Between Processor and Memory
- CPU Modules
- Memory Controllers
- Memory Modules
- CPU Modules (with on-chip memory controllers)
SPARC64 X and X+ processors have on-chip memory controllers directly connected to 16 memory modules per CPU socket without the use of buffer chips. The processors and memory modules can be placed very closely together thanks to our Liquid Loop Cooling technology. In combination, these innovations provide high bandwidth and low latency memory access.
Actually, directly connecting 16 DDR3 1600MHz memory modules to the processor is extremely difficult in terms of transmission conditions. From the stand point of transmission efficiency, it is desirable to make the distance between memory and the CPU as short as possible. In contrast, from the stand point of cooling the system, the distance between components should be as large as possible to provide efficient air flow. Even with Liquid Loop Cooling, enough space should be allowed for coolant pipes. After many heated discussions, many design reviews and more than 10 different prototypes, the present design was finally achieved. As a result, we were able to pack a high performance system into a more compact chassis. The chassis is 4U in size, and therefore the maximum configuration of 16 Building Blocks can fit in two standard racks. And, chassis size was not the only consideration; the system must, of course, be serviceable. It was quite an achievement getting everything packaged tightly together, with sufficient air flow and the ability to maintain fans, hard disks, and PCI cards; all without having to remove the chassis from the rack.
Fujitsu UNIX server innovation will continue to advance our customers' business.
Hideyuki Unno, Manager
Second ES Development Department
Mission-Critical Server Division
Enterprise Server Business Unit
Hiromi Fukumura
Second ES Development Department
Mission-Critical Server Division
Enterprise Server Business Unit
Yoshiki Mizusawa
Third ES Development Department
Mission-Critical Server Division
Enterprise Server Business Unit
Kayako Kawano
Third ES Development Department
Mission-Critical Server Division
Enterprise Server Business Unit
