************************************************************************** ** ** ** InfiniBand Card (EDR,100Gbps)/PHE2OIB1, ** ** (HDR100,100Gbps)/PHE2OIB2 ** ** Firmware Update Procedure version: 3.0 ** ** ** ************************************************************************** Fujitsu Limited -------------------------------------------------------------------------- [High Risk Activity] The Customer acknowledges and agrees that the Product is designed, developed and manufactured as contemplated for general use, including without limitation, general office use, personal use, household use, and ordinary industrial use, but is not designed, developed and manufactured as contemplated for use accompanying fatal risks or dangers that, unless extremely high safety is secured, could lead directly to death, personal injury, severe physical damage or other loss (hereinafter "High Safety Required Use"), including without limitation, nuclear reaction control in nuclear facility, aircraft flight control, air traffic control, mass transport control, medical life support system, missile launch control in weapon system. The Customer, shall not use the Product without securing the sufficient safety required for the High Safety Required Use. In addition, Fujitsu (or other affiliate's name) shall not be liable against the Customer and/or any third party for any claims or damages arising in connection with the High Safety Required Use of the Product. -------------------------------------------------------------------------- This document describes the InfiniBand Card firmware update procedure. The contents of the document is shown as follows. [contents] 1. About using this product 2. Target model 3. Support environment 4. Revision history(modified item) 5. Installation procedures 6. Trademark and copyright -------------------------------------------------------------------------- 1. About using this product When using this software, please follow the license agreements below. If you are unable to follow them, please do not use this software. License agreements 1) This software is confirmed usable under the target model described latter. 2) Redistribution of this software is not permitted. 3) This software may not be modified, reverse engineered, or disassembled. 4) You may not change any copyright notices for this software. 5) Even if damage occurs by using this software, our company does not assume any responsibility. -------------------------------------------------------------------------- 2. Target model InfiniBand Card (EDR,100Gbps)/PHE2OIB1 InfiniBand Card (HDR100,100Gbps)/PHE2OIB2 -------------------------------------------------------------------------- 3. Support environment - Refer to the following website for the support of distributions and kernels. https://www.fujitsu.com/global/products/computing/servers/supercomputer/documents/ -------------------------------------------------------------------------- 4. Revision history (modified item) version 1.0 (20/07/20): First Release version 2.0 (21/01/08): Add InfiniBand Card (HDR100,100Gbps) to Target model version 3.0 (21/02/05): Correct errors -------------------------------------------------------------------------- 5. Installation procedures Install as follows. (OFED must be installed.) 5.1 Installation procedures 1) Transfer the download file. Transfer the download file to FX700 node by using scp command, etc. The following step assumes that the file have been transferred to the /SOMEWHERE directory on the FX700 node. 2) Change to the super-user (root) privilege. $ su - 3) Unpack the download file. Perform the following command on the FX700 node to unpack the file. # gunzip .bin.gz 4) Check the MST device name. Perform the following command on the FX700 node to check the device name. In this document, the device name is /dev/mst/mt4119_pciconf0. If it is different, replace it. # mst start # mst status MST modules: ------------ MST PCI module is not loaded MST PCI configuration module loaded MST devices: ------------ /dev/mst/mt4119_pciconf0 - PCI configuration cycles access. domain:bus:dev.fn=0000:05:00.0 addr.reg=88 data.reg=92 cr_bar.gw_offset=-1 Chip revision is: 00 5) Apply the bin file. Perform the following command on the FX700 node to apply the bin file. # flint -d /dev/mst/mt4119_pciconf0 -y -i .bin burn Current FW version on flash: 16.26.1040 New FW version: 16.27.1016 Initializing image partition - OK Writing Boot image component - OK -I- To load new FW run mlxfwreset or reboot machine. 6) Change the Relaxed order setting. Perform the following command on the FX700 node to change the setting. # mlxconfig -d /dev/mst/mt4119_pciconf0 set PCI_WR_ORDERING=1 7) Reflect setting. Perform the following command on the FX700 node to restart the node. # shutdown -r now 8) Check the installation status. Perform the following command on the FX700 node. If all states are success, installation has been successfully completed. $ mst start Starting MST (Mellanox Software Tools) driver set Loading MST PCI module - Success Loading MST PCI configuration module - Success Create devices Unloading MST PCI module (unused) - Success 9) Check the version. Perform the following command on the FX700 node to check that the firmware version is updated. $ ibstat CA 'mlx5_0' CA type: MT4119 Number of ports: 1 Firmware version: 16.27.1016 Hardware version: 0 Node GUID: 0xb8599f0300c3e80e System image GUID: 0xb8599f0300c3e80e (Omitted) 10) Check the Relaxed order setting. Perform the following command on the FX700 node to make sure the Relaxed order setting is force_relax(1). $ mlxconfig -d /dev/mst/mt4119_pciconf0 query | grep PCI_WR_ORDERING PCI_WR_ORDERING force_relax(1) -------------------------------------------------------------------------- 6. Trademark and copyright Red Hat and Red Hat Enterprise Linux are trademarks of Red Hat, Inc., registered in the U.S. and other countries. Linux(R) is the registered trademark of Linus Torvalds in the U.S. and other countries. All the other trademarks are trademarks of the company that owns them. Copyright FUJITSU LIMITED 2020 --------------------------------------------------------------------------