*************************************************************************** ** ** ** Software Information for vSphere 7.0, 8.0 ** ** ** ** VMware Multi-Pathing plug-in for ETERNUS AB/HB ** ** 1.0.0 ** ** ** ** Dec 2024 FUJITSU LIMITED ** *************************************************************************** Preface This guide explains installation/uninstallation procedures and notes of VMware Multi-Pathing plug-in for ETERNUS AB/HB. Abbreviations In this document, the disk storage system supported by this product is referred to as "ETERNUS storage system". In this document, VMware's Knowledge Base is described as "KB". More information on each KB can be found on the VMware website. Reference - Fujitsu Disk storage system ETERNUS AB/HB series https://www.fujitsu.com/global/products/computing/storage/all-flash-hybrid.html Trademarks Microsoft, Windows, Windows Server are either registered trademarks or trademarks of Microsoft Corporation in the United States and other countries. VMware is a registered trademark of VMware,Inc, in the United States and other countries. All other company/product names mentioned herein maybe trademarks or registered trademarks of their respective holders and are used for identification purposes only. High Safety Required This Product is designed, developed and manufactured as contemplated for general use, including without limitation, general office use, personal use, household use, and ordinary industrial use, but is not designed, developed and manufactured for use in situations with accompanying fatal risks or dangers that, unless extremely high safety is secured, could lead directly to death, personal injury, severe physical damage or other loss (hereinafter "High Safety Required Use"), including without limitation, nuclear reaction control in nuclear facility, aircraft flight control, air traffic control, mass transport control, medical life support system, and missile launch control in weapon systems. Do not use this Product for High Safety Required Use without securing the sufficient safety level required. If you wish to use this Product for High Safety Required Use, please consult with our sales representative before such use. 1. Introduction 1-1. VMware Multi-Pathing plug-in for ETERNUS AB/HB VMware Multi-Pathing plug-in for ETERNUS AB/HB is offered as a sub plug-in of Native Multipathing Plugin (NMP) of the VMware standard used when the multipath is connected with the ETERNUS storage system. VMware Multi-Pathing plug-in for ETERNUS AB/HB is Storage Array Type Plug-in(SATP) that controls error handling corresponding to the ETERNUS storage system. 1-2. Audience This document is intended for VMware administrators who are using ETERNUS storage system as an infrastructure of virtualization deployments. It is assumed that the reader has a general knowledge of VMware and ETERNUS storage systems. 1-3. Release Information 1-3-1. Modification History +----------------+--------------------------------------------------+ | version | Modification effect | +----------------+--------------------------------------------------+ | 1.0.0 | New Release | +----------------+--------------------------------------------------+ 1-3-2. Updates +--------------+---------+------------------------------------------+ | date | version | changes | +--------------+---------+------------------------------------------+ | Feb 28, 2023 | 1.0.0 | first version | +--------------+---------+------------------------------------------+ | Jul 21, 2023 | 1.0.0 | - Added support OS | | | | - VMware vSphere 8.0 Standard | | | | - VMware vSphere 8.0 Enterprise Plus | +--------------+---------+------------------------------------------+ | Dec 26, 2024 | 1.0.0 | - This document was corrected. | | | | - Added 7. Notes Regarding to PDL | | | | (Permanent Device Loss) | +--------------+---------+------------------------------------------+ 2. Requirements This section describes support environment and necessary resources. Please be sure to check this section before installing VMware Multi-Pathing plug-in for ETERNUS AB/HB. 2-1. Supported Environment Please visit the following product's web site for the latest information (supported models and operating systems, important notes, etc.). https://www.fujitsu.com/global/support/products/computing/storage/download/vmware-mp-plug-in-abhb.html 2-2. Required Disk Capacity for Installation The following table shows the static disk space requirements for each directory used to install VMware Multi-Pathing plug-in for ETERNUS AB/HB. +----------+----------------+----------------+----------------------+ |No |Directory |Disk Capacity |Note | +----------+----------------+----------------+----------------------+ |1 |/usr |0.1MB | | +----------+----------------+----------------+----------------------+ |2 |/etc |0.1MB | | +----------+----------------+----------------+----------------------+ 3. Product configuration The product configuration of VMware Multi-Pathing plug-in for ETERNUS AB/HB is as follows. VMwareVAMPAHB_1.0.0 +-- FJT_satp_eternus_AHB_100.zip File used for installation +-- readme.txt Software Information(JP) +-- readme_en.txt Software Information(EN) 4. Installation and Uninstallation procedure 4-1. Installation This section describes three methods to install VMware Multi-Pathing plug-in for ETERNUS AB/HB. You may select method based on your environment. - Using ESXCLI - Using esxcli command - Using vSphere Lifecycle Manager All commands must be run as root user in this section. Stop GuestOS or move GuestOS to other hosts before it installs. Specify the IP address of ESXi host at of the following procedures. The procedure to install the upgrade of VMware Multi-Pathing plug-in for ETERNUS AB/HB is the same as a new installation. 4-1-1. Installation VMware Multi-Pathing plug-in for ETERNUS AB/HB 4-1-1-1. Installation using ESXCLI This section describes how to install using ESXCLI. 1) Copy the installation file to a local directory on your ESXi host. FJT_satp_eternus_AHB_130.zip is preserved in "/tmp" in the example of executing the following commands. 2) Open a command prompt and change to the directory in which ESXCLI is installed only Windows. For default, installed pass is as follows. >cd C:\Program Files (x86)\VMware\esxcli 3) Run the following command to place the ESXi host into maintenance mode. >esxcli --server system maintenanceMode set --enable=1 4) Run the following command to install VMware Multi-Pathing plug-in for ETERNUS AB/HB into the ESXi host. >esxcli --server software vib install -d /tmp/FJT_satp_eternus_AHB_100.zip 5) Run the following command to reboot the ESXi host. >esxcli --server system shutdown reboot -d 10 -r satpinstall 6) Run the following command to confirm VMware Multi-Pathing plug-in for ETERNUS AB/HB installation. >esxcli --server software vib list example: Name Version Vendor Acceptance Level Install Date -------------------- --------- ------- ---------------- ------------ fjt-satp-eternus-ahb 1.0.0-0 FJT VMwareAccepted 2023-01-06 7) Run the following command to confirm it is a setting that uses FJT_SATP_ETERNUS_AHB. >esxcli --server storage nmp satp rule list --satp FJT_SATP_ETERNUS_AHB example: Name Device Vendor Model Driver Transport Options Rule Group Claim Options Default PSP PSP Options Description -------------------- ------ ------- ------------- ------ --------- ------- ---------- ------------- ----------- ----------- ----------- FJT_SATP_ETERNUS_AHB FUJITSU ^ETERNUS_AHB* user VMW_PSP_RR 8) Run the following command to confirm it is a setting that uses SATP. Please confirm FJT_SATP_ETERNUS_AHB is displayed in "Storage Array Type:". >esxcli --server storage nmp device list example: naa.6d039ea000019d5600005cb1639949e4 Device Display Name: FUJITSU Fibre Channel Disk (naa.6d039ea000019d5600005cb1639949e4) Storage Array Type: FJT_SATP_ETERNUS_AHB Storage Array Type Device Config: {FJT_SATP_ETERNUS_AHB device config options dtime=20 mtime=3 limit=6} Path Selection Policy: VMW_PSP_RR Path Selection Policy Device Config: {policy=rr,iops=1000,bytes=10485760,useANO=0; lastPathIndex=1: NumIOsPending=0,numBytesPending=0} Path Selection Policy Device Custom Config: Working Paths: vmhba69:C0:T0:L68, vmhba6:C0:T0:L68 Is USB: false 9) Run the following command to exit maintenance mode. >esxcli --server system maintenanceMode set --enable=0 4-1-1-2. Installation using esxcli command This section describes how to install using esxcli command. Perform the following steps on the ESXi Tech Support Mode. 1) Copy the installation file to a local directory on your ESXi host. FJT_satp_eternus_AHB_100.zip is preserved in "/tmp" in the example of executing the following commands. 2) Place the ESXi host into maintenance mode. Use vSphere Client or Host Client. 3) Run the following command from the local directory where you copied the FJT_satp_eternus_AHB_100.zip file to install VMware Multi-Pathing plug-in for ETERNUS AB/HB into the ESXi host. # esxcli software vib install -d /tmp/FJT_satp_eternus_AHB_100.zip 4) Reboot the ESXi host. Use vSphere Client or Host Client. 5) Run the following command to VMware Multi-Pathing plug-in for ETERNUS AB/HB installation is confirmed. # esxcli software vib list example: Name Version Vendor Acceptance Level Install Date -------------------- --------- ------ ---------------- ------------ fjt-satp-eternus-ahb 1.3.0-0 FJT VMwareAccepted 2023-01-06 6) Run the following command to confirm it is a setting that uses FJT_SATP_ETERNUS_AHB. # esxcli storage nmp satp rule list --satp FJT_SATP_ETERNUS_AHB example: Name Device Vendor Model Driver Transport Options Rule Group Claim Options Default PSP PSP Options Description -------------------- ------ ------- ------------ ------ --------- ------- ---------- ------------- ----------- ----------- ----------- FJT_SATP_ETERNUS_AHB FUJITSU ^ETERNUS_AHB* user VMW_PSP_RR 7) Run the following command to confirm it is a setting that uses SATP. Please confirm FJT_SATP_ETERNUS_AHB is displayed in "Storage Array Type:". # esxcli storage nmp device list example: naa.6d039ea000019d5600005cb1639949e4 Device Display Name: FUJITSU Fibre Channel Disk (naa.6d039ea000019d5600005cb1639949e4) Storage Array Type: FJT_SATP_ETERNUS_AHB Storage Array Type Device Config: {FJT_SATP_ETERNUS_AHB device config options dtime=20 mtime=3 limit=6} Path Selection Policy: VMW_PSP_RR Path Selection Policy Device Config: {policy=rr,iops=1000,bytes=10485760,useANO=0; lastPathIndex=1: NumIOsPending=0,numBytesPending=0} Path Selection Policy Device Custom Config: Working Paths: vmhba69:C0:T0:L68, vmhba6:C0:T0:L68 Is USB: false 8) ESXi host exit maintenance mode. Use vSphere Client or Host Client. 4-1-1-3. Installation using vSphere Lifecycle Manager Refer to the "Managing Host and Cluster Lifecycle". Reboot the ESXi host after installation. 4-2. Uninstallation This section describes two methods to uninstall VMware Multi-Pathing plug-in for ETERNUS AB/HB. You can use one of two methods for your environment. - Using ESXCLI - Using esxcli command Stop GuestOS or move GuestOS to other hosts before it uninstalls. Specify the IP address of ESXi host at of the following procedures. 4-2-1. Uninstallation VMware Multi-Pathing plug-in for ETERNUS AB/HB 4-2-1-1. Uninstallation using ESXCLI This section describes how to uninstall VMware Multi-Pathing plug-in for ETERNUS AB/HB using ESXCLI. Perform the following steps on server in which ESXCLI is installed. 1) Open a command prompt and change to the directory in which ESXCLI is installed only Windows. For default, installed pass is as follows. >cd C:\Program Files (x86)\VMware\esxcli 2) Run the following command to place the ESXi host into maintenance mode. >esxcli --server system maintenanceMode set --enable=1 3) Run the following command to uninstall. >esxcli --server software vib remove -n fjt-satp-eternus-ahb 4) Run the following command to confirm the package is not displayed. >esxcli --server software vib list 5) Run the following command to release the setting that uses FJT_SATP_ETERNUS_AHB. >esxcli --server storage nmp satp rule remove --satp="FJT_SATP_ETERNUS_AHB" --vendor="FUJITSU" --model="^ETERNUS_AHB*" --psp="VMW_PSP_RR" After that, the following error occur depending on the case. Error deleting SATP rule: No matching user rule found for SATP FJT_SATP_ETERNUS_AHB matching vendor FUJITSU model ^ETERNUS_AHB* claim Options PSP VMW_PSP_RR and PSP Options If the error occur, please execute the following (7) command, and there is no display FJT_SATP_ETERNUS_AHB information, then that is no problem. >esxcli --server storage nmp satp rule list 6) Run the following command to reboot the ESXi host. >esxcli --server system shutdown reboot -d 10 -r satpinstall 7) Run the following command to confirm the setting has been deleted. >esxcli --server storage nmp satp rule list --satp FJT_SATP_ETERNUS_AHB 8) Run the following command to confirm it is a return to standard VMW_SATP_ALUA. Please confirm VMW_SATP_ALUA is displayed in "Storage Array Type:". >esxcli --server storage nmp device list example: naa.6d039ea000019d5600005cb1639949e4 Device Display Name: FUJITSU Fibre Channel Disk (naa.6d039ea000019d5600005cb1639949e4) Storage Array Type: VMW_SATP_ALUA Path Selection Policy Device Config: {policy=rr,iops=1000,bytes=10485760,useANO=0; lastPathIndex=1: NumIOsPending=0,numBytesPending=0} Path Selection Policy: VMW_PSP_RR Path Selection Policy Device Config: {policy=rr,iops=1000,bytes=10485760,useANO=0; lastPathIndex=1: NumIOsPending=0,numBytesPending=0} Path Selection Policy Device Custom Config: Working Paths: vmhba69:C0:T0:L68, vmhba6:C0:T0:L68 Is USB: false 9) Run the following command to exit maintenance mode. >esxcli --server system maintenanceMode set --enable=0 4-2-1-2. Uninstallation using esxcli command This section describes how to uninstall VMware Multi-Pathing plug-in for ETERNUS AB/HB using esxcli command. Perform the following steps on the ESXi host Tech Support Mode. 1) Place the ESXi host into maintenance mode. Use vSphere Client or Host Client. 2) Run the following command to uninstall. # esxcli software vib remove -n fjt-satp-eternus-ahb 3) Run the following command to confirm the package is not displayed. # esxcli software vib list 4) Run the following command to release the setting that uses FJT_SATP_ETERNUS_AHB. # esxcli storage nmp satp rule remove --satp="FJT_SATP_ETERNUS_AHB" --vendor="FUJITSU" --model="^ETERNUS_AHB*" --psp="VMW_PSP_RR" After that, the following error occur depending on the case. Error deleting SATP rule: No matching user rule found for SATP FJT_SATP_ETERNUS_AHB matching vendor FUJITSU model ^ETERNUS_AHB* claim Options PSP VMW_PSP_RR and PSP Options If the error occur, please execute the following (6) command, and there is no display FJT_SATP_ETERNUS_AHB information, then that is no problem. # esxcli storage nmp satp rule list 5) Reboot the ESXi host. Use vSphere Client or Host Client. 6) Run the following command to confirm the setting has been deleted. # esxcli storage nmp satp rule list --satp FJT_SATP_ETERNUS_AHB 7) Run the following command to confirm it is a return to standard VMW_SATP_ALUA. Please confirm VMW_SATP_ALUA is displayed in "Storage Array Type:". # esxcli storage nmp device list example: naa.6d039ea000019d5600005cb1639949e4 Device Display Name: FUJITSU Fibre Channel Disk (naa.6d039ea000019d5600005cb1639949e4) Storage Array Type: VMW_SATP_ALUA Storage Array Type Device Config: {implicit_support=on; explicit_support=off; explicit_allow=on; alua_followover=on; action_OnRetryErrors=off; {TPG_id=0,TPG_state=ANO}{TPG_id=1,TPG_state=AO}} Path Selection Policy: VMW_PSP_RR Path Selection Policy Device Config: {policy=rr,iops=1000,bytes=10485760,useANO=0; lastPathIndex=1: NumIOsPending=0,numBytesPending=0} Path Selection Policy Device Custom Config: Working Paths: vmhba69:C0:T0:L68, vmhba6:C0:T0:L68 Is USB: false 8) ESXi host exit maintenance mode. Use vSphere Client or Host Client. 5. About Path Recovery If the diagnostic continues to respond normally for more than 20 minutes, return the path state from dead to online. Also, if the online to dead transition is repeated 6 times within 3 hours from the first dead transition, it is judged to be an unstable path and the status changes to "fataldead". "fataldead" is not automatically recovered by diagnostics. 5-1 How to recover from fataldead After correcting the error, restore the path as follows. 1) Check for fataldead in the following ways:. # esxcli storage nmp path list or # esxcli storage nmp path list -d naa.XXXXXXXXXXXXXXX example: # esxcli storage nmp path list -d naa.6d039ea000019d5600005cb1639949e4 fc.200000109b3dba8b:100000109b3dba8b-fc.2006d039ea019d88:2017d039ea019d88-naa.6d039ea000019d5600005cb1639949e4 Runtime Name: vmhba6:C0:T0:L2 Device: naa.6d039ea000019d5600005cb1639949e4 Device Display Name: FUJITSU Fibre Channel Disk (naa.6d039ea000019d5600005cb1639949e4) Group State: dead Array Priority: 0 Storage Array Type Path Config: {FJT_SATP_ETERNUS_AHB path vmhba6:C0:T0:L2 config info: vendor FUJITSU model ETERNUS_AHB FATALDEAD} Path Selection Policy Path Config: PSP VMW_PSP_RR does not support path configuration. A path in the fataldead state will show FATALDEAD in "Storage Array Type Path Config". 2) Release fataldead with the following command: # esxcli storage nmp satp generic pathconfig set -p "Runtime Name" --config="fataldead=off" -p must be the "Runtime Name" of 1). example: # esxcli storage nmp satp generic pathconfig set -p vmhba3:C0:T2:L0 --config="fataldead=off" 3) Verify that the fataldead state of the path is cleared with the following command: # esxcli storage nmp path list -p vmhbaX:X:X:X example: # esxcli storage nmp path list -p vmhba3:C0:T2:L0 fc.200000109b3dba8b:100000109b3dba8b-fc.2006d039ea019d88:2017d039ea019d88-naa.6d039ea000019d5600005cb1639949e4 Runtime Name: vmhba6:C0:T0:L2 Device: naa.6d039ea000019d5600005cb1639949e4 Device Display Name: FUJITSU Fibre Channel Disk (naa.6d039ea000019d5600005cb1639949e4) Group State: dead Array Priority: 0 Storage Array Type Path Config: {FJT_SATP_ETERNUS_AHB path vmhba6:C0:T0:L2 config info: vendor FUJITSU model ETERNUS_AHB} Path Selection Policy Path Config: PSP VMW_PSP_RR does not support path configuration. Notes: * The Group State immediately following the command is dead. Transition to active at next diagnostic. * Only the Runtime Name specified by -p will be fataldead. Do 1) to 3) for each. 6. Message Output to /var/log/vmkernel.log. Message: FJT_SATP_ETERNUS_AHB: vmhbaX:CX:TX:LX XX->YY Meaning: Received XX host error and converted to YY. Action: If a path transition occurs, investigate the cause of the XX error. Message: FJT_SATP_ETERNUS_AHB: vmhbaX:CX:TX:LX is fataldead Meaning: Transition to fataldead successful. Action: Investigate the cause of the error. After removing the cause, release fataldead as described in "5-1 How to recover from fataldead". 7. Notes - About updating the SANtricity of ETERNUS storage system Please execute the following steps. 1) Run the 5 -1 2) Command on all LUNs and paths. or the following command: # for i in `esxcfg-mpath -L | grep naa.XXXXXXXXXXX | awk '{print $1}'`; do esxcli storage nmp satp generic pathconfig set -p $i --config="fataldead=off" ; done naa.XXXXXXXXXX should specify the common part of the farm replacement unit. 2) Then, Upgrade the firmware. In case of changing to dead state during upgrading SANtricity, 1)'s step is prevented from changing to fataldead state when reaching to recurrence count. The current recurrence count will be cleared by 1)'s command. - Timeout on Guest OS If activating virtual disk of RDM on GuestOS of Linux, set the timeout period to at least 40 seconds. Ex. Red Hat Enterprise Linux7) Create "/etc/udev/rules.d/99-rdm-scsi-udev.rules" file. Add the following three lines. ACTION=="add", SUBSYSTEMS=="scsi", ATTRS{vendor}=="FUJITSU ", ATTRS{model}=="ETERNUS_AHB ", RUN+="/bin/sh -c 'echo 40 >/sys$DEVPATH/timeout'" Ex. Red Hat Enterprise Linux8, and Red Hat Enterprise Linux9) Create "/etc/udev/rules.d/99-rdm-scsi-udev.rules" file. Add the following three lines. ACTION=="add", SUBSYSTEMS=="scsi", ATTRS{vendor}=="FUJITSU ", ATTRS{model}=="ETERNUS_AHB ", ENV{DEVTYPE}=="disk", RUN+="/bin/sh -c 'echo 40 >/sys$DEVPATH/device/timeout'" If you are not using an RDM to show the LUN to the guest, refer to the KB below to set the timeout value. https://knowledge.broadcom.com/external/article?legacyId=1009465 If you do not change the timeout value, the guest may detect the timeout before the host detects the timeout, causing an I/O error or LUN unavailability. - Regarding to upgrading VMware ESXi If the using version of VMware Multi-Pathing plug-in for ETERNUS AB/HB doesn't adapt VMware ESXi's version, please do the following. 1. First, Un-install VMware Multi-Pathing plug-in for ETERNUS AB/HB. 2. Upgrade VMware ESXi. 3. After that, install the compatible version's VMware Multi-Pathing plug-in for ETERNUS AB/HB, and setup it. (Please refer the install/uninstall sequences.) - Regarding to execution of the maintenance work On the connecting server, when the changing to "one-sided" pathing of the controller which activated the volume occurs, the preferred ownership controller can't get back automatically. The following cause the changing to the "one-sided" pathing. - Server failure, or Switch failure - Cable failure of the connection - Changing the pathing on the server side - Upgrading of the SANtricity OS and so on. The following meaning message is displayed. "Volume Not On Preferred Path." message is displayed on the Recovery Guru of the SANtricity System Manager. Execute "The relocation of the volume" from the SANtricity System Manager of ETERNUS AB/HB. As system operational, in case the "one-sided" pathing of the owned controller, the recovery operation doesn't need. - Regarding to PDL(Permanent Device Loss) VMware Multi-Pathing plug-in for ETERNUS AB/HB doesn't become PDL. If a hardware trouble occurred, VMware Multi-Pathing plug-in for ETERNUS AB/HB detect APD. Then, even you reduce LUN, the path state keep on active. If so, you may do Rescan Storage, and LUN will be decreased on device list.