***************************************************************************
** **
** Software Information for vSphere 7.0, vSphere 8.0 **
** **
** VMware Multi-Pathing plug-in for ETERNUS **
** 1.3.0 **
** **
** Dec 2024 FUJITSU LIMITED **
***************************************************************************
Preface
This guide explains installation/uninstallation procedures and notes of
VMware Multi-Pathing plug-in for ETERNUS.
Abbreviations
In this document, the disk storage system supported by this product is
referred to as "ETERNUS Disk storage system".
In this document, VMware's Knowledge Base is described as "KB". More
information on each KB can be found on the VMware website.
Reference
- Fujitsu ETERNUS Disk storage systems
https://www.fujitsu.com/global/products/computing/storage/disk/
Trademarks
Microsoft, Windows, Windows Server are either registered trademarks or
trademarks of Microsoft Corporation in the United States and other
countries. VMware is a registered trademark of VMware,Inc, in the United
States and other countries. All other company/product names mentioned
herein maybe trademarks or registered trademarks of their respective
holders and are used for identification purposes only.
High Safety Required
This Product is designed, developed and manufactured as contemplated for
general use, including without limitation, general office use, personal
use, household use, and ordinary industrial use, but is not designed,
developed and manufactured for use in situations with accompanying fatal
risks or dangers that, unless extremely high safety is secured, could
lead directly to death, personal injury, severe physical damage or other
loss (hereinafter "High Safety Required Use"), including without
limitation, nuclear reaction control in nuclear facility, aircraft flight
control, air traffic control, mass transport control, medical life
support system, and missile launch control in weapon systems. Do not use
this Product for High Safety Required Use without securing the sufficient
safety level required. If you wish to use this Product for High Safety
Required Use, please consult with our sales representative before such
use.
1. Introduction
1-1. VMware Multi-Pathing plug-in for ETERNUS
VMware Multi-Pathing plug-in for ETERNUS is offered as a sub plug-in
of Native Multipathing Plugin (NMP) of the VMware standard used when
the multipath is connected with the ETERNUS Disk storage system.
VMware Multi-Pathing plug-in for ETERNUS is Storage Array Type
Plug-in(SATP) that controls error handling corresponding to the
ETERNUS Disk storage system.
1-2. Audience
This document is intended for VMware administrators who are using
ETERNUS Disk storage system as an infrastructure of virtualization
deployments. It is assumed that the reader has a general knowledge
of VMware and ETERNUS Disk storage systems.
1-3. Release Information
1-3-1. Modification History
+----------------+--------------------------------------------------+
| version | Modification effect |
+----------------+--------------------------------------------------+
| 1.1.0 or latar | None |
+----------------+--------------------------------------------------+
1-3-2. Updates
+--------------+---------+------------------------------------------+
| date | version | changes |
+--------------+---------+------------------------------------------+
| Nov 30, 2012 | 1.0.0 | first version |
+--------------+---------+------------------------------------------+
| Mar 18, 2013 | 1.1.0 | - Error handling is improved. |
| | | - Added support OS |
| | | - VMware vSphere 5.0 Enterprise |
| | | - VMware vSphere 5.0 Enterprise Plus |
| | | - VMware vSphere 5.1 Enterprise |
| | | - VMware vSphere 5.1 Enterprise Plus |
+--------------+---------+------------------------------------------+
| Sep 19, 2013 | 1.1.0 | - This document was corrected. |
| | | - Added support OS |
| | | - VMware vSphere 5.5 Enterprise |
| | | - VMware vSphere 5.5 Enterprise Plus |
+--------------+---------+------------------------------------------+
| Jun 19, 2014 | 1.1.1 | - Error handling is improved. |
| | | (vSphere 5 Only) |
+--------------+---------+------------------------------------------+
| May 11, 2015 | 1.1.1 | - This document was corrected. |
| | | - Added support OS |
| | | - VMware vSphere 6.0 Enterprise |
| | | - VMware vSphere 6.0 Enterprise Plus |
+--------------+---------+------------------------------------------+
| Jul 1, 2015 | 1.1.2 | - VVOL of vSphere 6 is supported. |
| | | - Added support OS |
| | | - VMware vSphere 6.0 Standard |
+--------------+---------+------------------------------------------+
| Sep 30, 2016 | 1.1.3 | - Stop bundling the plug-in for |
| | | VMware vSphere 4.1 |
| | | - Error handling is improved. |
| | | - Script for jumpstart is improved. |
| | | (vSphere 6 Only) |
+--------------+---------+------------------------------------------+
| Jul 17, 2019 | 1.2.0 | - Added support OS |
| | | - VMware vSphere 6.5 Standard |
| | | - VMware vSphere 6.5 Enterprise Plus |
| | | - VMware vSphere 6.7 Standard |
| | | - VMware vSphere 6.7 Enterprise Plus |
+--------------+---------+------------------------------------------+
| Dec 20, 2019 | 1.2.1 | - Bug fix |
+--------------+---------+------------------------------------------+
| May 22, 2020 | 1.2.1 | - This document was corrected. |
| | | Abbreviations |
| | | 4. Installation and Uninstallation |
| | | procedure |
| | | 8. Appendix |
+--------------+---------+------------------------------------------+
| Sep 25, 2020 | 1.3.0 | - Added support OS |
| | | - VMware vSphere 7.0 Standard |
| | | - VMware vSphere 7.0 Enterprise Plus |
+--------------+---------+------------------------------------------+
| Aug 05, 2022 | 1.3.0 | - This document was corrected. |
| | | - Added 7. Notes Timeout on Guest OS |
| | | Ex. Setting for |
| | | Red Hat Enterprise Linux7/8 |
+--------------+---------+------------------------------------------+
| Aug 22, 2022 | 1.3.0 | - This document was corrected. |
| | | Added 7. Notes |
| | | - Timeout on Guest OS |
| | | Ex. Setting for |
| | | Red Hat Enterprise Linux7/8 |
| | | - Upgrading VMware ESXi |
+--------------+---------+------------------------------------------+
| Jul 10, 2023 | 1.3.0 | - Added support OS |
| | | - VMware vSphere 8.0 Standard |
| | | - VMware vSphere 8.0 Enterprise Plus |
+--------------+---------+------------------------------------------+
| Dec 26, 2024 | 1.3.0 | - This document was corrected. |
| | | - Added 7. Notes Regarding to PDL |
| | | (Permanent Device Loss) |
+--------------+---------+------------------------------------------+
2. Requirements
This section describes support environment and necessary resources.
Please be sure to check this section before installing VMware
Multi-Pathing plug-in for ETERNUS.
2-1. Supported Environment
Please visit the following product's web site for the latest information
(supported models and operating systems, important notes, etc.).
https://www.fujitsu.com/global/support/products/computing/storage/download/vmware-mp-plug-in.html
2-2. Required Disk Capacity for Installation
The following table shows the static disk space requirements for each
directory used to install VMware Multi-Pathing plug-in for ETERNUS.
+----------+----------------+----------------+----------------------+
|No |Directory |Disk Capacity |Note |
+----------+----------------+----------------+----------------------+
|1 |/usr |0.1MB | |
+----------+----------------+----------------+----------------------+
|2 |/etc |0.1MB | |
+----------+----------------+----------------+----------------------+
3. Product configuration
The product configuration of VMware Multi-Pathing plug-in for ETERNUS
is as follows.
VMwareVAMP_1.3.0
+-- FJT_satp_eternus_130.zip File used for installation
+-- readme.txt Software Information(JP)
+-- readme_en.txt Software Information(EN)
4. Installation and Uninstallation procedure
4-1. Installation
This section describes three methods to install VMware Multi-Pathing
plug-in for ETERNUS. You may select method based on your environment.
- Using ESXCLI
- Using esxcli command
- Using vSphere Lifecycle Manager
All commands must be run as root user in this section.
Stop GuestOS or move GuestOS to other hosts before it installs.
Specify the IP address of ESXi host at <ESX_ip-address> of the
following procedures.
The procedure to install the upgrade of VMware Multi-Pathing plug-in
for ETERNUS is the same as a new installation.
4-1-1. Installation VMware Multi-Pathing plug-in for ETERNUS
4-1-1-1. Installation using ESXCLI
This section describes how to install using ESXCLI.
1) Copy the installation file to a local directory on your ESXi host.
FJT_satp_eternus_130.zip is preserved in "/tmp" in the example
of executing the following commands.
2) Open a command prompt and change to the directory in which ESXCLI
is installed only Windows. For default, installed pass is as follows.
>cd C:\Program Files (x86)\VMware\esxcli
3) Run the following command to place the ESXi host into
maintenance mode.
>esxcli --server <ESX_ip-address> system maintenanceMode set --enable=1
4) Run the following command to install VMware Multi-Pathing plug-in
for ETERNUS into the ESXi host.
>esxcli --server <ESX_ip-address> software vib install -d /tmp/FJT_satp_eternus_130.zip
5) Run the following command to reboot the ESXi host.
>esxcli --server <ESX_ip-address> system shutdown reboot -d 10 -r satpinstall
6) Run the following command to confirm VMware Multi-Pathing plug-in
for ETERNUS installation.
>esxcli --server <ESX_ip-address> software vib list
example:
Name Version Vendor Acceptance Level Install Date
----------------- --------- ------ ---------------- ------------
fjt-satp-eternus 1.3.0-0 FJT VMwareAccepted 2023-06-10
7) Run the following command to confirm it is a setting that uses
FJT_SATP_ETERNUS.
>esxcli --server <ESX_ip-address> storage nmp satp rule list --satp FJT_SATP_ETERNUS
example:
Name Device Vendor Model Driver Transport Options Rule Group Claim Options Default PSP PSP Options Description
---------------- ------ ------- ------------ ------ --------- ------- ---------- ------------- ----------- ----------- -----------
FJT_SATP_ETERNUS FUJITSU ^ETERNUS_DX* user VMW_PSP_MRU
8) Run the following command to confirm it is a setting that uses
SATP. Please confirm FJT_SATP_ETERNUS is displayed in
"Storage Array Type:".
>esxcli --server <ESX_ip-address> storage nmp device list
example:
naa.600000e00d0000000001056700000000
Device Display Name: FUJITSU Fibre Channel Disk (naa.600000e00d0000000001056700000000)
Storage Array Type: FJT_SATP_ETERNUS
Storage Array Type Device Config: {FJT_SATP_ETERNUS device config options dtime=20 mtime=3 limit=6}
Path Selection Policy: VMW_PSP_MRU
Path Selection Policy Device Config: Current Path=vmhba2:C0:T0:L0
Path Selection Policy Device Custom Config:
Working Paths: vmhba2:C0:T0:L0
Is USB: false
9) Run the following command to exit maintenance mode.
>esxcli --server <ESX_ip-address> system maintenanceMode set --enable=0
4-1-1-2. Installation using esxcli command
This section describes how to install using esxcli command.
Perform the following steps on the ESXi Tech Support Mode.
1) Copy the installation file to a local directory on your ESXi host.
FJT_satp_eternus_130.zip is preserved in "/tmp" in the example
of executing the following commands.
2) Place the ESXi host into maintenance mode.
Use vSphere Client or Host Client.
3) Run the following command from the local directory where you
copied the FJT_satp_eternus_130.zip file to install VMware
Multi-Pathing plug-in for ETERNUS into the ESXi host.
# esxcli software vib install -d /tmp/FJT_satp_eternus_130.zip
4) Reboot the ESXi host.
Use vSphere Client or Host Client.
5) Run the following command to VMware Multi-Pathing plug-in for
ETERNUS installation is confirmed.
# esxcli software vib list
example:
Name Version Vendor Acceptance Level Install Date
----------------- --------- ------ ---------------- ------------
fjt-satp-eternus 1.3.0-0 FJT VMwareAccepted 2023-06-10
6) Run the following command to confirm it is a setting that uses
FJT_SATP_ETERNUS.
# esxcli storage nmp satp rule list --satp FJT_SATP_ETERNUS
example:
example:
Name Device Vendor Model Driver Transport Options Rule Group Claim Options Default PSP PSP Options Description
---------------- ------ ------- ------------ ------ --------- ------- ---------- ------------- ----------- ----------- -----------
FJT_SATP_ETERNUS FUJITSU ^ETERNUS_DX* user VMW_PSP_MRU
7) Run the following command to confirm it is a setting that uses
SATP. Please confirm FJT_SATP_ETERNUS is displayed in
"Storage Array Type:".
# esxcli storage nmp device list
example:
naa.600000e00d0000000001056700000000
Device Display Name: FUJITSU Fibre Channel Disk (naa.600000e00d0000000001056700000000)
Storage Array Type: FJT_SATP_ETERNUS
Storage Array Type Device Config: {FJT_SATP_ETERNUS device config options dtime=20 mtime=3 limit=6}
Path Selection Policy: VMW_PSP_MRU
Path Selection Policy Device Config: Current Path=vmhba2:C0:T0:L0
Path Selection Policy Device Custom Config:
Working Paths: vmhba2:C0:T0:L0
Is USB: false
8) ESXi host exit maintenance mode.
Use vSphere Client or Host Client.
4-1-1-3. Installation using vSphere Lifecycle Manager
Refer to the "VMware vSphere 8.0 Software Manual".
Reboot the ESXi host after installation.
4-1-2. Setting after installation
Please change "Path Selection" of each LUN to "RoundRobin" if necessary.
Notice:
Please refer to ETERNUS AF, ETERNUS DX Server Connection for a
detailed procedure.
4-2. Uninstallation
This section describes two methods to uninstall VMware Multi-Pathing
plug-in for ETERNUS. You can use one of two methods for your
environment.
- Using ESXCLI
- Using esxcli command
Stop GuestOS or move GuestOS to other hosts before it uninstalls.
Specify the IP address of ESXi host at <ESX_ip-address> of the
following procedures.
4-2-1. Uninstallation VMware Multi-Pathing plug-in for ETERNUS
4-2-1-1. Uninstallation using ESXCLI
This section describes how to uninstall VMware Multi-Pathing plug-in
for ETERNUS using ESXCLI. Perform the following steps on server in which
ESXCLI is installed.
1) Open a command prompt and change to the directory in which ESXCLI
is installed only Windows. For default, installed pass is as follows.
>cd C:\Program Files (x86)\VMware\esxcli
2) Run the following command to place the ESXi host into maintenance
mode.
>esxcli --server <ESX_ip-address> system maintenanceMode set --enable=1
3) Run the following command to uninstall.
>esxcli --server <ESX_ip-address> software vib remove -n fjt-satp-eternus
4) Run the following command to confirm the package is not displayed.
>esxcli --server <ESX_ip-address> software vib list
5) Run the following command to release the setting that uses
FJT_SATP_ETERNUS.
>esxcli --server <ESX_ip-address> storage nmp satp rule remove ^
--satp="FJT_SATP_ETERNUS" --vendor="FUJITSU" ^
--model="^ETERNUS_DX*" --psp="VMW_PSP_MRU"
6) Run the following command to reboot the ESXi host.
>esxcli --server <ESX_ip-address> system shutdown reboot -d 10 -r satpinstall
7) Run the following command to confirm the setting has been deleted.
>esxcli --server <ESX_ip-address> storage nmp satp rule list ^
--satp FJT_SATP_ETERNUS
8) Run the following command to confirm it is a return to standard
VMW_SATP_ALUA. Please confirm VMW_SATP_ALUA is displayed in
"Storage Array Type:".
>esxcli --server <ESX_ip-address> storage nmp device list
example:
naa.600000e00d0000000001056700000000
Device Display Name: FUJITSU Fibre Channel Disk
(naa.600000e00d0000000001056700000000)
Storage Array Type: VMW_SATP_ALUA
Storage Array Type Device Config:
{implicit_support=on;
explicit_support=off;
explicit_allow=on;
alua_followover=on;
{TPG_id=16,TPG_state=AO}
{TPG_id=17,TPG_state=ANO}}
Path Selection Policy: VMW_PSP_MRU
Path Selection Policy Device Config:
Current Path=vmhba2:C0:T0:L0
Path Selection Policy Device Custom Config:
Working Paths: vmhba2:C0:T0:L0
9) Run the following command to exit maintenance mode.
>esxcli --server <ESX_ip-address> system maintenanceMode set --enable=0
4-2-1-2. Uninstallation using esxcli command
This section describes how to uninstall VMware Multi-Pathing plug-in
for ETERNUS using esxcli command. Perform the following steps on
the ESXi host Tech Support Mode.
1) Place the ESXi host into maintenance mode.
Use vSphere Client or Host Client.
2) Run the following command to uninstall.
# esxcli software vib remove -n fjt-satp-eternus
3) Run the following command to confirm the package is not displayed.
# esxcli software vib list
4) Run the following command to release the setting that uses
FJT_SATP_ETERNUS.
# esxcli storage nmp satp rule remove --satp="FJT_SATP_ETERNUS" \
--vendor="FUJITSU" \
--model="^ETERNUS_DX*" \
--psp="VMW_PSP_MRU"
5) Reboot the ESXi host.
Use vSphere Client or Host Client.
6) Run the following command to confirm the setting has been deleted.
# esxcli storage nmp satp rule list --satp FJT_SATP_ETERNUS
7) Run the following command to confirm it is a return to standard
VMW_SATP_ALUA. Please confirm VMW_SATP_ALUA is displayed in
"Storage Array Type:".
# esxcli storage nmp device list
example:
naa.600000e00d0000000001056700000000
Device Display Name: FUJITSU Fibre Channel Disk
(naa.600000e00d0000000001056700000000)
Storage Array Type: VMW_SATP_ALUA
Storage Array Type Device Config:
{implicit_support=on;
explicit_support=off;
explicit_allow=on;
alua_followover=on;
{TPG_id=16,TPG_state=AO}
{TPG_id=17,TPG_state=ANO}}
Path Selection Policy: VMW_PSP_MRU
Path Selection Policy Device Config:
Current Path=vmhba2:C0:T0:L0
Path Selection Policy Device Custom Config:
Working Paths: vmhba2:C0:T0:L0
8) ESXi host exit maintenance mode.
Use vSphere Client or Host Client.
5. About Path Recovery
If the diagnostic continues to respond normally for more than 20 minutes,
return the path state from dead to online.
Also, if the online to dead transition is repeated 6 times within 3 hours
from the first dead transition, it is judged to be an unstable path and
the status changes to "fataldead".
"fataldead" is not automatically recovered by diagnostics.
5-1 How to recover from fataldead
After correcting the error, restore the path as follows.
1) Check for fataldead in the following ways:.
# esxcli storage nmp path list
or
# esxcli storage nmp path list -d=naa.XXXXXXXXXXXXXXX
example:
# esxcli storage nmp path list -d=naa.600000e00d11000000112eb900820000
fc.20000090fa07df16:10000090fa07df16-fc.500000e0d46eb900:500000e0d46eb980-naa.600000e00d11000000112eb900820000
Runtime Name: vmhba2:C0:T0:L0
Device: naa.600000e00d11000000112eb900820000
Device Display Name: FUJITSU Fibre Channel Disk (naa.600000e00d11000000112eb900820000)
Group State: dead
Array Priority: 0
Storage Array Type Path Config: {FJT_SATP_ETERNUS path vmhba2:C0:T0:L0 config info: vendor FUJITSU model ETERNUS_DXL FATALDEAD}
Path Selection Policy Path Config: {non-current path; rank: 0}
A path in the fataldead state will show FATALDEAD in "Storage Array Type Path Config".
2) Release fataldead with the following command:
# esxcli storage nmp satp generic pathconfig set -p "Runtime Name" --config="fataldead=off"
-p must be the "Runtime Name" of 1).
example:
# esxcli storage nmp satp generic pathconfig set -p vmhba3:C0:T2:L0 --config="fataldead=off"
3) Verify that the fataldead path is cleared with the following command:
# esxcli storage nmp path list -p "Runtime Name"
-p must be the "Runtime Name" of 1).
example:
# esxcli storage nmp path list -p vmhba3:C0:T2:L0
fc.20000090fa07df16:10000090fa07df16-fc.500000e0d46eb900:500000e0d46eb980-naa.600000e00d11000000112eb900820000
Runtime Name: vmhba2:C0:T0:L0
Device: naa.600000e00d11000000112eb900820000
Device Display Name: FUJITSU Fibre Channel Disk (naa.600000e00d11000000112eb900820000)
Group State: dead
Array Priority: 0
Storage Array Type Path Config: {FJT_SATP_ETERNUS path vmhba2:C0:T0:L0 config info: vendor FUJITSU model ETERNUS_DXL}
Path Selection Policy Path Config: {non-current path; rank: 0}
Notes:
* The Group State immediately following the command is dead.
Transition to active at next diagnostic.
* Only the Runtime Name specified by -p will be fataldead.
Do 1) to 3) for each.
6. Message
Output to /var/log/vmkernel.log.
Message:
FJT_SATP_ETERNUS: vmhbaX:CX:TX:LX XX->YY
Meaning:
Received XX host error and converted to YY.
Action:
If a path transition occurs, investigate the cause of the XX error.
Message:
FJT_SATP_ETERNUS: vmhbaX:CX:TX:LX is fataldead
Meaning:
Transition to fataldead successful.
Action:
Investigate the cause of the error.
After removing the cause, release fataldead as described in
"5-1 How to recover from fataldead".
7. Notes
- About Firmware Update for "Online Update"
Can be run in "Automatic mode".
1) Run the 5 -1 2) Command on all LUNs and paths.
or the following command:
# for i in `esxcfg-mpath -L | grep naa.XXXXXXXXXXX | awk '{print $1}'`; do esxcli storage nmp satp generic pathconfig set -p $i --config="fataldead=off" ; done
naa.XXXXXXXXXX should specify the common part of the farm replacement unit.
2) Replaced active firmware in "Automatic mode"
If 1) cannot be performed, perform a Firmware Update using "Operator Intervention mode".
The VMware documentation provides information on how to disable and enable paths.
Reference:
https://knowledge.broadcom.com/external/article?legacyId=2000552
- Timeout og Guest OS
If activating virtual disk of RDM on GuestOS of Linux, set the timeout period to at least 40 seconds.
Ex. Red Hat Enterprise Linux7)
Create "/etc/udev/rules.d/99-rdm-scsi-udev.rules" file.
Add the following three lines.
ACTION=="add", SUBSYSTEMS=="scsi", ATTRS{vendor}=="FUJITSU ", ATTRS{model}=="ETERNUS_DXL ", RUN+="/bin/sh -c 'echo 40 >/sys$DEVPATH/timeout'"
ACTION=="add", SUBSYSTEMS=="scsi", ATTRS{vendor}=="FUJITSU ", ATTRS{model}=="ETERNUS_DXM ", RUN+="/bin/sh -c 'echo 40 >/sys$DEVPATH/timeout'"
ACTION=="add", SUBSYSTEMS=="scsi", ATTRS{vendor}=="FUJITSU ", ATTRS{model}=="ETERNUS_DXH ", RUN+="/bin/sh -c 'echo 40 >/sys$DEVPATH/timeout'"
Ex. Red Hat Enterprise Linux8, and Red Hat Enterprise Linux9)
Create "/etc/udev/rules.d/99-rdm-scsi-udev.rules" file.
Add the following three lines.
ACTION=="add", SUBSYSTEMS=="scsi", ATTRS{vendor}=="FUJITSU ", ATTRS{model}=="ETERNUS_DXL ", ENV{DEVTYPE}=="disk", RUN+="/bin/sh -c 'echo 40 >/sys$DEVPATH/device/timeout'"
ACTION=="add", SUBSYSTEMS=="scsi", ATTRS{vendor}=="FUJITSU ", ATTRS{model}=="ETERNUS_DXM ", ENV{DEVTYPE}=="disk", RUN+="/bin/sh -c 'echo 40 >/sys$DEVPATH/device/timeout'"
ACTION=="add", SUBSYSTEMS=="scsi", ATTRS{vendor}=="FUJITSU ", ATTRS{model}=="ETERNUS_DXH ", ENV{DEVTYPE}=="disk", RUN+="/bin/sh -c 'echo 40 >/sys$DEVPATH/device/timeout'"
If you are not using an RDM to show the LUN to the guest,
refer to the KB below to set the timeout value.
https://knowledge.broadcom.com/external/article?legacyId=1009465
If you do not change the timeout value, the guest may detect
the timeout before the host detects the timeout,
causing an I/O error or LUN unavailability.
- Regarding to upgrading VMware ESXi
If the using version of VMware Multi-Pathing plug-in for ETERNUS doesn't adapt
VMware ESXi's version, please do the following.
1. First, Un-install VMware Multi-Pathing plug-in for ETERNUS.
2. Upgrade VMware ESXi.
3. After that, install the compatible version's VMware Multi-Pathing plug-in
for ETERNUS, and setup it.(Please refer the install/uninstall sequences.)
- Regarding to PDL(Permanent Device Loss)
VMware Multi-Pathing plug-in for ETERNUS doesn't become PDL.
If a hardware trouble occurred, VMware Multi-Pathing plug-in for ETERNUS detect APD.
Then, even you reduce LUN, the path state keep on active.
If so, you may do Rescan Storage, and LUN will be decreased on device list.