In the last episodes we failed an entire datacenter with and without a witness. This time we will be failing a PowerStore array only, and leave all ESXi hosts running. This should trigger an APD (All Paths Down) scenario in the impacted datacenter, and we should see VMware High Availability (HA) kick in and recover from this scenario.
As always you can also choose to skip to the video directly.
Failing a storage array without failing hosts
This particular failure scenario is more of a VMware thing; we fail a storage array (with the use of a metro witness), and as a result the surviving array should continue to serve IO from both preferred and non-preferred volumes in the unimpacted datacenter.
The real trick here comes from the stretched cluster that hoovers overhead: VMware HA should discover that some of the workloads have gone into and All Paths Down (APD) state as the storage array has been powered off. As we have been using Dell PowerStores in a metro config (being a non-uniform access method) without “cross zoning” (so two “clean” locally FC zoned arrays using IP networking to mirror across) all paths will disappear in one datacenter.
In this demo I have setup VMware HA to look out for APD, and power-off and then restart any impacted VMs. A restart should be possible as the Dell PowerStores in metro underneath have witness support, so the surviving array should be allowed to continue to serve all volumes including non-preferred ones. Check out the video below!
One thought on “Blog Series: Metro – Break Stuff 5 – Failing a PowerStore”