I haven’t posted in a while; it’s been pretty nuts, which is a good thing if you’re in the business of selling computer hardware/software, but it puts a crimp on my free time to post blog entries.
Lately I have been asked a lot about a topic in storage management that I thought most companies have a solid handle on already. But I’m being asked more and more what my recommendations are around path management. I suspect that this has something to do with the path management changes in VMware vSphere causing people to readdress this topic for VMware, and ask if they should look at it on a wider level in the data center.
In the following blog entry I describe the current state of path management and outline some recommendations for the use/implementation of path management software in the data center.
The history of path management is very wrapped up with the history of the storage array vendors. Prior to the advent of PowerPath from EMC, path management was very much a part of the operating system. OS’s such as IBM’s VM, DEC’s VMS, and other mainframe and mini-computer operating systems all had the ability to communicate with their storage via more than one path and to load balance the I/O across those paths. However, when EMC and other storage vendors started to move into the open systems world, where the OS’s had a “small system” background, they found that path management was not something offered by the operating system. Early version of MS Windows, and various flavors of UNIX all had no way to address storage across more than one path, or if they did, it was simply a failover path without the ability to load-balance the I/Os. In response to this situation EMC developed PowerPath, and other storage vendors such as Hitachi soon followed suit. At that time, the path management software developed by the storage vendors only supported that storage vendor’s particular arrays. In other words EMC’s PowerPath only supported EMC arrays, Hitachi’s HDLM software only supported Hitachi arrays, and so forth. While this allowed a customer to optimize the connections between their hosts and their storage, it also had the effect of locking the customer into a single vendors arrays simply because it became very difficult to support more than one vendor’s array on the same host, and switching vendors also became more difficult by adding a significant level of effort to the migration process since all of the path management software all had to be replaced as well.
This situation remained the same for a number of years, until EMC announced support for non-EMC vendors in their PowerPath product. This announcement was a part of EMC’s plans at the time to move from a pure hardware company to a more software driven business. Along with the announcement of PowerPath for other storage vendors, EMC also announced a set of APIs that would allow management of non-EMC arrays from their flagship Control Center product and several other smaller changes to help position EMC as a software company.
Unfortunately, these changes were never fully recognized in the EMC software, nor were the EMC Sales teams particularly enthusiastic about the move away from a focus on “big iron” hardware that they had made so much money with for such a long time. This left some of the EMC products, such as PowerPath, in a position where they had some support for other vendors arrays, but that support was not complete from either an array specific feature perspective, or in the case of PowerPath, from a array vendor perspective. For example, aside from support for their own arrays, EMC PowerPath provides support for HDS (and some of the OEMs of HDS products such as HP) and some IBM arrays as well. However, support for other significant array vendors in the market such as 3PAR, Compellant, NetApp, etc. is notably missing. As a matter of fact, EMC has not added support for any array vendor since their original announcement in 2003 of support for HDS, IBM, and HP.
Things began to change when Microsoft introduced MPIO as part of the Windows operating system starting with the Windows 2000/2003 versions of the software. Microsoft, having learned from those that went before them, decided to provide a standardized mechanism for path management, but at the same time they also allow the storage array vendors to provide a plug-in (call a DSM) if they wanted to add additional capabilities for path management to MPIO. SUN developed it’s version of MPIO, called MPxIO in 2003, IBM introduced it’s version of MPIO in 2002, Linux announced Device Mapper in 2005, and the last vendor to announce MPIO software as part of the operating system was HP which announced their version of MPIO in 2007. Note that HP had a non-OS integrated version of path management called PVLinks available since the early 1990s. Therefore, by 2005 virtually every operating system in use in the data center had path management built in and the need for array vendor supported software simply no longer existed in order to provide path management.
At the same time as all of the above was happening, a company called VERITAS as part of their VERIAS Volume Manager was developing one true independent piece of path management software. VERITAS was positioning itself as an OS and array independent storage management company, and therefore it developed it’s own suite of tools for volume management, a file system, and path management software called DMP (Dynamic Multi Pathing). DMP has had its issues over the years, but was particularly popular with SUN customers, if for no other reason than it came with Foundation Suite which was very popular with SUN Solaris customers.
VMware Path Management
All of the above addresses path management purely from a host and storage array perspective. However, with the introduction of VMware another player entered the path management landscape. From the beginning, VMware required users to utilize the path management tat was built into VMware. The path management software built into VMware 3.5 and older provided basic path management feature, specifically path failover, but did not provide load balancing or any array specific features. This provided users of VMware some ability to tolerate path outages, but defiantly limited VMware’s ability to provide for high I/O applications. This limit wasn’t the only reason that early versions of VMware didn’t support high I/O applications, but it certainly needed to be addressed should VMware ever want to be able to support these high I/O applications. With the introduction of vSphere, VMware has finally addressed this issue, and more. Much like with MPIO VMware has now introduced a mechanism for storage array vendors to provide a “plug-in” into the path management functions built into vSphere called a storage array type plug-in (SATP) for the new Native Multipathing Plugins (NMP) module. One of the first vendors to take advantage of this capability was EMC with their PowerPath/VE product which provides support for EMC arrays.
Recommendations for Current Approach
There currently is no panacea when it comes to picking an approach to path management in the data center. Once again it boils down to two philosophies. Do you want to try to utilize a single path management software, or do you want to utilize the path management software that is built into the operating system?
Option #1 – Utilize a single path management software
For this option, a single piece of software is chosen and then loaded on every host to provide path management. There are really only two options for this software. You can utilize Symantec (once VERITAS) DMP, or you can use EMC PowerPath. These are the only two products in wide distribution that provide multi-vendor array support and a wide variety of host support as well.
The main advantage to utilizing a single piece of software is management of the path management software. You have a single product that is well understood provided by a single vendor to manage. Theoretically this should reduce your management costs, and provide for a more reliable and stable environment.
The down side to this approach is the cost involved. Since the software is a purchased product, a license for every host in the datacenter must be purchased from the vendor, or some kind of enterprise license obtained. In either case, tracking of the software licenses, and general management of the software often offsets the cost benefits of having a single vendor for path management. The second major disadvantage of utilizing a single path management software product is that you are locked into the list of supported hosts and arrays that the vendor chooses to provide. In the case of PowerPath the list of non-EMC arrays is very short, and appears to show no signs of ever growing any larger. In the case of DMP, there is also a question of where that product is heading, and how much development in the form of additional array support Symantec intends to provide. This creates a situation where the path management policy can prevent the organization from purchasing a new array that might provide significant cost benefit advantages. Finally, the management of versions of the path management software, the switch firmware, the operating system, and the array firmware create a complex matrix and increase the cost of support significantly. It can also slow down the rollout of newer models of arrays, switches, and host operating systems delaying cost saving.
Option #2 – Utilize the path management software built into the operating system
For this option, the built in path management software supplied with the operating system is utilized, along with the DSM (where available) to provide path management for all of the arrays in the datacenter.
With this approach dependence on third party software to provide path management is eliminated, and the support matrix for the host OS, switch and array is reduced making support much more straightforward. With the addition of a DSM to provide array specific features, all of the capabilities of the array are made available, without the need for third part software or vendor lock in. The operating system vendor, with whom a close support relationship already exists, provides support for the path management software reducing the possibility of three way finger pointing in the case of a problem. Finally, the ability to support any array which you feel provides you with a business advantage in terms of saving costs, providing additional features, and/or additional speed is once again on the table rather than the short list of arrays supported by the third party path management software vendors.
Since each operating system provides it’s own path management software some additional learning and understanding of those different path management software products would need to be maintained by the appropriate support team (storage or OS). This could add some additional costs in terms of training and support.
At this point in time, it is my recommendation that path management software integrated with the operating system be utilized rather than third party software. I believe that the flexibility to utilize nearly any array on the market today to address any storage issues you might face without being locked into a particular vendor, or even a short list of vendors, far outweighs any minor added overhead involved with learning multiple path management software interfaces.