When Bad Things Happen to Good Technology: RIP, Drive Extender

In this age of ten-second attention spans, bloggers are often guilty of over-dramatizing non-events to make them seem more important.

This is not such a time.

Microsoft today announced that it is removing its Drive Extender technology from Windows Home Server "Vail" (WHS v2) and Small Business Server 2011 Essentials (formerly "Aurora"). I mentioned this in today's daily update, and discussed it on Windows Weekly today, but based on the number of emails I've gotten, few people were aware that I had already discussed this topic elsewhere.

That said, I have some additional information that Microsoft didn't discuss in either of their blog posts (here and here) today announcing this decision.

First, let's review what Drive Extender was, and what it was going to become.

Drive Extender debuted in the first version of Windows Home Server in 2007. It supplies two key benefits: Data redundancy and a single, expandable pool of storage that doesn't require drive letters and is easily expandable.

The data redundancy feature is easy enough to explain: Through a simple UI in the WHS console, you can ensure that all of the files in a given share were duplicated across two physical hard disks. This ensures that, should a drive fail, your only copy of an important photo, document, or other file doesn't go with it.

The second one is like RAID for Dummies: It lets you add hard drives to your server and have their storage added to the WHS storage pool. You don't have to deal with drive letters, and because content is essentially stored in shares, the available storage available to those shares is limitless (or at least only limited to your server's ability to add more disks).

Drive Extender is awesome, and as the underlying technology responsible for two of WHS' best features, Microsoft was planning to expand its use beyond WHS, first to Small Business Server (in "Aurora") and then later to other versions of Windows Server and the Windows client.

So here's where we get to the bit that Microsoft didn't communicate for some reason: Why Drive Extender is being killed.

In a briefing last month, I was told that Microsoft and its partners discovered problems with Drive Extender once they began typical server loads (i.e. server applications) on the system. This came about because Drive Extender was being moved from a simple system, WHS, to a more complex, server-like OS )(SBS "Aurora") that would in fact be used to run true server applications. And these applications were causing problems.

"Drive Extender was a neat feature, but the implementation was off, and we discovered some application compatibility and disk tool problems related to its ability to correct data errors on the fly," Microsoft general manager Kevin Kean told me. "We don't want to give customers problems; we want to give them solutions. So ultimately, we decided that we needed to cut out Drive Extender. Removing Drive Extender will make file shares easy, and it's possible to accomplish most of its features otherwise. For example, you use the server's centralized backup or even RAID as an alternative to data duplication."

Microsoft had tried to fix these problems, of course, and anyone who's familiar with WHS and tested Vail or Aurora will know that these products introduced a weird multiple-drive-letter scheme, tied to the shares, that was part of Microsoft's attempted fix. In fact, I found this "feature" so confusing, I went back to Microsoft with three separate sets of questions about it at the time.

So the question remains, of course, now what?

For SBS, the answer isn't all that painful: There is already a well established ecosystem of SBS partners that support Microsoft technologies, and the expectation is that hardware makers will bundle their own data duplication technologies with their products.

For WHS, the answer isn't great. WHS is a niche product--beloved, yes, but by a very tiny number of people--and is less well served by partners. Too, data products are expensive, and one of the best things about WHS wasn't that it had Drive Extender per se, but that it had this huge collection of useful features, some of which were DE-related.

Microsoft says it will deliver a new Vail beta in January, and at that time we can see what a DE-less WHS version looks like and start thinking about what we're going to do going forward. My home "infrastructure" is currently on the Vail beta refresh/release candidate/whatever that was, and it's working well. But clearly, that's a temporary situation, and I'll have to figure that out. Whether that something is still Vail, or some combo of Vail and Drobo, or some future NAS, or whatever ... I don't know. I may have known about this longer than some, but I'm as confused as anyone.

Discuss this Article 15

kcarson97404
on Nov 23, 2010
This is very disappointing. Drive Extender was one of the best features of WHS. In fact, just this weekend I needed to add storage to my WHS. It was a simple matter of plugging in a new external USB hard drive, answering a quick question in the WHS console, and "ta-da!" it I had an extra 500GB of storage. I hope the removal of DE doesn't mean having to reconfigure RAID servers and other complicated processes. If so, I'm not sure why I should upgrade to the new version of WHS.
dtatgenho
on Nov 23, 2010
This is beyond disappointing. It makes no sense to take a consumer product, try to extend it to meet SMB needs it wasn't designed for, and when it doesn't work, you cripple everything and make all of the products suffer. As one commenter on the WHS Team blog said: "If you have to fork the code, fork the code." Pardon the poor analogy, but if you have a greyhound that wins every race at the dog track, and then you make him run in the Kentucky Derby and he (understandably) loses to the horses, wouldn't you rather just let him continue to run and win at the dog track? Microsoft is doing the equivalent of taking him out back and shooting him because he can't do something he was never supposed to do.
Waethorn
on Nov 23, 2010
One of the big problems with DE occurs when you have large data file sets, and I've noticed this with WHS in a couple of test scenarios. Say you have a client that has unique data in large sets. I have one such client. They have data sets for their own customers that range in the gigabyte folder sizes from over a decade of work, and they have hundreds of folders like that, with no repeat files (remember, this is just data, not PC backups or anything). They have a server with a 2TB hard drive size, and it's roughly 80% full. Now, if they were running WHS, what would happen is it would duplicate the data across each 2TB drive that I throw at it, and because new data is going to be no less unique than the existing data, they won't gain any additional space in their storage pool. Single-instance storage only saves space if there is redundant data, but in their case, no two files are the same. DE would continue to replicate data across every single 2TB drive in the system. Ok, so I could put a 3TB drive in, you say. Well, that's not exactly a good option either, and here's why: all of the data on the 2TB drive(s) would be replicated on the 3TB drive, but that extra 1TB of space will have absolutely no redundancy in the system at all in this scenario. If the remaining 20% has no duplicate files, the 2TB drives would just fill, and I would have 1TB of non-redundant storage space. I would be better off putting 2x2TB hard drives in and mirroring them, and an additional 2x2TB mirrored array would still double the capacity of the initial server storage. OR, I could put 2x2TB in and use a 4TB NAS for backup for the same level of redundancy (although at the expense of uptime during recovery). DE doesn't have support for multiple storage pools in Aurora from what I can tell, and neither does WHS (correct me if I'm wrong). Hence, DE offers no benefit over RAID and conventional backup. If Windows gets to a point where it doesn't support drive letters, this may work. For DE to work properly, I really think that NTFS compression needs to be run independently on each drive, separate from DE duplication. Processing power for that would likely have fairly hefty requirements though.
USArcher
on Nov 23, 2010
If OEMs can offer consumers affordable 3-4 TB systems in 2011 then I don't see this being a big deal. However, what has the Business & Tools division done with Vail to enhance consumer appeal. They say they will have some new stuff to reveal/demo at CES..we'll see. Right now, I'm curious if they will now be going back to shares instead of drive letters.
GoodThings2Life
on Nov 23, 2010
This is nonsense! Microsoft should leave Drive Extender in the OS and "rework it" so that it warns against using it when used for "applications" or other "potentially risky" scenarios. They have plenty of features that prompt you and either allow you to continue anyway or prevent you from using them.
rr0de74@live.com
on Nov 23, 2010
Seriously WTeF are they thinking????? I love WHS, recommend it to so many. Being able to add drives, and click a few options to duplicate data was HUGE! Well buh bye WHS, hello.... http://www.synology.com/us/index.php Micrsoft, poor Microsoft. Right when things start to get better for you, Windows 7, new Xbox, Windows Phone 7, you go and mess them up?????? AT LEAST allow us to duplicate with a single button, provided we have installed two drives of the same size?????
Waethorn
on Nov 23, 2010
"AT LEAST allow us to duplicate with a single button, provided we have installed two drives of the same size?????" Windows Server operating systems support software RAID 1, or did you forget that already? It's also extremely easy to set up. Business client OS's also support software RAID, but they don't allow you to do it on the system drive, unlike the servers. Also, people that were beta testing Vail and Aurora have to take responsibility for this decision by Microsoft, because it was them that wanted to have all kinds of unsupported applications and server roles running on a box that was only meant for storage. The Connect forums are full of users wanting to run SQL right on the box, or WDS, or WSUS, or some LOB app. Newsflash: database apps don't run well on DE, and after the data loss issue of WHS 2008, there are no application vendors that will support storing data on it, hence the decision. Even Microsoft says not to use it for SQL, and that's why it was never added to enterprise server products. And yet there are literally dozens of questions on Connect asking how to get it to work. Putting Db data on DE is worse than putting it on a software RAID 5 running on an Atom.
rr0de74@live.com
on Nov 23, 2010
@Waethorn its my experience that in your example, WHS would only duplicate that Data to one other drive, not 3 or 4 or every drive. I thought when you checked the box on a share to duplicate, say your music share, that it would make sure that the contents of that share was on more than one drive, as in two. So if I put in 4-2TB drives and I had 100gig of music in the music share it would only be on one drive....until I turned on duplication. At that point it would make sure that that 100 gigs of music was on 2 drives and not all 4. If one of those drives failed it would then copy it to another drive maintaining the 2 copies.
rr0de74@live.com
on Nov 23, 2010
@USarcher the big deal is what if OEM gives you a single 3TB drive, and it dies while in use? With out DE you are going to have to RAID 1 multiple drives. You are going to have to manage that RAID. Those drives will have to be the same size or the second drive has to be the same or bigger. With DE there was no management. You could have a 1TB drive and 5 250gig drives all in one pool.
cjnovi
on Nov 24, 2010
Paul, I don't see anything bad about drive letters in current VAIL beta and don't see why you would ask Microsoft all those questions. In fact I liked them, cause you could easily setup Mesh sync which would not work with shares, but work perfectly if drive appear as regular local one. And normal user will never see them I guess. I see SMB concerns about having drive extender, buseneses usually can plan what they need ahead and can spend some more in return of RAID solidness. But for home I'd rather have Drive Extender, #1 feature for me in WHS. RAID solutions can never be as flexible and seamless as Drive Extender in terms of adding and replacing drives. All other features in VAIL wasn't impressive at all comparing to WHSv1. If there's no WHSv1 I'd rather add some more drives into one of my existing Windows 7 boxes instead of having another box which hardly add significant value. If I want to spend some more money, range is rather wide with all drobo-s etc. We'll see what they come up with in next beta, but it seems that VAIL no longer will stand out from bunch of SOHO NAS devices with unique feature.
bluvg
on Nov 24, 2010
I think the real solution is to update NTFS or replace it with a more modern file system. Not that NTFS is bad--it's quite good, really--but with file systems like ZFS out there, the bar has been raised. ZFS would be a very nice option for Home Server, or for that matter, all their Windows Servers. Microsoft hasn't talked about this much (they must be thinking about it quite a bit internally, right???), but the various limits of NTFS are becoming an issue more and more frequently. They have a great opportunity here--and perhaps a big selling point for Windows 8 (again, not that NTFS is bad!).
pokeystuff
on Nov 24, 2010
Check out.. https://connect.microsoft.com/WindowsHomeServer/feedback/details/624029/... It is the place to send feedback to MS... Up to 2004 votes and 531 comments... people are passionate about this. I think Home Server will flop now.
ByteFlipper
on Nov 24, 2010
@Waethorn That's not how Drive Extender works at all. It doesn't copy all of your data to all of your drives, only to 2 drives. So please stop the FUD. Second, RAID is a horrible solution for HOME users. Windows *Home* Server was designed for *home* users, where people often have multiple older drives lying around, or where people can only afford to upgrade one drive at a time. In those scenarios, having the ability to add any drive of any size and type was dead-simple. With DE, you can replace your smallest drive with a new, larger drive. Repeat as required. Easy and cheap upgrade path. Try that with RAID. And what is this talk of databases? This is a HOME product... The sad part is, now MS is trying to make Windows *Home* Server work in SMB for which it was NOT designed, and ripping out parts that is important and that works well for home users just to make it work in SMB. I fail to see why they can't leave DE in there - I mean, you add a new drive, it asks you whether you want to add it to DE, just answer freakin "NO"! Then use it in a RAID configuration if that is what you really want. What is so difficult? Unbelievable. I hope "Vail" is a big fat FAIL. I also hope I can keep my WHS v1 running for many many years...
ByteFlipper
on Nov 24, 2010
@Waethorn You say people are asking for support of databases on DE. I think the solution is to tell those people to add new drives, but say NO when asked whether to add the drive to DE. Then use those drives however you want (RAID them, if they want). If it is never added to DE, it won't interfere with it (and if it does, fix THAT instead). The solution is NOT to remove DE because some people asked to put databases on DE. That is a braindead solution that screws up everyone else that DO find DE to be a valuable solution. I guess I can understand them abandoning WHS, as it probably wasn't making them much money. All I ask though is to not lie about the reasons for doing it. I doubt any of the home users partook in some imaginary feedback that told them DE wasn't working and that it should be removed. I wish MS would just grow a spine and tell home users that they are just too unimportant to really matter either way.
Waethorn
on Nov 24, 2010
@rrode: that is not correct. DE duplicates data across all disks when a new disk is inserted. It does not go through a resync when a disc is pulled, but it still maintains redundancy. DE also includes ECC data to rebuild bad data on a single drive too.

Please or Register to post comments.

IT/Dev Connections

Las Vegas
September 30th - October 4th

Paul ThurottYou'll have the opportunity to experience:
• 120 Technical
Sessions
• Networking with Peers
• Expert Speakers


Come See Paul Thurrott & Mary Jo Foley in Person!

Register Now

Office 365 InfoCenter

Get the latest insight and info from Paul

Read Now!

What I Use