backtop


Print 63 comment(s) - last by bgm063.. on Mar 14 at 1:12 AM


The Windows Home Server continues to eat up files, but it appears that this product's very hungry bug isn't going to turn into a butterfly and fly away anytime soon.  (Source: Microsoft)
The very hungry Windows Home Server continues to whet its appetite on unfortunate users files as the scope of the problem grows and grows

Like the children's book, The Very Hungry Caterpillar, the scope of the Windows Home Server bug simply grew and grew as it ate its way through users' files.  Although Microsoft promised that a solution would be made available to resolve the current issues, a full fix will not be available until at least June 2008.

What was once an attractive home service solution with a wealth of hardware partners has, in essence, become an unsightly pest for many.  Until a solution is found, use of WHS brings with it serious risks of data corruption -- something many consider to be a cardinal sin of networking hardware or software.

One DailyTech reader, Tim Slocum from Roscoe, Illinois, contacted us a couple weeks back with a rather incredible personal story of data loss, which he hoped would serve as a warning to others. 
Slocum was an eager WHS user and states that he copied 16,000+ family pictures and videos to the system.  Around Christmas he discovered that many of these files had become corrupted.  He rebuilt and reformatted the system, only to experience unpleasantly surprising results.

Slocum states in an email to DailyTech, "I then reformatted and rebuilt the system with NO ADD-INS or extra software. Copied all photos to the server, setup PC backups, and let the system sit with no usage because of the lack of trust. This weekend I again noticed photos are now corrupted."

Slocum acknowledges that a family member who works for MS as a consultant has had no issues that he knows of, though he planned on emailing him to verify this.  Slocum adds  that while not a "real techie" he is fairly knowledgeable.  He states, "I have been a developer for over 20 years ... last 2-3 have been moving into VB.NET.  So I have some knowledge of testing and have built PC's in the past."

Having worked hard to stabilize his system,
Slocum plans to continue his efforts with a third build, turning off file duplication, which reportedly may affect the likelihood of occurrence.  Tim feels that WHS is a promising product, but Microsoft is failing to take its issues seriously enough. 

The really surprising part of
Slocum's story at the time DailyTech received it was that he did not edit the files.  While some users had alleged corruption on transfers in unverified reports floating around the internet, previously, Microsoft stated that corruption only occurred when editing files.

Now Microsoft says the problem is that the underpinnings of WHS are broken, and that a fix is required at a very low level.  This will take a good deal of time to develop and validate, according to the WHS Team at Microsoft.  The WHS Team hopes to release beta versions of a patch over following months, but states that June is the soonest a finished patch might appear.

The WHS Team also warns that some users are mistaking other problems for the issue.  Says the Team, "Some of the instances that were initially attributed to this issue ended up being something else, such as a faulty network card/driver, old routers with outdated firmware, or people incorrectly testing the limits of their home servers."

However the Team did not rule that the WHS may have other problems causing trouble on a low level, though they state that they feel very confident that they understand the underlying issue that’s currently causing the main known problem.

And it turns out that
Slocum was correct -- the knowledgebase article has just been updated to encompass file transfers.  The new knowledge base article also has additional information on the cause -- how the NFTS file system, the cache, and the memory manager can get out of whack and beginning eating up user data.  The article explains it thusly:

Windows Home Server uses a file system mini-filter driver in addition to the NTFS file system to implement Shared Folders storage technology. File system mini-filter drivers are an extensibility mechanism that is provided by Windows to enable storage scenarios. For distributing data across the different hard drives that are managed by Windows Home Server, the Windows Home Server mini-filter driver redirects I/O between files that are stored on the main hard drive and files that are stored on other hard drives. This redirection mechanism is enabled only when Windows Home Server is managing the Shared Folder storage of multiple physical hard drives. A bug has been discovered in the redirection mechanism which, in certain cases, depending on application use patterns, timing, and workload, may cause interactions between NTFS, the Memory Manager, and the Cache Manager to get out of sync.

A link to a full technical page on the situation can be found here.

While the Windows Home Server Team is working hard to have a fix ready by summer time, in the meantime WHS users are left with the unpleasant reality that editing or storing files on the server may lead to corruption.  And with the scope of the flaws in WHS's low level file handling growing weekly, like a certain hungry caterpillar; it leaves one to wonder whether there are more aspects of the problem yet to be discovered.



Comments     Threshold


This article is over a month old, voting and posting comments is disabled

QC Anybody
By scoprio487 on 3/12/2008 12:04:24 AM , Rating: 3
How in the heck did a "bug" this big get past QC? This is supposed to be a sever os, doesn't software used in a sever have to go through more rigorous testing and validation? Sombody needs to really wake up and smell the coffee.




RE: QC Anybody
By noirsoft on 3/12/2008 12:16:59 AM , Rating: 3
Because the bug is extremely rare and happens almost never to almost nobody? It sounds like you have to get to a really special-case scenario in order to enact the bug. From the reports, MS had not even been able to replicate the bug. Perhaps they have now, and the extended report.

I use my WHS all the time (with replication across 2 drives) and have yet to see the problem. That is as valid as the one reported failure above.


RE: QC Anybody
By hopsandmalt on 3/12/2008 2:06:04 AM , Rating: 3
This is what I thought as well. I run WHS with 7 computers throughout my house. I never had any problems until one day when I logged in to do bills and found that my Quicken files were corrupted.

Now I do have about 70 gig of music on my WHS and with teens in the house it is streamed constantly. We have never had any problems with that at all.

So... It appears that it doesnt like Quicken. Thats for sure.

Andrew


RE: QC Anybody
By OpaqueBubble on 3/12/2008 6:40:16 AM , Rating: 2
Microsoft has clearly stated since December that no files should be directly edited on the server, and specifically mentions Quicken. You were warned!


RE: QC Anybody
By mindless1 on 3/12/2008 10:57:28 PM , Rating: 3
Unless I missed a memo or something, Dailytech is not a distribution point for MS bugtracking to end users.

MS should have IMMEDIATELY set up a patch that warns users and limits the potential for corruption, even completely locking acess to the files if that's all they can get done on a moment's notice, rather than allow a potentially damaging access.

What MS is doing appears to be closer to treating your data as if it's an acceptible loss, leaving it in jeopardy every day, after week, after month. Were WHS boxes and license sales pulled from shelves and vendors? What exactly is MS doing to limit further data loss from existing and future customers henceforth until the problem(s) are fixed? I'd be pretty pissed if I bought a product already known to corrupt files using the advertised feature sets, if that's not clearly defective and grounds for a lawsuit then what would be?


RE: QC Anybody
By JustTom on 3/12/2008 4:00:13 AM , Rating: 3
quote:
MS had not even been able to replicate the bug.


Well actually they have replicated the bug. This was reported on back in December. They even have a pretty good handle on what is causing it.
http://support.microsoft.com/kb/946676/en-us

This has happened to me. My clumsy workaround is I just don't edit files on the server. When I am done working with a file I then copy it over. This is a little bit of a hardship since my wife and I work on the same files all the time, so we are back to thumbsticks and sneakernets.


RE: QC Anybody
By TheDoc9 on 3/12/2008 11:24:40 AM , Rating: 2
Never make excuses for a company, this is a HUGE issue and could cause serious problems for people. It's more likely that managers were over pressured, likely having too small of a testing team and decided to put green checks on all the test case scenarios to meet their deadline. Happens all the time in many software companies.
What people don't understand is how late that these development and test teams might stay every day and they still can't get these things under control without more time, but the deadline remains the same.


RE: QC Anybody
By cochy on 3/12/2008 11:32:58 AM , Rating: 2
If this bug is extremely rare and happens to almost nobody why do all the headlines state that WHS is "broken". Broken to me means unusable. Either the headlines are blowing this problem way out of proportion or this isn't as rare a bug as you say.


RE: QC Anybody
By BMFPitt on 3/12/2008 1:19:56 PM , Rating: 5
quote:
Broken to me means unusable.
The primary purpose of the product is to store your files. It cannot be trusted to do so. This makes it unusable as a file server.


RE: QC Anybody
By Gholam on 3/12/2008 4:43:55 PM , Rating: 3
The headlines ARE blowing this out of proportion, read the real story here:

http://blogs.zdnet.com/Bott/?p=348

It's just that Microsoft is given less slack than, well, pretty much any company out there. Sun, for instance, can ship a data corruption bug in OS meant for highest-end datacenters, and it'll only get a passing mention in very specialized sites. Microsoft, it'll get blown all over the internet.


RE: QC Anybody
By BMFPitt on 3/12/2008 9:41:10 PM , Rating: 2
quote:
Sun, for instance, can ship a data corruption bug in OS meant for highest-end datacenters, and it'll only get a passing mention in very specialized sites. Microsoft, it'll get blown all over the internet.
DailyTech is a specialized site. 90% of people have never even heard of WHS, and I'm sure far fewer know about his issue. And that's for a consumer product. People who get tech news from mainstream sources have never heard of Sun.


RE: QC Anybody
By Shoal07 on 3/12/2008 12:03:23 PM , Rating: 3
quote:
Because the bug is extremely rare and happens almost never to almost nobody?


Correct. Also, if you only have one drive IT IS IMPOSSIBLE TO GET CORRUPTED DATA. I shut my 2 drive system down to 1 until June. Guess what, I can edit on the server all day long with any progam and never worry. YOU MUST HAVE MORE THAT 1 DRIVE.

FYI - I was running two drives until yesterday and never had an issue. I only shut down to one, honestly, because sensationaist articles like this by half-baked bloggers like the above.


RE: QC Anybody
By OpaqueBubble on 3/12/2008 6:37:41 AM , Rating: 3
The majority of users do not have any problems with WHS - the conditions for this to occur are rare. This article is way over the top and overblown as usual from this "writer"


RE: QC Anybody
By PAPutzback on 3/12/2008 9:59:35 AM , Rating: 1
It is an OS not a shareware app. It can't be overblown. If Amazon's database became corrupt because the OS messed up would you think that was a non issue. You have to be on the WHS team to be so blind to such a critical flaw.


RE: QC Anybody
By OpaqueBubble on 3/12/2008 12:30:11 PM , Rating: 3
Nobody said it was a non-issue - it is definitely way overblown and MS has given workarounds.


RE: QC Anybody
By deeznuts on 3/12/2008 1:05:11 PM , Rating: 1
OpaqueBubble,

so how long have you been on the WHS team and how bad are you guys getting reamed over this right now?


RE: QC Anybody
By PAPutzback on 3/12/2008 1:56:52 PM , Rating: 1
Not to mention file corruption when the server does its automated updates, issues with Avast Antivirus, failed backups via wireless pcs.

I understand they had to release the O.S. at some point in time to get the bugs reported from the larger audience but loss of data is not a bug. This O.S. is crippled.


RE: QC Anybody
By OpaqueBubble on 3/12/2008 3:35:26 PM , Rating: 2
Sorry to disappoint you but I have nothing to do with MS. I am just a very happy user of WHS.
I have not had any corruption problems, or problems backing up wireless PC's. In fact my WHS has performed perfectly. I have used it on more than one occasion to recover from some accidents (self made) on client PC's (3 Vista PC's).
I am a software developer, myself and I suppose I sympathize more than perhaps I should, but I feel that the WHS team will fix this bug and if it's done in June then that's fine as long as it's done right.
When I got my EX475 in November I did not put all of my jpg's or music library on the WHS without ensuring that I had other backup means for all "precious" files. This was even before this bug was uncovered. This just seemed like common sense to me.
I do sympathize with folks who lost their files, but I feel that MS gave valid workarounds and "not to do"'s to avoid the bug.


RE: QC Anybody
By bhieb on 3/12/2008 11:36:53 AM , Rating: 2
Is it? How many issues of this scope, regardless of rarity, would be ok for a server OS? Would it be ok if this happened on Server 2003/2008? Just because it is for home use does not make the data less valuable, nor does it make this somehow not a server. This is a server and failures of this kind (even if rare), should not be tolerated.

That said the article does come off a little alarmist.


RE: QC Anybody
By encryptkeeper on 3/12/2008 8:54:46 AM , Rating: 2
For years people in the tech community have been saying that MS rushes products to the market and uses the public as free beta testers, and I think this is probably proof of our suspicion. Over time, they've probably gotten to be more and more half-assed when it came to rigorous testing.

"Some of the instances that were initially attributed to this issue ended up being something else, such as a faulty network card/driver, old routers with outdated firmware, or people incorrectly testing the limits of their home servers."

Wait a minute. How often is old, outdated equipment used in office buildings, homes, schools, and government facilities and it does NOT lead to data corruption?


RE: QC Anybody
By darkpaw on 3/12/2008 10:25:55 AM , Rating: 2
Not that I'm defending MS for this big of a bug, but considering this product was designed for controlled pre-built systems it is different from a typical OS. The OEM version wasn't even going to be released initially, they did it due to demand and it was pretty much clear that you were on your own with it.

WHS is not designed to be a general purpose OS that runs on any old hardware.

That said, I'm of the opinion that any data corruption bug (even if only 1% of users are effected) should be a very high priority to be fixed.


RE: QC Anybody
By bhieb on 3/12/2008 11:43:58 AM , Rating: 2
quote:
WHS is not designed to be a general purpose OS that runs on any old hardware.


Acutally it is specifically designed to do just that. Why else would you have a suedo-raid that can dynamically expand and collapse with different size/speed drives (heck even different interfaces). The hardware requirements are very loose, and it really is desinged to run on any old machine.

However if your argument is true then they never should have realeased it, thus proving the "rushed to market" theory.


RE: QC Anybody
By Gholam on 3/12/2008 12:20:23 PM , Rating: 2
Wait a minute. How often is old, outdated equipment used in office buildings, homes, schools, and government facilities and it does NOT lead to data corruption?

Not that rarely. I've personally seen an HP Smart Array 5i controller eat a RAID5 array on a server used by a government agency - and latest backup they had was 4 months old because that was when the tape drive broke down.


RE: QC Anybody
By ToeCutter on 3/13/2008 2:33:58 PM , Rating: 2
More importantly, how is it that MS is looking at old equipment causing the bug instead of their broken code.

Says volumes about MS attitude towards coding:

"It can't be our code that's causing problems, check everyone else's first!"


RE: QC Anybody
By GreenEnvt on 3/12/2008 9:41:55 AM , Rating: 2
As other have said, it's pretty rare.
I have been running since the day it was released, and the only issue I've had is some bittorrent downloads are corrupt. We stream video's off of the WHS to our Media center every day, and no issues there. We have a blogging site hosted on our server, it's also been rock steady.
Webguide running too, no problems.


RE: QC Anybody
By Gholam on 3/12/2008 4:40:57 PM , Rating: 2
You can run BitTorrent (uTorrent client) on WHS in service mode, and there's an add-in that plugs its interface into the console. Only thing you have to do is to put the temp files folder outside the duplicated shares.


RE: QC Anybody
By Gholam on 3/12/2008 4:39:13 PM , Rating: 2
You might want to present that question to Sun, who shipped ZFS with a data corruption bug present:

http://www.datacenterknowledge.com/archives/2008/J...


RE: QC Anybody
By ToeCutter on 3/13/2008 2:39:23 PM , Rating: 2
quote:
You might want to present that question to Sun, who shipped ZFS with a data corruption bug present


Why? No one mentioned Sun's ZFS bug here because no one here is using Solaris to store their pics, movies, music, etc!

WHS is targeted towards non-technical users; you can buy them at B&M retails stores.

I'm still astonished at how many here have a "don't worry about it" attitude towards this issue. If those folks can live with it, fine.

But it's certainly no reason to defend WHS as an excellent storage platform to entrust your data with. That suggestion is simply reckless and ignorant.


"This is from the DailyTech.com. It's a science website." -- Rush Limbaugh














botimage
Copyright 2014 DailyTech LLC. - RSS Feed | Advertise | About Us | Ethics | FAQ | Terms, Conditions & Privacy Information | Kristopher Kubicki