Wednesday, February 11, 2015

The Day My Computer Caught Fire

I never intended for this blog to become "Will's Occasional Rants About Bad Customer Service." Honest. But in this case, I think the story is worth telling in detail. Tweeting it 140 characters at a time and/or a couple of technical message board posts aren't going to cover it.

Yesterday, my house nearly burned down. The fact that it didn't was due to nothing more than time. If two or three more hours had passed, our home would be a smoking cinder burying the remains of Maggie the bullmastiff.

Here's what happened. Around 4:30 on Tuesday morning, my wife and I both woke up to an acrid odor. It initially smelled like somebody had just struck several matches.

We got up and had a look around. There was no smoke in the air, and our first thought was that something was wrong with the furnace. We turned it off, and the odor seemed to fade (we realized later we'd just stopped it from being recirculated through the HVAC system after the fan was turned off).

I checked my office, across the hall from our bedroom, and noticed my computer had rebooted, which was odd, but nowhere near an indication of anything dangerous. It was stuck on the BIOS boot screen, which didn't surprise me; I'd never bothered to fix the boot drive settings, and it always did that on startup unless I held down the F12 key and told it which hard drive to boot from. Not noticing anything else unusual, I turned it off from the main switch on the front of the box and left the room.

By now it was too late to go back to sleep. My wife had started to get ready to go to work when, simultaneously, the smoke alarm and main house fire alarm both went off.

We still couldn't see any smoke yet, but the alarms were enough. My wife grabbed Maggie by her collar and hustled her out to one of our cars. I picked up the phone and dialed 911.

While I was waiting for the operator to pick up, I walked back into my office. Now the combustion smell was overwhelming, and I could see light coming from under my desk. When I walked around it, I saw flames shooting out of the case of my desktop computer, which was on the floor under the desk.

I dropped the phone and ran downstairs for a fire extinguisher. I'm pretty sure the only thing the operator ever heard me say was something unprintable. Fortunately our Ooma phone service had sent our address along with the call.

I grabbed the extinguisher from under the sink and pulled its safety tab out as I hurtled back up the stairs. Did you know the thing that looks like a trigger on some fire extinguishers is not actually a trigger? I didn't.

Turns out to make the thing work, you're supposed to mash on the lever on top top, not yank on what looks like a trigger. It felt like it took half an hour for me to figure that out, but it must have just been a few seconds. The extinguisher finally belched out its powder. A couple of blasts were enough to put out the fire. The room was now nearly filled with billowing smoke; maybe a minute or more of that and I wouldn't have been able to get back in to put out the fire. I opened a window and backed out the door.

The Cobb Fire Department arrived a few minutes later, checked out the scene, and confirmed the only damage was to the computer, the front of which was melted into plastic slag. They carried the computer out to the driveway, made another sweep of the house, declared all-clear and went back to the station.

Once the computer cooled off, I got my first good look at the damage.

From top to bottom, what you see there are the bay where the computer's data drive used to live (I pulled it out before this picture was taken; the bay was destroyed but I have some hopes the drive itself survived), two optical drives (both ruined), and below them, right at the point of the most intense destruction, the bay that had held a pair of sleds for 2.5" boot drives.

Now, here's what's unusual about that bottom bay: If you look closely, you can see the remains of a 2.5" hard drive, buried in the melted plastic just above the vestigial floppy drive. It's a Western Digital HGST Travelstar, model H2IK500852SP (HGST is the former Hitachi hard drive business that was bought out by Western Digital a few years ago). I bought that drive on Sunday, and installed it roughly twelve hours before the fire. On closer inspection of the sides and surrounding damage, it's clear that the fire started in that bottom bay, which at the time only held the new HGST drive. The motherboard and CPU (normally the hottest component in any computer) and power supply all appear to be undamaged.

How exactly the fire started, I couldn't tell you, not least because that hard drive is now solidly encased in post-fire slag. But based on the fact that the drive was installed on a removable sled similar to the one in the picture below, we know that the cable from the power supply was neither moved nor ever plugged directly into the hard drive itself.

That boot drive was, by far, the newest component in the computer. I hadn't opened the box up in months, and I feel safe in saying the power and data connectors to that drive bay hadn't been touched in years.

While manufacturers will tell you that it's impossible for a hard drive to start a fire (and indeed, the young guy I spoke to at HGST yesterday said just this) a quick search indicates that such fires, while certainly rare, are far from unheard-of. In a few minutes on Google, I found multiple user reports of fires starting from Western Digital drives: here, here, here and here.

Again, this kind of failure is extremely rare. Hundreds of millions of hard drives operate 24/7 for years on end without getting so much as overly hot to the touch, much less catching fire. Google's massive data centers, using thousands of drives at a time, operate day in and day out at well above 80 degrees Fahrenheit, and Google wouldn't do that if they were worried about hard drive fires. I've owned dozens of drives personally and probably handled thousands of them during my career as an engineer, and this was (lucky me) the first time I'd ever heard of--much less nearly been scorched by--a flaming hard drive.

But in this case, that's what appears to have happened.

I'm sorry (but not terribly surprised) to say that neither Western Digital nor their HGST subsidiary were quick to express concern when I contacted them yesterday. As noted above, a young HGST support rep (not meaning to be ageist here, but I'm pretty sure I have t-shirts older than the kid who answered my call) read off what I assume is the lawyer-crafted script that's to be used in case of any call reporting physical damage ("our drives are built to a specification to meet a requirement. The problem had to be in the environment"). I was reminded briefly of this guy from "Titanic":

I get it: no company is going to admit any kind of liability based on a phone call and/or a few Tweets. But honestly, I'm not all that concerned about my loss here, and least of all any warranty issues with that HGST drive. A $50 hard drive is close to the smallest of my worries, and the few hundred bucks worth of trashed hardware won't break me.

What concerns me is the thought of what would have happened if my wife and I hadn't woken up thanks to a bad smell. What worries me is what would have happened if I hadn't ambled into my office just in time to see the fire before the smoke became unbearable.

What kept me up for most of the last night was thinking about how lucky we were that this happened at 4:30 instead of a couple of hours later, when we'd have been on the road to work, and Maggie would have been locked inside an inferno.

That, my friends at Western Digital and HGST, is why I am not letting this drop, and why you ought to be taking this situation a lot more seriously today.

UPDATE (21 Feb, Thursday): Just got off the phone with a Global Service Operations Manager from HGST, who I'm happy to say was far more concerned about the incident than anybody else from either HGST or Western Digital proper that I've talked to so far. He'd seen the blog post and pictures, and seems to be at least as gobsmacked as anyone else at the level of the damage. They want to get the remains of the drive for analysis, and as that's exactly what I want to happen as well, I take this as a good sign.

He offered to replace the drive... I had to decline. "Don't take this personally, but at this point I really don't want another one of your hard drives in my house." Granted, not the most rational decision in the world given how unlikely it is this would happen once, much less twice, but at this point I think I'm going to stick with it.