Pablo Averbuj

Useless start-up: Afeeda

Saw this over on Mashable. Basically the same idea that I wrote about in December. It’s starting to feel like the dot-com days are back, for better or worse.

ReadyNAS round 3: Fatality!

Today went much better than yesterday. I turned the ReadyNAS on and at first the fan only spun slowly. After it started getting past the early boot stages it spun up and stayed at high RPM. The problem is not entirely fixed. The OS doesn’t seem to recognize the fan, so I can’t get metrics on fan speed, and if the fan stops, I won’t know until it overheats and shuts down. However, it seems to be working ok since 7am when I started it. I did seems to get warm around 6:45pm but seems to be alright since. Frankly, I’d be surprised if it stays up the entire time we are gone. I think that either the fan is flaky or the motherboard is having issues. My hope is to have Infrant send a new fan tomorrow and install it when I get home from the trip. If that fan doesn’t work then I’ll start thinking it’s more than just bad luck with fans.

Sun Jun 17 19:00:23 EDT 2007 Disk on channel 2 has reached warning level (55C/131F).
Sun Jun 17 18:58:23 EDT 2007 Disk on channel 2 has reached warning level (55C/131F).
Sun Jun 17 18:56:23 EDT 2007 Disk on channel 2 has reached warning level (56C/132F).
Sun Jun 17 18:54:23 EDT 2007 Disk on channel 2 has reached warning level (56C/132F).
Sun Jun 17 18:53:04 EDT 2007 Disk on channel 2 has reached warning level (57C/134F). Disk on channel 3 has reached warning level (55C/131F).
Sun Jun 17 18:48:24 EDT 2007 Disk on channel 2 has reached warning level (56C/132F).
Sun Jun 17 18:46:23 EDT 2007 Disk on channel 2 has reached warning level (56C/132F).
Sun Jun 17 18:44:38 EDT 2007 Disk on channel 2 has reached warning level (56C/132F). Disk on channel 3 has reached warning level (55C/131F).
Sun Jun 17 18:42:23 EDT 2007 Disk on channel 2 has reached warning level (56C/132F).
Sun Jun 17 08:55:37 EDT 2007 System is up.

ReadyNAS round 2: FIGHT!

Today I fought with the ReadyNAS on and off. At first I was not able to get the fan to spin up at all. I tried the new fan, the old fan, I tried spinning it by hand. It sensed the fan (ie, when I spun it by hand the RPM registered (side note, I got it up to 7000 rpm without assistance from the motor). Finally as I was about to give up, I made an odd contact and for a brief moment it spun up. It seemed that if I held the connecter in a particular way, it would spin, but as soon as it moved the smallest amount it would stop. The connector on the new fan was not what I would consider a snug fit, whereas the old fan had a snug fit. i noticed with the old fan if I spun it manually it would continue to spin indefinitely although at a very slow speed. I considered the possibility that the fan connector on the motherboard was making a bad connection, but I could pretty reliably get the old fan to spin at low RPM, and the new fan would not. I chose to overlook the fact that it worked perfectly and consistently when I first plugged the fan in, and decided the new fan was not making good contact.

I scoured the house looking for a 3-pin fan connecter I could cannibalize and shockingly found none. Left with no other choice I sacrificed both fans to create one SUPER FAN consisting of the new fan assembly and the old fan connector. During this process I remembered why it is I stick with software and stay away from hardware hacking. I know, it’s pretty grotesque. So I took my handy work and hooked it up, nothing. I fidgeted here and there, tried to make it connect the way it did before. Again, just as the hard drives are about to overheat (and cause the ReadyNAS to shut down), it made contact! The beatiful sound of 3000 RPM generated by the wonders of electricity. I set the fan gently in its approximate location and let it cool the hard drives down for an hour or two. When I came back it was still working, so I decided to reassemble the appartus without shutting it down. I assembled it one way, but the fan cable wasn’t long enough (I was trying to keep the cable taut so the bare union wouldn’t touch metal (I know, genius!). I tried a second way and it was long enough but I just barely couldn’t make the pieces fit back together. I took it apart and started on the third attempt when tragedy struck, it stopped spinning! I tried in vain to resuscitate my creation to no avail. Eventually the drives overheated and the device aborted it glorious mission.

I’ve checked the websites of the nearby hardware suppliers and nobody seems to carry the 92mm fan I need. I even probed a few friends known for their hardware hoarding compulsions with no luck. Tomorrow I may try again, but I think at this point it looks like resolution won’t come without talking to Infrant on Monday.

ReadyNAS rollercoaster

First a side note. When I was corresponding with the Infrant support guy, he asked for a copy of my invoice. I forwarded him a copy of the invoice along with thread (in case the ticket tracking system needed something in the body). In the process, I accidentally copied the sales guys at ExcaliberPC (where I bought the ReadyNAS last year). I noticed my mistake but i figured they would get the email, see it was regarding a purchase made last year, and delete it.  Much to my surprise, last night (Thursday) at around 10pm I got a call from a support guy at ExcaliberPC calling to check if everything was alright with the device and if there was anything he could do to help. Despite the time of the call, it was a pleasant surprise to see customer service handled so proactively. I am very impressed.

Now, for the roller coaster that was today’s adventure. When I got home from work, Leigha told me I had received a package and I knew it was the replacement fan. It never really crossed my mind that the replacement fan wouldn’t work, so I didn’t get around to replacing it until after 7pm. At around 7:30pm I had it all reassembled, turned it on, fan didn’t spin. Checked the control panel and no fan was detected.

It was at this precise moment that a lot of information came together.

1. Infrant is based in California where it was now 4:30pm
2. It’s Friday
3. I’m out of town all next week!
4. If I don’t get this working now, I won’t be able to troubleshoot remotely!!
5. I won’t have this fixed for like another 2 weeks!!!

This is around when panic set in and I launched a full assault on Infrant. One if by landline, two if by SMTP.

Now I’ll add another bit of information to the puzzle. Early versions of the ReadyNAS NV had a fan issue that required a little adapter. My unit was supposed to be affected by the issue, so when they shipped the fan they helpfully included the adapter, which I dutifully installed. As soon as I hung up after leaving voicemail, it occurred to me that maybe the adapter is unnecessary/faulty and I should try without it. I take the thing apart, remove the adapter, and (without reattaching all the screws this time) as the fan spins up, the phone rings. I talk to my tech guy (kudos for always talking to the same tech) and tell him it’s spinning now without the adapter. He tells me the adapter is only necessary for certain board/fan combinations. I’m happy so I apologize for assaulting their communications but explain the 5pm thing and the vacation thing and he confirms they close at 5pm.

At 4:fucking59pm PST (8pm EST) the thing stops spinning. The control panel says “out of spec” for the fan. RPM == 0. I assault again, but this time I’ve cried wolf and everyone is busy drinking beer so no return phone call. After a few minutes it seems to  restart itself. It’s certainly not comforting but if something goes wront it shuts itself down so I let it ride.

Subject: Unexpected enclosure status detected [nas0]
Date: Fri, 15 Jun 2007 21:14:47 -0360
Fan 1 has failed.

For extra insult, in the meantime I had gone back and added my cellphone to the list of emails to notify in case of failure. So it starts losing it’s mind as 10 more messages come in alarming of ever increasing hard drive temperatures, culminating with:

Subject: Disk temperature limit exceeded [nas0]
Date: Fri, 15 Jun 2007 21:42:00 -0360
The reported temperature of a disk in the NAS has exceeded maximum recommended temperature.  Shutting down the system to prevent hardware damage.
Disk on channel 2 reports a temperature of 61C / 141F.
Fan 0: 0 RPM

Since I’m sure they are closed over the weekend, I’ll spend my time searching the message boards for similar issues and tinkering with the fan adapter to see why it wouldn’t work.

Score:

ReadyNAS: 2
Infrant: 1
Pablo: 0

Interesting Screenshot

I was just about to start looking for somethingon the Costco site when I saw a smaller picture of this Dell Laptop. Before I even read the caption my first thought was “Neat, Costco is selling MacBook Pro’s”. I clicked through and found a Dell. I confirmed that sure enough that is the default OSX Tiger aqua blue background. I find it hard to believe that Apple would ship with stock art, so I think someone decided to co-opt the background for marketing purposes.

Why I paid extra for a NAS chassis

As you’ll see below, the main fan in my Infrant ReadyNAS NV died recently. It died on Wednesday and I didn’t notice until Saturday. Good thing it turned itself off before the drives overheated. Below is the transaction of emails I received before it turned itself off to prevent damage to the disks (this is an option, but I can’t remember if it’s on by default or not).

Subject: Unexpected enclosure status detected [nas0]
Date: Wed, 06 Jun 2007 05:25:07 -0360
Fan 1 has failed.

Subject: Unexpected enclosure event detected (nas0)
Date: Thu, 07 Jun 2007 02:32:56 -0360
Disk on channel 2 has reached warning level (55C/131F).
[Thu Jun 7 02:32:36 EDT 2007]

More emails after the jump but you get the idea. I just noticed the odd notation for the timezone (-0360 == -0400). Also the host monitoring I use (‘spong’) also caught the shutdown and emailed me. The reason I didn’t notice until Saturday is because I’ve been too busy to keep up with my personal e-mail.

Thursday, 06/07/07
——————————————————————————
red S 02:54 nas0 http no response from http server
red S 02:54 nas0 ping ping failed for 10.0.0.7

I submitted a ticket to Infrant support and they responded Monday, I provided them a copy of my invoice on Tuesday and they shipped out the replacement fan this morning (hoping to have it by Friday or Saturday before we leave town). It would have been nice to get the replacement part faster (or be given the option to upgrade the shipping) but it’s not usually critical to have access to the data on a day-to-day basis, just very very convenient.

It’s odd that I didn’t get a shutdown email from the NAS, but given the choice between sending an email and shutting down, I’d rather the device shut down. It’s clear from the message trail what happened. Also, I should configure spong to page me on NAS outages.

Continue reading Why I paid extra for a NAS chassis…

Where I’ve been

The last time I had the time, energy, and opinion to write something was December, nearly six months ago, and it ended an unprecedented 10-month streak of posts. Around that time the speed of life shifted into fifth gear and slammed the accelerator because although I’ve had things I’ve wanted to post aboud I rarely had the time or energy to do it.

In December I changed roles at work. I went from being a programmer to an architect, and I moved from Internet Operations in the CTO organization to the Product division (in Video). I’ve been doing video-y things since then. The move was dramatic in many ways; we spent nearly the entire month of January visiting our remote offices, meeting with our new colleagues and getting to know them, what they do, and how they do it. It was a whirlwind trip in and of itself, except at the end of the month of January, we closed on a house, a work in progress since Thanksgiving.

Combine the duties of a new job and the stress of a moving and a new house and you’ve got yourself a perfect storm.  In case you might think I’m over dramatizing the whole thing, consider the following. We were scheduled and locked to close on the house on Monday, January 29th. On Sunday the 21st, I flew to California. On Tuesday in California I was diagnosed with an ear infection and advised not to fly until Sunday the 28th. On Wednesday, my colleagues departed for the Tucson office, while I boarded the train that would take me across the country for the next three a half days. I arrived back in DC on the Saturday before the closing (incidentally, the following Tuesday I still had the same ear infection, so the gamble paid off).

I’ve had a lot of things on my mind, especially regarding work and video and I’d like to share them once I’m clearer on the blogging policy at work. Hopefully this post is a prelude to turning the slow drip of postings back on.

Comment Spam Solution

Comment spam seems to be a recent plague that has reached the cat/mouse stage of the arms race. I haven’t had too much of a problem on our blogs, Akismet catches it for the most part. But other people have been lamenting it lately, like on the Jira Developer Blog.

Yesterday I saw the marginally-useful NSFW HTML attribute mentioned on Slashdot and it occurred to me that this was actually a useful solution for comment spam. Follow me on this…

1. Create an attribute to the <a> tag called “spammy” or “user-submitted” or something that effectively translates into “this could be spam”.
2. Then the search engines can choose to threat it differently. Either by ignoring it entirely, treating the link suspiciously, or giving it less credence.
3. Blog and software developers can add this either forced or as an option to the blog/software configuration. I’m guessing about 7 vendors account for 99% of all blogs. Digg, Jira, anyone who suffers noise due to spam can implement it.
4. Blog spam becomes ineffective for rasing search engine standings; spammers stop (hopefully). Or at least they pursue sites/software that doesn’t implement the new attribute.

I’m having trouble seeing why this wouldn’t work. Once spam decreases people can allow anonymous comments or turn off moderation and the world can be happy again. There is one critical assumption in this plan: that a significant amount of the comment spam is used in order to try to raise search engine result position (SEO basically). If spammers are hoping to just get eyeballs to click then this wouldn’t really solve that problem at all. However based on the comment spam I’ve seen it looks like SEO is the primary goal. This seems a lot more beneficial that NSFW attributes.

Stupid Camera Trick

I’m not sure what poltergeist lives in my computer but I am constantly having crazy problems. For example, tonight I rebooted for the heck of it, when I looked at the computer, it says “No boot device found.” The following is my internal dialogue:

Ok, is there a CD in? No. Huh. Ctrl-Alt-Delete.
“No boot device found.”
Son of a … Power off, power on. Great it’s booting into windows.
[Blue screen of death flashes by...]
“No boot device found.”

Now, I’ve played this game before. I get out the video camera, set it to record, point it at the monitor and try again.

[Blue screen of death flashes by...]
“No boot device found.”
[Spend 5 minutes trying to pause the tape at just the right frame]

There had to be a better way. I did the math.

The image stays on the camera about 2 frames. Ok, at 30fps, that’s 1/15th of a second. But then I have to time the opening just right.
[Lightbulb goes off in my head.]

I set the camera to manual, bulb, smallest aperture possible. Point it at the monitor, and when I know it’s about to BSOD, I hold the shutter open. Perfect! Well, except for the broken computer. Click the thumbnail and you’ll see that even at f/29, the message was up long enough to leave a ghost image that is easily readable.

On an unrelated note, the last time I had this problem it was bad memory (it took me a long time to figure that one out, and only after it ate the registry). This time seemed like a similar problem. Something ate the SECURITY registry and it couldn’t load the hive. It booted in safe mode so system restore worked, but every time something like this happens a little part of gets closer to running Mac OS X.

Like I was telling Vlad, I can’t think of a single thing short of filesystem corruption that could happen to a Unix box that would prevent me from being able to boot it. In Windows one magic file gets corrupted and I’m up the creek. I don’t know much about Vista, but I would be surprised if they’ve solved these sorts of fragility issues. And the memory tested fine this time.

The Megafeed

It occurred to me that one of the neat things about Google Reader is that you can create meta-feeds based on any tag you have. So for example, I make my starred items public so anyone can (and should!) subscribe to my items. You don’t have to subscribe to the feeds I subscribe to, but I can filter my feeds for interesting things for you.

Then it occurred to me that there’s probably a few people who are interested in all the websites we have, so I created what I call the megafeed. This is a single feed which you can (and should!) subscribe to that will aggregate the posts from all of the Averbuj family blogs. And the best part is I can add new feeds (such as the wedding feed, for example) without you having to do anything. The feed currently aggregates my blog, Leigha’s blog, Anna’s blog, and a feed for Anna’s photo gallery. One stop shopping!