Systems Administration

Sometimes, asking a question can be a big challenge. Because even if your question is very clear and precise, there’s always someone that will get it wrong. Perhaps that would be something to expect in fields like fine arts or philosophy, where so many things are open to interpretations and debate. But when it comes to highly technical matters, it’s amusing to see all the ways that things can go wrong after a question has been asked.

How many time have you asked a technical question on a forum or newsgroup, and have only received lots of useless, clueless, wrong or even arrogant answers?

Obviously, human beings can’t always provide a right answer to a given question, and that’s something we have to live with. But nonetheless I wonder: what’s the point in giving an answer when...

• you’re not answering the question that was asked
• your answer is clearly not helpful (and you would have noticed that if you had thought for a minute before answering)
• you really don’t know what the answer is (and maybe you even know that you don’t know)
• you don’t have real world experience on the subject (and no, having a friend who knows, or having read a paper on the subject is not equal to having real experience)
• you’re not qualified for answering (for example, because a question is about medieval history, and you happen to be a car mechanic who hasn’t even studied history as a hobby)

So let’s see how a clear, simple, precise question can be answered in every possible wrong way.

Fair warning

Continue reading "How (not) to answer technical questions (or "the cow riddle")"

Posted by Marcus Friedman

In case you’ve missed it (even when the chances of that happening are low), there’s a new website for system administrators and IT professionals that you’ll probably find useful. Server Fault was launched on April 2009 as a private beta, and went into an open beta on May 2009.

Basically, Server Fault is a place where system administrators can ask questions and get answers from peers willing to share their expertise. It uses OpenID, so if you already have for example a Google, Yahoo or Technorati account, you can use those credentials for authentication.

The website implements an interesting feature called reputation, which is a scoring system similar to those used in some bulletin boards.

There are literally thousands of questions that have already been asked (and answered), which indicates that the site’s popularity is quite good. If you take a look around, you’ll find very different kind of questions: from the most basic you can think of, to the extremely specific or complex ones.

From time to time you’ll also come across some question that is totally irrelevant to systems administration, but that’s something that cannot be avoided (anyway, the voting mechanisms offered by the website help a lot to separate the wheat from the chaff).

Asking a question is a different story. Sometimes you can be lucky and get pretty good and clever answers. And some times the only thing you’ll get will be useless ones. Or even none at all.

I’ve been testing Server Fault by submitting some tough technical questions, and while sometimes I got really good answers or hints on how to solve specific issues, I also got quite a lot of irrelevant replies. But even in those cases, I’ve found the overall experience quite amusing.

As with any other online resource, this one certainly can’t replace the professional advice you could get from an expert on a given field. But I think it’s a good place for sharing knowledge amongst peers.

Defined tags for this entry: server fault, systems administration

Posted by Marcus Friedman

“Have you tried turning it off and on again?“ — In the British sitcom “The IT Crowd“, that’s the first thing that Roy asks every time a user calls the help desk. He portrays a technical support guy who may know more about computers than the average person, but whose troubleshooting skills are rather limited.

How many times have you seen a system administrator rebooting a server or a workstation in order to “fix” it?

Unless there’s a very good reason to perform a hardware reset, that’s one of the most unprofessional things that a sysadmin or a support technician can do. Unfortunately, it’s something so common that you will see it happening again and again.

In most cases, performing a hard reboot before exhausting all other options prevents you from diagnosing the underlying problem. And without a proper diagnosis, there’s no way to find the root cause of the issue. So instead of solving the problem and devising a solution or implementing a documented workaround, you’d be just masking its symptoms. But the problem will still be there, and sooner or later it will come back affecting your users, your work or disrupting your company’s operations.

If you’re managing open source platforms, there are hundreds of tools and resources to help you get to the bottom of the problems you may face. And of course you can always get to the source code level if you need to. There are really no excuses for resetting a system without performing a thorough diagnostic and following proper troubleshooting procedures.

Rebooting isn’t always a bad practice though, and if you’re dealing with heisenbugs it might be necessary (and even recommended) to perform scheduled service restarts. It can also help to reclaim stale resources, and it can be beneficial in high availability environments. [ George Candea and Armando Fox wrote about this in a paper titled “Recursive Restartability : Turning the Reboot Sledgehammer into a Scalpel“ ]

But every time a professional sysadmin performs a reboot for the sake of it, or because he doesn’t know better, he’s just showing his true level of incompetence. To paraphrase Isaac Asimov, we could say that “Rebooting is the last refuge of the incompetent“.

Posted by Marcus Friedman