More on Admin Responsiblity

Since the drama is over, I suppose I can finally touch on San Francisco's recent network issues, namely a network admin holding the network devices "hostage". Sort of.

If you aren't familiar with the story, here's a brief rundown.

Network admin Terry Childs built the San Francisco FiberWAN, the backbone that municipal data travels on. To say he was an insanely-protective admin is an insult to the insanely-protective admin community. According to one report, he was so secretive that he refused to write configs to flash. Now THAT, my friends, is being too paranoid. In the end, the city tried to fire him, and he refused to hand over the authentication information, and he booby-trapped the network so that he could disable it and erase the config from outside if necessary. The mayor of San Francisco eventually talked him from his proverbial ledge and coaxed the passwords out of him, as apparently the mayor was the only man Childs trusted.

The end result was that San Francisco went a week and a half without having access to their WAN equipment. Now, I imagine the remaining admins are scouring every line of configuration trying to make sure that Terry didn't leave any other backdoors or vulnerabilities. I don't envy their job at all.

You've probably realized how this relates to people like us, who admin small networks by ourselves. The major mistake San Francisco made was placing one (apparently unstable) person in charge of the infrastructure with no oversight. That sounds almost like my job.

We're in this position by design. By being the only admin of a small infrastructure, we have a high bus factor. Unnecessary secrecy has no business on our networks. We touched on this last month, after MSNBC reported on IT worker ethics. Nothing has changed since then. Being prepared as an organization means guarding against employees who get hit by buses or turn evil.