Square Pegs In Charge Of Round Holes (Long Rant)

At first glance, this may seem like a geek story, but it's actually indicative of a significant societal problem. This is the story of Terry Childs. Google his name and you will see loads of news stories about a "maniacal" or "rogue" IT Admin that is holding the infrastructure of San Francisco hostage. I'll invite you to temporarily withhold judgement.

At the outset, let me say that what he is doing (not what he has done) is wrong. I think I can shed some light on the situation even though I've never met the man, and in doing so indict the underlying problems of non-technical people making technical policy, typically inappropriate workloads, and the media- rushing to make you live in fear and conduct trials in public opinion.

I've lived some of these things first hand and can speak authoritatively about the nature of these problems, and how even well meaning idiots make matters worse. This tale starts with Terry Childs being responsible for the infrastructure of the city of San Francisco network. I'll keep it simple and say that he was THE responsible person for the security and uptime of this network. Really dumb idea number one. There is no network of that size that should be the responsibility of just one man. The alleged back story is that his boss and peers were (and possibly still are) incompetent, and for quite a spell they were all content to let him make all the decisions and do all the work. This is damning enough to get me to side with Childs at the get-go, but at some point recently a new boss came along who apparently rubbed Childs the wrong way. For those of you who watched Seinfeld, recall the episode where Jerry's mechanic stole his car in order to repair it properly and give it the TLC that jerry wasn't. Much the same is happening here. IT guys* can be quirky in their passion, and it's not uncommon for us to look upon our infrastructures as "our babies", including all the protective behavior that a parent would exercise with a child. Or perhaps it's simple pragmatism. "You can't touch anything, because you'll break something and I'll have to fix it at 2AM or in the middle of the busiest part of the workday, when the harm is greatest." I'm guilty of this to a degree, as are most IT guys who care about what they do.

This sort of thing needs to be tempered obviously, but in Childs case, no one was willing or able to work with him in this network. Lazy, stupid, or weak willed- perhaps all three characteristics apply to his supervisor(s). He built the network himself, apparently alone, and was in charge of it for years without help. Anyone with an interest in disaster recovery will already spot problems. What if he got hit by a bus? Dumbasses are in charge apparently in San Fran. He was profoundly overworked and not compensated for overtime. Interestingly, the medical and IT professions are the only two (last time I checked) that are specifically exempted form the usual rules of overtime compensation, according to the law. Just the two groups of people that you should be pissing off, eh? As a part of this creation, he established passwords necessary for network hardware configuration changes. This is as ordinary as copper wire, and would only be noteworthy if he DIDN'T do so. And for years no one but he knew/knows what these passwords were/are. Yes, it should be documented. Yes, he should be telling SOMEONE. No, the powers that be never asked or insisted.

So- to be clear, things are working, and apparently working well the entire time TO DATE. Childs gets this new boss with whom there is friction, and the ham handed fool presses Childs into a space where the boss has no leverage beyond employment, a thing Childs is willing to abandon. New Dumb Boss threatens Childs, and surprisingly, the boss (and by extension, the city of San Fran) gets burned. Wow. Who'd have thunk it? I like to know two things- one, what college turned out this genius of a boss, and two, who hired him? Don't get me wrong- it may have been unavoidable. I don't know Childs, and I may be giving more benefit of the doubt than warranted, but in my experience I side with the IT savvy every time over the bureaucrats. You show me a man* who has a passion for his work, and I'll show you a man who can be reasoned with regarding the welfare of said work.

So, now the problem is that the whole CIty of San Francisco network is on "auto-pilot", Childs did no harm (that has been substantiated so far- I discredit the reports that claim he placed bug or other nefarious mechanisms until I see credible evidence) to his creation, and in an ironical manner the quality of his work is now on display in that it has lasted this long without intervention- despite all the popularity and attempts to "break in". The City got the fuzz involved, and they demanded that Childs disclose the password. He either did, and the password he gave is no longer valid (mighty unlikely), or he gave them a false password. Either way, no progress was made and this even includes the active involvement of the hardware manufacturer, Cisco. Again, I'll say that he SHOULD give up the password, and short of causing embarrassment or pain to the City, I don't know why he wouldn't. In that he wouldn't or didn't give up the correct password, they arrested him on charges of computer tampering? Huh? I mean, huh?

Tampering is a ridiculous charge considering the history, and unless they have more evidence that shows Childs did something beyond the scope of his work, I'd say they've got nothing but a scare tactic on the man. Meanwhile he sits in jail with a $5M bail. Yeah. 5 Mil. The City says that undoing what he's done (or really, NOT done) will cost millions. If that's true, why did they allow such a big deal to be done by only one man? Again- what if he'd dropped dead? Maybe they should have some auditors stop by and get up their butts about how they so depleted the IT staffing budget that they are clearly failing in their duties? Bare with me while I focus my rant here for a moment- it's insane how most places these days are expecting something from nothing when it comes to IT expenses. Perhaps it's a backlash against the absurdity that was the dot-com bubble, I can't say. What I can say is that it is now the norm to see IT shops using their own version of "Enron accounting" to meet goals and these lies are only exposed when it makes the front pages. There are regulations (of arguable merit) that are not voluntary that govern IT systems- HIPAA, Sarbanes-Oxley, an assortment of CFR's and credit card industry regs- and I know how burdensome they can be because I make the effort to comply with the ones that apply to my work. I talk shop with others and I'm astounded at the degree to which these are not truly met, or are obfuscated in one way or another. Mostly, they are deferred with a shrug, and this is mostly the result of either the wrong people driving the department, or lacking resources to make compliance happen.

[Sub-rant]
One area related to this is H1B Visas for foreign workers. I know three facts about this that are important. One- having hired H1B's, I can say that this system is a fine way to bring in the kind of talent that we can use. I don't regret the hires I've made, and can say with clear conscience, that they were the best candidates for the jobs. Two, there are too may people in IT who are in it for the money and have no passion or skill. Yes, it's lucrative. So is air traffic control. Let's be selective- some of you need to go back to whatever it was you were doing before. Three, the businesses that hire H1B's almost always do so to undercut the prevailing wage (which is low, relative to the responsibility in my estimation) and to have slave workers who can never complain or leave. The H1B Visa rules should be amended to include a 15% premium over the prevailing wage for foreign workers. This way business will have no incentive to ignore the plentiful capable domestic workers that are available. Only the foreign workers that truly cannot be met with local help will be accepted. Hell, you might actually see some money coughed up for training now and again.
[/Sub-rant]

Then the mayor and others start characterizing Childs in the press as "rogue" and "maniacal". Maybe it takes one to know one? In any event, I'd throw out every word these people said, as they are not in the industry and do not know what they are talking about. Would you listen to the mayor tell you about induction in microcircuits, or about laser surgery on the hippocampus? He knows nothing- trust me. The day Bob Metcalf or Vint Cerf become mayor somewhere is the day I take that back. The mayors handlers have tried to distill it all down to sound clips designed to get the media types all hot and bothered. Great- situation normal, which is to say detached from reality. Remember, politicians make the rules about things they don't have the first idea about- The could intuitively tell the condition of Terry Schiavo and they know that they internet is "not like a truck", but rather it's a "bunch of tubes". It scares me that they can legislate what they can't even describe. These are the minds that decided to jail Childs on a charge of tampering. Holding hostage, perhaps- but not tampering.

Next the media does their part and sensationalizes the crap out of this, making Childs sound like he's all but shot up the town with an AK-47. Bullshit litmus #1- if you hear the work "hacker" in the coverage- ignore the whole story. Childs is no hacker. Problem employee, burned out by too many years of unrewarded overtime, perhaps. Pain in the ass? Likely. Hacker? No way in hell- in the same way the warden at the prison is not an escape artist.

Now, if it turns out that Childs has a ransom demand for release of the password, I'll need to revise this view, but until then, no matter how ugly this gets, I point the blame squarely at Mayor Gavin Newsome and his IT department leaders.

* In IT circles, there is a shocking imbalance of gender, such that I say "IT guy" or "Man" in lieu of gender neutral terms merely because in using male terms, I'll be accurate. Why females avoid IT is a sad mystery.