“It is the first responsibility of every citizen to question authority.” Benjamin Franklin
Hopefully I’ve been faithful to this responsibility over the past 4+ years of this blog.
We’ve seen the spectacle of the President of the United States (POTUS) berating the (Republican) Congress in front of a gaggle of first responders. So let me paint a mental picture of some folks that should stand in front of the POTUS and the tax paying public and take a verbal beating; The United States Civil Service system and many of its employees.
A long term career in the private sector but working mostly DoD projects alongside Civil Service has given me a perspective on the inner workings of Civil Service, and believe me, these organizations have been and are being given a huge pass on this whole budget mess and their part in it. Allow me to tell of a few experiences and observations I’ve accumulated since the late 1960s”
- In 1969 as a student at San Diego State I had my first encounter with a Civil Service organization at a large Navy lab. What I saw gave me pause at the time, and left a sour taste. Federal employees in this particular organization would often play cards during lunch time, and then spend a considerable time afterwards discussing the game they had just played. This was a regular occurrence with this bunch. I left with the impression that this particular organization was a dumping ground of sorts for other more professional organizations who needed to rid themselves of incompetent or underperforming team members. Lest you think I am being unnecessarily harsh, I found out several years later (see my NUC comments below) that there were indeed very competent and professional organizations in the labs around San Diego and presumably around the nation.
After graduation and four years working in the private sector commercial world, I found myself once more working at a (different) Navy lab in San Diego. The difference I experienced from my earlier experience was remarkable and much of what I learned from this group stayed with me thorough out my career.
- The group I was working with was at the Naval Undersea Center (NUC). The group was made up almost exclusively of Civil Service workers with a few of us contractor hangers on. They were developing the prototype of what was to become the Advanced Lightweight Torpedo. I can say many years later that I have never worked for a group as competent and professional as this group of federal workers. They developed computer hardware and software that actually ran inside of a torpedo and controlled its search and attack missions; further, they developed off-line software to aide in analysis and simulation of the torpedoes missions. I felt privileged and honored to have been a small part of this program.
My next job was as a software developer for the Top Gun training system with Cubic Defense Systems. Most of the next 13 years I was engaged in software development with minimal actual contact with civil service. However, there was one episode worth mentioning as follows:
- In the mid to late 1980s our system had evolved to the point where the computers of one of the main subsystems cost in excess of 1.2 million dollars. In the late 1980s, the computer vender we were using announced a new system which would increase the compute power six fold at a cost of approximately $750,000 as I recall. We were, at that point, under contract for six systems, and I convinced my management that we should offer to build the six systems using the new technology. We offered this to the government program office, and they rejected our proposal on the grounds that we would have to develop a new baseline. Mind you, we as the “greedy” contractor were willing to take less profit in this move but were rejected. The initial cost savings would have been enormous, and as it turns out, subsequent systems were in fact soon thereafter built using the new technology. Had the government program office been more diligent in analyzing our proposal, much tax payer savings would have accrued (more on this later).
In 1990 I started work as a civilian support contractor at the Naval Weapons Center in China Lake, CA, one of the premier defense labs in the nation. There I continued developing software at the support facility for the Top Gun system. For the next ten years I worked shoulder to shoulder on a daily basis with civil service counterparts. It was during this period that I came to see both the best and worst of government controlled system development. The following are some of the abuses I witnessed and was reluctantly a part of:
- The major part of our task at China Lake was to provide software updates to all of the Navy/Air Force ranges deployed around the country, and as the decade of the 90s progressed we found ourselves supporting four distinct flavors of systems; the old technology systems I talked about previously, as well as the newer technology systems that had come on line. Further, each service required unique tweaks which gave us in essence four separate baselines to maintain; old and new technology for each of the two services, with each tracking and processing the exact same set of combat aircraft. So the decision earlier to not capitalize on new technology exploded the maintenance costs for software upgrades across the four baselines.
- In the late 1990s the government decided to replace the Top Gun system with an entirely new system, complete with a brand new contractor. The new system was a direct threat to the existence of the lab in which I worked, so the Civil Service bosses decided to carve out a piece of the new system for itself. This carved out piece was entirely unnecessary, and very costly and ultimately failed, as did the replacement system by the way. I wrote about this fiasco several years ago here, and will not elaborate further here.
- During my tenure at China Lake I also witnessed for the first time in my career the phenomena known as “use it or lose it.” This is an exercise engaged in towards the end of each fiscal year whereby government entities scurry around to make sure they spend all of the money for the current year so as not to lose any funding for the upcoming year. This is the time of year where you can see truck loads of computers unloaded whether or not they are needed. One year at China Lake about a dozen new Dell computers were purchased and sat unused in a trailer, and as far as I know may still be there. I saw the same thing at the Marine Corp Air Station at Yuma AZ.
I left China Lake shortly after the failure of the carved out piece I talked of earlier, and returned to San Diego to work once more for Cubic, my former employer. I came back to Cubic to work on a software package related to the Top Gun system, but was unsure of the long term prospects for continued work on the system.
As it turned out, the replacement system I talked about earlier failed, and Cubic was selected to build a replacement system, which by the way was a success. But this was not the end of the abuses of the China Lake lab which was assigned to be the technical monitor over the new system.
- The first abuse was once again the “carving out” of a substantial piece of the new system for development at China Lake. Mind you, this carved out piece had been a successfully integrated part of our baseline system for close to 20 years, since the mid 1980s, but China Lake set off to develop an entirely new computer system, complete with new hardware, an unnecessary new interface to the main (our) system, and a new set of operator interfaces. Program meetings with up to 38 people having various engineering titles failed to point out the unnecessary complication and expense this brought to the program, and once again I was the lone wolf crying foul.
- On top of this added system complication, it was determined that the basic and very efficient inter-system interfaces that had been in place since the early 1970s needed to be “upgraded”, and TENA was added to the mess. TENA added new computers, complex and inefficient interfaces, software and user commands to accomplish interfaces that had been running successfully in their basic form since the early 1970s. Adding TENA to interface computers that were a mere cable length away from one another was akin to an English language conversation across a kitchen table, but with two additional participants involved; one to translate English to Swahili, and another to translate the Swahili back to English.
What is especially galling to me is that the efficiencies and solid system designs of past legacy systems were being undone in the name of modernization. Ironically, the massive enhancements in computer technology allowed this to happen as opposed to the old days when there were real and severe constraints as to what could be accomplished.
My experiences may very well be unique, but my strong sense is that they are common throughout most if not all government agencies and organizations. But human nature being what it is, similar practices go unreported in the spirit of “going along to get along.”
The time for a sweeping reform of Civil Service is long overdue, and it must begin with the federal employees themselves, especially middle managers. These folks need to suck it up and come to grips with what their very name “civil servant” implies; service to the public before service to themselves or even their own organizations. Much waste and unnecessary spending could be curtailed in short order if more at every level would step up to this challenge.
I’ve had some thoughts on reform at the level in which I was involved for many years, and presented them to a major software conference a few years back, and lately have been thinking of some other reforms that I may try to present as an article to technical journals.
Thanks for listening, and yes I am aware of and appreciate those dedicated Civil Servants who take their servant status seriously.
Don Johnson – February 2013