BASIC RESEARCH REPORT

Y2K and Nuclear Arsenals:
A Final Report

(Part 2)

The Status of US Y2K Remediation Efforts

When BASIC originally reported on US Department of Defense (DoD) remediation efforts in November 1998, the verdict was largely pessimistic. The Executive Summary of Bug in the Bomb: The Impact of the Year 2000 Problem on Nuclear Weapons stated:

There are severe and recurring problems across the entire DoD Y2K remediation program, including ill-defined concepts and operating procedures, ad-hoc funding and imprecise estimates for final costs, lax management, insufficient standards for declaring systems ‘Y2K compliant,’ insufficient contingency planning in case of Y2K-related failures, and poor inter-departmental communications.¹³

Although the DoD and government audit agencies are still in the process of evaluating the final results of Y2K remediation programs, the upper management of Y2K-related programs has improved dramatically in the period November 1998-July 1999. This improvement is largely thanks to persistent audits by the General Accounting Office (GAO), pressure from the Office of Secretary of Defense (OSD) and the Joint Staff, criticism by Senate and House committees, and (perhaps most importantly) requirements written into both the FY 1999 and FY 2000 Defense Appropriations Bills.

Because of these measures, top-down oversight by service-level and component-level commanders increased, and memoranda on "Y2K Guidance" proliferated within the DoD. For instance, one of the persistent problems in 1997-98 was a lack of agreement on the meaning of the term "mission critical," which was being used to focus attention on those systems that could undermine operations if not made Y2K-compliant by year’s end. There were large disparities in reporting criteria used by different agencies in tallying their mission critical systems. As a result, trivialities such as public relations video conferencing equipment were being included alongside nuclear C4I systems in the Joint Staff Y2K lists. In turn, this was due indirectly to the lack of command directives identifying those organizational components that perform truly critical military missions, so that departments not related to fulfilling battlefield requirements were getting the same level of attention as top command posts in meeting Y2K objectives. In general, there was a lack of DoD-wide coordination in the construction of mission-critical systems databases, so that discrepancies existed between the still-developing Joint Staff lists of mission critical systems and the databases kept by officers within the individual services.

To bolster the accurate accounting of system names and system status, the Chief of Naval Operations (among others) mandated that all mission critical systems identified by the Joint Chiefs of Staff without a corresponding Service-level "Y2K tracking systems ID number" be assigned a number without delay. He noted, "All systems designated by unified CINCS for Y2K operational evaluations, if not already classified as mission critical in the Navy Y2K tracking systems, will be reclassified [as mission critical]."¹⁴

This same Memorandum also defined the legal Y2K validation requirements set down by Congress and mandated that each Navy Component dutifully supply systems for testing in DoD-wide Operational Evaluations (OpEvals). The CNO specified:

It is now the service responsibility... to ensure that each system appearing on the JCS [Joint Chiefs of Staff] list has at least one additional validation – CINC OpEval, OSD, or Service end-to-end testing- scheduled in order to meet the congressional requirement... To that end, the following actions are required: for each system, provide two event names and dates for when they are scheduled to be operationally validated; confirm system Y2K fixes are/will be installed in time to support CINC OpEvals; ensure appropriate technical support (procedures, equipment, and personnel) is arranged.

The CNO ended the message by noting that "Y2K validation of Navy systems in an operational environment is an essential element of the Y2K remediation process… Validation is not only critical, but is mandated by [the FY 1999 Defense Appropriations Bill]."¹⁵ Similar mandates have been distributed throughout every department and every agency of the DoD.

These trends finally resulted in a more reliable mission-critical Y2K list in late spring 1999. In the May 1999 DoD Quarterly Report to the President’s Office of Management and Budget (OMB), the Office of Secretary of Defense (OSD) stated that "DoD has made a concerted effort to refine the number of mission critical systems to a more consistent and manageable level. The total number of mission critical systems has stabilized since November 1998, each mission critical system is tracked by name, and the Assistant Secretary of Defense for C4I specified that any further change will require his approval."¹⁶

These developments were dramatically reflected in the ever-changing number of mission-critical systems from late 1997 to May 1999. Between November 1997 and May 1998, the number decreased from 3,143 to 2,803, then rose again to 3,135 by August 1998. Within just another three months, however, the list suddenly dropped to 2,642 systems, then experienced a further decline to 2,387 systems by February 1999 before stabilizing at 2,283 in May. Particularly notable is the roughly 33% decrease in the list between August 1998 and May 1999, representing a cut of almost 1000 systems from the central DoD database over a nine month period. According to the May 1999 Quarterly Report, most of this rather precipitous decline was "due to the Army’s correct removal of a number of small, tactical systems from the mission critical list" between August and November 1998.¹⁷

In addition to these Service-level improvements, six new management positions were added to the office of Y2K director William Curtis, who is the supervisor of the overall DoD process. These "temporary senior executive servants" have increased inter-service coordination by providing advice on such subjects as contingency planning, Y2K testing, and financial management reporting. There is also an information center for distribution of key technical facts on the repair and testing of individual systems. According to the Defense Information and Electronics Report:

The information center will be a Year 2000 "situation cell" that will rely heavily on the Defense Information Systems Agency [DISA]... which already provides technical support 24 hours a day, seven days a week to DoD customers. It is envisioned to be a "triage" center for DoD’s Y2K related problems and is being compared to a tactical operations center during a battle.¹⁸

Because of these positive trends, Secretary of Defense William Cohen ultimately did not find it necessary to carry out his threats to freeze program development on all information technology systems.¹⁹

The separate military services are now following the example of the OSD "situation cell" by opening up their own information and technical coordination centers. For instance, the Air Force has constructed a central point for collection, consolidation and reporting of the inevitable Y2K problems that will arise and for resolving automated information systems errors. A cross-functional team of experts is consolidating and networking numerous uncoordinated help desks throughout the service. The Fusion Center is being equipped with a variety of specialized and secure communications systems that include internet networks, secure faxes, and video teleconferencing. According to one spokesperson, "We have people working now with all other appropriate agencies so we'll know exactly who is responsible for what. So if a customer has a problem and doesn't know who the point of contact is, they can call the Fusion Center and get the name and number of the agency that can help."²⁰ The fusion desk also draws upon the expertise of logistics and maintenance centers within the AF, such as the AF Communications Agency, the Electronic Systems Center, the Aeronautical Systems Center, and others.

Finally, the practice of having at least one major "Y2K Steering Committee Meeting" every month within the OSD has continued. Each of these meetings has been grouped around a major functional area of operations, with participation by the responsible services, agencies, and John Koskinen, the President’s Y2K advisor. For instance, the April 13, 1999 meeting concentrated on civil engineering installations, while the May 1999 meeting addressed the Y2K readiness of private and DoD-owned information technology (IT) infrastructure, including telecommunications and internet networks.²¹

However, despite fairly rapid improvement across the DoD as a whole, reporting requirements for nuclear operations remain ambiguous from the standpoint of effective Presidential oversight of ground-level details. Because of narrowly-defined reporting standards instituted by the Office of Management and Budget, the DoD’s quarterly Y2K reports to the OMB do not give data in a truly useful format. Major systems integral to nuclear operations are not systematically identified and grouped by either functional area or their contributions to larger missions. For instance, the early warning functions of NORAD systems are not represented in one specially-delineated grouping of systems, and neither are systems contributing to the larger mission of nuclear command, control, and communications. Instead, nuclear systems are scattered throughout appendices that include numerous conventional systems, or they are not listed at all in the DoD Quarterly Reports. Moreover, contingency plans are simply identified by a "Y" for "yes, a contingency plan exists" or "N" for "no, a contingency plan does not exist." The actual contents of such plans are not reported. Finally, contingency plans are specified only for individual systems standing alone, rather than for wider functional areas and mission-level requirements that inherently involve the connection of systems to each other for integrated performance of multiple tasks. In the absence of a more representative reporting system, the monthly Y2K Steering Committee meetings have remained the principal tool for OMB and Presidential Council oversight.

Fortunately, the Council is indirectly aided in its task by both the General Accounting Office (GAO) and the DoD Inspector General’s Office (DoD IG), the latter of which had at least 60 ongoing Y2K audits in late spring 1999.²² Most of these studies are centered on the Y2K testing or "validation" stage, which aids the Presidential Council in judging the veracity of DoD testing methods. Especially helpful is an ongoing audit of nuclear operations started in April 1999 by GAO’s National Security and International Affairs Division (NSIAD). It is unclear whether any of the DoD IG audits are also geared towards nuclear-related systems.

For nuclear operations, most mission critical US systems have now been "renovated" and the Pentagon has completed the testing or "validation" stage of the Y2K remediation process through "sensor to shooter" nuclear alert simulations. This stage was the most time consuming, due to the difficult logistics of "mission level" simulations that required the stringing-together of all relevant interfaces involving the exchange of data.

Integrated tests for early warning systems were performed in both early December 1998 and late February 1999, involving computer systems under three separate Commands: NORAD, Space Command (SPACECOM), and STRATCOM. These first two simulations included at least 30 separate attack scenarios for each of five critical Y2K-related dates, incorporating both single ICBM launches and an all-out first strike by the opponent. Besides the Millennial rollover date (December 31, 1999-January 1, 2000), dates associated with the leap year were included. The inclusion of February 29, 2000, and the 366^th day of 2000 in these tests was an important step, because the Year 2000 is an unusual leap year that was not accounted for in many of the "legacy" systems and software code produced over 10 years ago. (The rule of thumb is that millennial years are only Leap Years if they can be divided by 400.) No "hard failures" were reported for the mission of "Integrated Tactical Warning and Attack Assessment [ITW/AA]." ²³Table 1 lists some of the primary systems involved in these mission-level exercises and denotes each system’s general function in the nuclear C4I process.

In addition to early warning sensors, these first two Y2K OpEvals incorporated 25 million lines of computer code from multiple STRATCOM systems at Offutt AFB, Nebraska, which are collectively known as the "Computing Environment STRATCOM Architecture (CESAR)." Also incorporated was the recently renovated "Cheyenne Mountain Upgrade" (CMU), encompassing five critical and relatively new NORAD computer systems for data reception, data processing, user interface, and communications.²⁴ The November 1999 Y2K repair completion for CMU at NORAD is a welcome surprise, given the persistent and severe operational deficiencies that the CMU has already experienced in a non-Y2K environment. The entire upgrade program was plagued with problems from its inception in 1981, became fully operational only in 1998, and finally ended 11-12 years overdue and billions of dollars over budget. In fact, it was not until the period 1995-1998 that successive simulations demonstrated adequate operational capabilities in a non-Y2K environment.²⁵ It is likely that Y2K repairs were instituted simultaneously with ongoing software patches for defective 1980s modernization programs, thereby allowing the CMU to take part in the first two Y2K OpEvals.

Table 1. The Y2K Program Status of Airforce and Central Command Systems Utilized in Nuclear Operations ^a

^a Unless otherwise noted, all information is taken from Appendix G, 7th Quarterly Report to OMB, 17 November 1998.

^b This information is not specifically listed in any of the Quarterly Reports. Evidence is taken from interviews between Brigadier General Robert Behler, head of C4I for STRATCOM and Joint Staff, and John Donnelly, editor of Defense Week, issues vol. 19 no. 49, 14 December 1998, pp. 1 and 7; and 4 January 1999, pp. 5-8.

^c Office of Secretary of Defense , 9th Quarterly Report to OMB, Appendix B, 14 May 1999.

In April and May 1999, Strategic Command also completed integrated systems tests for the Command and Control of deployed forces, dubbed by one officer as the "execution" and "post-execution" phases (i.e., those phases in operations that involve the actual implementation of orders and fighting in a nuclear war).²⁶ These last phases of the nuclear Y2K OpEval program involved the issuing of launch orders through private and DoD-owned telecommunications lines to the "owners" of the deployed forces, USSPACECOM for ICBMs and the Navy for Trident submarines. They also reached down into the silos and submarines themselves, including simulated launch activity. Test results were apparently positive, although they have yet to be fully publicized.

With these positive results, the main concern from a systems-of-systems standpoint is the criteria used by US Commanders-in-Chief (CINCS) for inclusion of specific systems in the full-scale OpEvals. The methodology for identification of critical systems is still an open question because the five-phase nuclear tests were originally planned in 1998 without oversight by either the President or Congressional agencies such as the GAO. Also, the huge decrease in the overall DoD Y2K systems database since August 1998 (see above) raises questions about the validity of the concept "mission critical." Was the 33% decrease in the DoD-wide Y2K list reflective of a true redefinition of the concept from the standpoints of efficiency and effectiveness, or was it rather a reflection of the need for top commanders to report impressive statistics to meet the demands of Secretary Cohen and Congress?

For instance, Deputy Secretary of Defense John Hamre and other OSD officials have repeatedly touted the success of the Y2K remediation effort in the media and in Congressional hearings by citing macro-level statistics, such as 81-99% validation rates of Y2K compliance across the entire DoD. These impressive DoD-wide figures came into common usage at about the same time that the Y2K database decreased by over 700 systems between August and November 1998, followed by a cut of another 359 systems between December 1998 and May 1999. The May 1999 DoD Quarterly Report presents a pie chart with 88% of systems Y2K-compliant and 12% still in the "renovation" or repair stage of the Y2K process, while statements by the OSD from July 23, 1999 put the compliance figure at 92%.²⁷

In this regard, one NORAD official said in a Senate hearing on February 24, 1999, that the ongoing tests of nuclear operations included "the thin line, the minimal number of [computer] systems required to execute the mission."²⁸ Similarly, one May 1999 Memorandum from the Chief of Naval Operations for Navy-wide circulation specified that "Each unified CINC has coordinated with the component commanders and produced functional thin-lines based on specific systems. The Joint Staff has reduced each CINC’s thin line to a list of systems and has consolidated this list in a spreadsheet."²⁹

Given this reliance on a "thin line" of systems, rather than all systems utilized in daily operations, there is some concern that completed and planned operational evaluations are not specifying the entire network of inter-system dependencies, some of which may involve "non-mission critical" components. Examples include telecommunications switching nodes, heating and ventilation systems, and internal security access systems, all of which could disrupt normal operations in unpredictable ways. The non-profit research group Business Executives for National Security (BENS) argued in February 1999 that the DoD was not consistently utilizing developing methodologies for "network analysis" that would identify these types of dependencies, a practice that could undermine existing contingency plans for nuclear operations. According to BENS, "Without dependency modeling, the map of potential failures is incomplete, and attempts to wargame Y2K problems will not capture the depth or breadth of the situation.³⁰

However, despite these continuing doubts by outside observers, there have been several positive reports about dependency-based infrastructure testing at selected bases. According to one recent Air Force article, "All of our installation commanders have certified their bases are Y2K-ready from the perspective of utilities, fire safety and security... They are also testing their continuity of operations plans [contingency planning]."³¹ Additionally, testing at Keesler Air Force Base in Missouri (not involved in nuclear operations) showed positive signs:

[There was a] wall-to-wall, base-wide test that looked at the Y2K performance of infrastructure items; communications networks; and logistics, personnel, medical, finance, civil engineering and security systems. During the test, more than 7,500 clocks on dozens of automated systems and infrastructure items were rolled forward through four crucial dates: Sept. 8 to 9, 1999; Sept. 30 to Oct. 1, 1999; Dec. 31 to Jan. 1, 2000; and Feb. 28 to 29, 2000. Only a few anomalies occurred, notably some involving Y2K-certified, commercial off-the-shelf software. None of the anomalies produced mission-stopping effects.³²

It is probable that these same exercises are being instituted, or have been completed, for all bases involved in nuclear operations.

To summarize, the "big picture" for US nuclear operations is generally quite positive, despite an extremely rocky start. However, ambiguities in the status of specific nuclear C4I systems remain for the Air Force, Navy, and DoD. Also, the US arsenal is only one side of the bilateral nuclear picture, and the situation in Russia is not nearly as comforting. The following sections will tackle each of these subjects.

Part 3: The Status of US Y2K Remediation Efforts:  Air Force Systems for Early Warning and Command of Forces

.
Back to Nuclear and WMD home page