Improved Metrics Should be Primary Goal of FISMA Reform

Cybersecurity policy generally focuses on one of three areas: 1) federal agencies, 2) critical infrastructure (which sometimes overlaps with #1), or 3) “everything else.” While much of the debate about cybersecurity legislation in Congress has been about the latter two, reforming the security policies and practices of federal agency is important as well. The Federal Information Security Management Act (FISMA) is the primary policy that specifies the security requirements for information systems managed by federal agencies. This year will mark the 10-year anniversary of FISMA which was signed into law as part of the E-Government Act of 2002. As we approach this milestone, it seems clear that agencies are better off today than they were 10 years ago, but more progress is needed. In particular, FISMA should be improved so that agencies report on security performance, not just security compliance. The purpose of FISMA was to institutionalize the information security programs that agencies had begun to develop as part of the Government Information Security Reform Act (GISRA). Under GISRA (and later FISMA) agencies were required to develop a comprehensive security plan for their IT systems. This included creating a risk-based, cost-effective security plan, performing periodic risk assessments, providing information security awareness training to staff, and requiring an agency head to authorize the security program. In theory, FISMA provided a roadmap for agencies to use to develop a strong cybersecurity program. In practice, agencies often merely focused on achieving compliance rather than effectiveness.

Under FISMA, the National Institute of Standards and Technology (NIST) developed a set of security best-practices. This allowed agencies to be evaluated against a common set of metrics. A decade ago this type of rigid framework was probably a necessary evil. The U.S. government had a poor track record of securing its IT systems and managing risk. Security issues neither received the attention they deserved nor were integrated into the lifecycle of IT systems. FISMA provided a top-down set of requirements to ensure a baseline set of security standards and practices, and while similar requirements would have been overly burdensome if applied to the private sector, they probably struck the right balance for federal agencies.

Federal agencies likely needed this push to implement a robust security program. Periodic evaluations through FISMA audits help identify problem areas. Audits reveal whether or not various controls, such as an incident response program, have been implemented. Negative findings receive Congressional scrutiny and force agencies to address issues such as personnel problems or lack of funding. Unfortunately this encourages a “teach to the test” mentality which at best promotes inefficiency and at worst results in poor security. Under FISMA, information security professionals at federal agencies are incentivized to become compliance-oriented rather than results-oriented. Agencies are evaluated on whether they have particular controls in place, not on whether these controls are working effectively. Not surprisingly this focus has caused many critics to see FISMA as nothing more than an exercise in government paperwork (somewhat ironically since GISRA was an amendment to the Paperwork Reduction Act).

In addition, compliance audits give agencies little incentive to take risks and obtain a better return on investment for their security investments. Right now agencies are only motivated to be as good as everyone else, which only encourages a march to the middle. This means that over time agencies will tend to become reactive, rather than proactive, with regards to security issues and will likely lead to a conservative approach to adopting IT innovations within government agencies. In a nutshell, when the penalties for taking risks vastly outweigh the benefits, we can expect government agencies to settle for mediocrity.

So how can the federal government improve FISMA?

First, instead of treating security as something that is reviewed “once every audit,” it should be constantly monitored by IT managers. Indeed this idea is found in the proposed legislative reforms to FISMA, such as H.R. 4257, the Federal Information Security Amendment Act.  Building in real-time risk management capabilities will allow agencies to strengthen their security programs and gain better situational awareness of the threats to their organizations. These data feeds will also help the Department of Homeland Security, and other intelligence agencies, better understand the computer-based threats facing the federal government. Automated, continuous reporting will also lessen the audit burden on IT security staff.

Second, it is important that FISMA reporting requirements not only reveal information about the presence or absence of security controls, but also provide real-time information about the overall effectiveness of these controls (To be fair, agencies began to report on outcomes for FY2010, although not nearly enough). For example, agencies currently report on the percent of staff that complete security awareness training. While useful, this does not give any indication of how well the lessons from this training were learned or the impact this had on security. To address this problem, as part of FISMA reform, Congress should task NIST with developing a comprehensive set of performance-based security metrics that agencies would be required to report on. This could include data such as length of time to deploy security patches, response times to security incidents, and the amount of time needed to detect intrusions. Better analytics of security operations will also eventually allow agencies to compare performance based on more meaningful metrics such as response time to security incidents. This, in turn, can be used to create a positive feedback loop for agencies to reward innovative ideas.

Third, it is important that agency staff support the results of FISMA audits. The perception that FISMA is a waste of time may itself encourage more waste. For example, an agency may spend hundreds of thousands of dollars developing a detailed security plan because FISMA requires it, but then never follow it (or intend to follow it) because managers believe FISMA is merely a regulatory requirement taking them away from their real work. Creating performance-based metrics rather than compliance metrics may help change this perception. However, Congress should go a step further and make annual FISMA audits a two-way evaluation. The security of agencies should be evaluated and reported using FISMA performance criteria, and IT security managers at federal agencies should evaluate the reliability and validity of the performance metrics that NIST develops. Using feedback from those on the front-lines of security at federal agencies will help ensure that achieving FISMA compliance is more than a paper-pushing exercise.

Fortunately, there is broad agreement on the need to reform FISMA, and policy makers rightfully have cybersecurity high on their agenda. While the federal agency cybersecurity outlook is probably much better than it was a decade ago, it is still important for Congress to take this opportunity to implement these needed reforms.

Image credit: Flickr User rybolov