The Wall Street Journal today published (alternate link) an in-depth review of a new, relatively unknown program run by the National Counterterrorism Center (NCTC). Although we have been warning about the dangers of the program for months, and I testified before Congress about the issue in July, the Journal’s story conveys how controversial the program was even inside the government. It also describes the broad scope of new authority the government is granting itself.

As the Journal reports, under new guidelines issued by the Attorney General back in March,

The rules now allow the little-known National Counterterrorism Center to examine the government files of U.S. citizens for possible criminal behavior, even if there is no reason to suspect them. That is a departure from past practice, which barred the agency from storing information about ordinary Americans unless a person was a terror suspect or related to an investigation.

Now, NCTC can copy entire government databases—flight records, casino-employee lists, the names of Americans hosting foreign-exchange students and many others. The agency has new authority to keep data about innocent U.S. citizens for up to five years, and to analyze it for suspicious patterns of behavior. Previously, both were prohibited.

The changes also allow databases of U.S. civilian information to be given to foreign governments for analysis of their own. In effect, U.S. and foreign governments would be using the information to look for clues that people might commit future crimes.

The program is striking in so many ways. Innocent people can be investigated and their data kept for years. It can be shared with foreign governments. All of this in service of not just terrorism investigations but also investigations offuture crimes. In effect, the U.S. government is using information it gathers for its ordinary business to turn its own citizens into the subjects of terrorism investigations.

Meanwhile, all of this is supposed to be against the law. The Privacy Act of 1974 says that information collected by the federal government for one purpose is not supposed to be used for another. However, agencies are attempting to circumvent these rules by publishing boilerplate notices in the Federal Register. Sadly, that practice has become far too common.

Government officials who have a firsthand look at how the program works are stunned by it:

“It’s breathtaking” in its scope, said a former senior administration official familiar with the White House debate.

And from Mary Ellen Callahan, then the Chief Privacy Officer at the Department of Homeland Security:

the rules would constitute a “sea change” because, whenever citizens interact with the government, the first question asked will be, are they a terrorist?

Worse, all of this happened in secret, approved by National Security Advisor John Brennan and signed off on by Attorney General Eric Holder. No public debate or comment and suddenly, every citizen can be put under the terrorism microscope.

Ironically, all of these changes to the rules came in response to an attempted attack that had nothing to do with information collection or a U.S. citizen. Thegovernment cites the attempted 2009 Christmas bombing by Umar Farouk Abdulmutallab as the impetus for the changes. However, as the Journal story makes clear, Abdulmutallab wasn’t a U.S. citizen, and collecting information on him wasn’t a problem. Instead, his own father had identified him to the U.S. government as a potential terrorist. In short, an attack by a known foreign terror suspect was used to justify changes to rules about collecting information on U.S. citizens.

Finally, credit must be given to those who fought the program. It’s clear that DHS, especially the Privacy Officer, Mary Ellen Callahan, and the Office of Civil Rights and Civil Liberties pushed back hard against this. Nancy Libin, the chief privacy officer at the Department of Justice, also expressed serious reservations and fought an internal battle against the changes. It’s probably not a surprise that none of them are still in government.

If you want to learn more here is a simple guide to the main changes created by the 2012 NCTC guidelines. And here are the Freedom of Information Act documents that we have gathered on NCTC—we will post more as we receive additional records.