April 28, 2009 | By Kurt Opsahl

EFF Issues Report on FBI Investigative Data Warehouse

Today EFF issued a report about the Investigative Data Warehouse, a gigantic billion-document storehouse of information maintained by the FBI. In addition, EFF wrote to Senate Judiciary Committee Chairman Patrick Leahy and House Judiciary Committee Chairman John Conyers, asking Congress to examine the IDW.

In August 2006, EFF sought documents about the IDW under the Freedom of Information Act. Three years of litigation later, the FBI has said that no more information will be forthcoming, despite the Obama Administration's new policies on open government. This report is based upon the documents provided in the litigation, as well as public information about the huge data warehouse.

The IDW contains at least 53 datasets, and includes more than four times as many unique documents as the Library of Congress. The report lists 38 of these datasets, which encompass not only information about suspects, but all individuals referenced in FBI investigations. In
addition, the report discusses the systems architecture and technical features of the IDW.

While the FBI has refused to publicly release any Privacy Impact Assessments about the IDW, the report discusses the FBI's efforts to avoid "raising congressional consciousness levels and expectations" about PIAs, and to give a “sense that we really do worry about the privacy interests of uninvolved people whose data we slurp up."

Finally, the report discusses the future of the IDW. Moving forward, the FBI has asked for millions of dollars to increase its use of the IDW for “link analysis” (looking for links between suspects and other people – i.e., the Kevin Bacon game) and to start “pattern analysis” (defining a “predictive pattern of behavior” and searching for that pattern in the IDW’s datasets before any criminal offense is committed – i.e., pre-crime).


Deeplinks Topics

Stay in Touch

NSA Spying

EFF is leading the fight against the NSA's illegal mass surveillance program. Learn more about what the program is, how it works, and what you can do.

Follow EFF

The Senate is reconvening at 10:30 am EDT tomorrow and is expected to consider USA Freedom. Stay tuned for updates. https://eff.org/r.bmpy

Jun 1 @ 5:16pm

The TPP Fast Track bill could go to the House floor as early as next week. Help us keep the pressure on our reps: https://eff.org/r.aku3

Jun 1 @ 3:53pm

Right now @SenBlumenthal on the Sen floor defending reform to secretive intelligence court. Reform is sorely needed: https://eff.org/r.i0rr

Jun 1 @ 2:25pm
JavaScript license information