USC Annenberg Online Journalism ReviewUSC

Flinging Open the Doors to
Intelligence Gathering


Much of the information governments use to shape policy is available via non-secret channels, observers say.  Interest in "open source intelligence" is growing, and journalists play a key role.

When the 9/11 Commission floated the idea of creating a new government intelligence agency in its 567-page official report, it went about it in a mighty quiet way. The pitch for a new "Open-Source Agency" didn't appear with persuasive argumentation in the report's Executive Summary, nor was it mentioned in the body of the report itself, not even as a footnote. Its only presence took the form of a brief mention inside the organizational chart on page 413.

What is "open-source intelligence" [1] -- OSINT in the military and intelligence worlds? Why might the US need a new agency to obtain and analyze it, and why should journalists take any special interest in it, particularly at this time?

OSINT is intelligence gained from open -- unrestricted, non-secret --sources, and it's one of the key forms of intelligence, alongside human-source intelligence (HUMINT), signals intelligence (SIGINT), imagery intelligence (IMINT), among others. It is openly available intelligence, and its sources include all manners of journalism, whether broadcast, printed or blogged.

In that sense, journalists should take note of the growing interest in this form of intelligence because it's on the menu, so to speak. It is also invited to the meal.

OSINT is unclassified information, and by the same token it can be collected, analyzed and distributed by people who don't have clearances. The media are already functioning in a parallel role, and the intelligence community is coming to terms with that fact. Indeed, as the doors open to wider participation in the intelligence gathering process, open-source intelligence has the potential to gather a cult following or become a "buzz" term, because it's inherently interesting and innovative.

The intelligence world lives and breathes by secrecy, perhaps more so than is necessary. Knowledgeable observers suggest that as much as 80 percent of the intelligence required to support informed policy-making is available via open-source channels, and that the habits of secrecy, bureaucracy and "stove-piping" are all that stand between us and greater access to those sources. In a world that's shifting in a thousand ways from the hierarchical to the distributed, from the top-down to the bottom-up, and -- gasp -- from the authoritative to the democratic, a more transparent approach to intelligence may be the wave of the future, and open-source intelligence may be the key to that effort.

There are a variety of reasons why journalists might wish to take a greater than usual interest in OSINT at this time: they may, as they change emphases within the profession, find themselves on the road to becoming OSINT gatherers and analysts; they may find an OSINT perspective gives them a new angle on their craft and new tools to go with it; their own writings are in many cases part of the fodder sought by existing and future purveyors of OSINT; and there appears to be a OSINT "story" -- about the growing initiative to make OSINT a bigger part of the intelligence picture -- in the wind.

Michael Wilson of Decision Support Systems, Inc. is one of the more original minds at work in this area [disclaimer: I studied OSINT online and in seminars with Wilson, and he's a friend]. In his view, open source intelligence is important not just because of what it tells us, but because it breaks us out of our fixed mental routines. "Attempts to cross specialization boundaries perturb the intelligence community, because they are completely unprepared to make such structural shifts," he wrote in his Web site's primer "Toward an Ontology of Integrated Intelligence and Conflict." "This is one of the critical pressures driving military intelligence toward OSINT--since the intelligence is open source, it isn't restricted by specialist or compartmentalization boundaries."

This should be an area where journalists and writers have an edge over intelligence community analysts and academics. Cross-disciplinary thinking tends to be associative and analogical rather than linear and literal, and while the intelligence services are criticized for groupthink and academia for its disciplinary blinders, journalists and writers are better prepared to see the relevant analogy and track it down wherever the chase may lead them.

Open-source intelligence is thus a story, an invitation and a threat -- or a promise. It's a threat of sorts to those journalists who might not welcome intelligence folk reading their articles with more than the usual care and indeed data-mining them with powerful new technologies. But it's a promise of relevance to those who would welcome their analytic skills and research receiving more attention in policy-making circles.

Involving Journalists and Academics

That curious loose end in the 9/11 Commission's report drew remarkably little press attention, but it wasn't a mistake, it was a marker for a significant idea. Chairman Thomas Kean and Vice Chairman Lee Hamilton recently confirmed their interest in such an agency in their "Prepared Statement before the House Armed Services Committee: The 9/11 Commission's Findings and Recommendations" of August 10, in which they proposed:

A focus on open source information and the development of a new office or agency to collect and analyze solely open source information would also add to the competition of ideas on tough national security issues.

The Collins-Lieberman Bill, Senate bill S. 2845, has now been adopted by the Senate by a 96-2 vote, although the House has since adopted a very different bill. The two are now in committee awaiting reconciliation. Collins-Lieberman contains language in section 113(j) that takes this project a small step further:

The National Intelligence Director shall establish and maintain within the intelligence community an effective and efficient open-source information collection capability.

That in turn implies a different slant on information gathering, with academics and journalists among those who (a) are already doing some of this kind of work, and have skills and resources for it, (b) can likely benefit from some of the research methodologies established in the intelligence community, and (c) may be called on to participate.

If the formation of a new Open-Source Agency is a story in the making, it's also one that may take a while. Government is not always quick to create massive new bureaucracies.  The pressures to maintain the status quo may be considerable, and although popular demand to reform the intelligence community in the wake of 9/11 is also considerable, "open source" as an idea that has to be explained, perhaps even twice, may fall into the same category as "asymmetric warfare" and remain a mental plaything of policy wonks and spooks without ever quite gathering the popular support a simplistic sound-bite can achieve.

'A Bottom-Up Initiative'

Tracking developments in intelligence reform and paying particular attention to "open source" matters, Robert Steele of OSS.Net is pitching the idea that open-source intelligence should include what he terms "the seven tribes," national, military, law enforcement, business, academic, NGO-media -- that's where the journalists come in -- , and citizen-labor-religion. That looks a bit more like the proverbial ten tribes -- with some clumping -- than seven, but the point is that he's working towards Everyman as both a source and a consumer of intelligence.

Steele quotes Thomas Jefferson, "A Nation's best defense is an educated citizenry," and in line with his public diplomacy and global orientation, presented in detail in a series of books on intelligence reform [2], he'd eventually like to see open-source intelligence networks, of the people, by the people, for the people, globally, and says, "I see this happening as a bottom up initiative around the world."

The "anonymous" senior CIA analyst who has now published two books on Al-Qaida and bin Laden [3] claims that one of his purposes in publishing them was to allow his colleagues to take his research home over the weekend, and thus show how powerful open-source information -- even without all the other secret ingredients -- can be.

Whether as an idea whose time should have arrived by now, a CIA desk officer's proof-of-concept, a cautious phrase in a proposed bill, a new attitude to journalistic research, a new agency to set beside CIA and NSA in the alphabet soup, or a global vision of an informed and alert citizenry, open source in the intelligence, not the software sense, is surfacing. Like the internet and blogging, it may well change journalistic practice as it emerges.

We should keep it in our sights.


[1] It is inconvenient but important to note that open-source intelligence with its acronym has nothing much to do with "open source software," a similarly named but quite different affair which has given us such products as Linux and Apache. The common nomenclature can cause problems when one is using a search engine to locate items dealing with open source: is it the intelligence or the software kind of open source material you're looking for? Nor was it entirely helpful that Felix Stalder and Jesse Hirsh used the title "Open Source Intelligence" for their discussion of open source software on the First Monday Web site.

[2] Robert David Steele, On Intelligence: Spies and Secrecy in an Open World, (OSS International, 2001); The New Craft of Intelligence: Personal, Public, & Political--Citizen's Action Handbook (OSS Press 2002); publisher / contributor / editor, Peacekeeping Intelligence: Emerging Concepts for the Future (OSS International, 2003).

[3] Anonymous (Michael Scheuer), Through Our Enemies' Eyes (Washington DC, Brassey's 2002) and Imperial Hubris, (Washington DC, Brassey's 2004)



Related Links
10-14-04 OSINT
9/11 Commission
9/11 Commission: Executive Summary
9/11 Commission: Official Report
Decision Support Systems, Inc.
Decision Support Systems, Inc.: "Toward an Ontology of Integrated Intelligence and Conflict"
First Monday: "Open Source Intelligence"
Glass Beads and Complex Problems: Charles Cameron's blog on Open Source Intelligence
House Armed Services Committee: "Prepared Statement - The 9/11 Commission's Findings and Recommendations"
Library of Congress: Senate bill S.2845 - "National Intelligence Reform Act of 2004" "An Open Source Intelligence Agency?"
OSS.Net: Who Is Robert David Steele Vivas?
OSS.Net: Foreword to SOF OSINT Handbook
OSS.Net: Seven tribes of intelligence
Online Journalism Review: "Blogging as a Form of Journalism"
The Boston Phoenix: "The Secret History of Anonymous"
United States Intelligence Community: Six basic intelligence sources