Community Open Source Program Office:
The Open Source Information System
James Lucas, Deputy Director

This is not OSIS .  This is an detailed article about OSIS.  We still need to work out direct access, currently available in INR only.   What is Open Source? 

- Publicly available information (i.e.., any member of the public can :lawfully obtain the information request or observation), as well: as other unclassified information that has limited public distribution or access  

- Includes information that may be used in an unclassified context without compromising national security: or intelligence sources and methods. If the information is not publicly available, certain legal requirements relating to collection, retention and dissemination may apply

Why Open Source ? Why Now ?

- Global Media Coverage in near-real-time
- Many things that used to be secret aren't
- Computers and networking technology have lowered the "entry cost of publication"
- Collection resources overall are diminishing
- Lack of "enemies" raises diplomatic cost of revealed clandestine collection

Characteristics of Open Source

Very large volumes of information
- tools that work with a small corpus may not "scale up"

Poor "signal-to-noise" ratio
- lots of low-grade ore must be mined to extract nuggets

Multi-media information
- tools and standards are embryonic

- Proliferation of Nation-States
- Technologies of low-power/cable & desktop publishing

Original (foreign) languages
- users' languages skills are uneven
- machine translation has not yet arrived
- handling non-Roman alphabets is hard

"Multi-level security"
- interacting with open sources is prohibited from classified computing space


- The U.S. Intelligence Community is a large, complex structure, organized and operated pursuant to a multitude of laws, executive order, policies, and directives.
- The essential role of intelligence is to provide timely, relevant information to U.S. policy-makers, decision-makers, and war-fighters

Origins Of COSPO 

1991 - DCI Open Source Task Force identified issues and made recommendations to enhance use of-open source information
1992 - Open Source Coordinator appointed and the Open Source Coordination Office created within the Community Management Staff
1994 - DCI Directive 2/12-1 established the Community Open Source Program Office (COSPO) within ClA's Directorate of Science and Technology


- Improve the acquisition, exploitation, and distribution of open source
- Provide timely access to pertinent open source in the users' work environment:
- Foster adoption of tools to increase open source usefulness and usability
- Make open source "the source of first resort" 



Director COSPO
Deputy Director

System Architecture

User Services

Resource Management

Resource Management Group

- Coordinate budget with Program Managers
- Assess past performance; execute current year budget
- Prepare IC annual and outyear program plans
- Conduct liaison with Congress

User Services Group 

- Assess customer needs
- Identify, acquire new sources of information
- Crosswalk to national process
- Produce collection capabilities statements
- Support to analysts, military, UN

Systems Architecture Group 

- Design and implement OSIS Architecture
- Evaluate and coordinate technical programs
- Identify, evaluate and track new technology
- Provide technical support to COSPO

Source Analysis Function

- Assess customer needs
- Identify relevant sources
- Establish Community interest/need
- Crosswalk to national needs process

National Process

National Policymakers => Issue Coordinators <= (Capability Statements | Needs) => INTs/COSPO Research Secretariat

Example Activities

Assess customer information needs
- Tier 3-4 Study
- Open Source Research Network
- Information Needs Committee
- Strategic Intelligence Reviews

Information Needs Initiatives 
- Collection Capabilities Statements
- Tier 3-4 Study
- Support to United Nations
- Support to Military Operations
- Military Reserves
- Capabilities & Coverage Database (CCDB)
- Information Needs Process

Identify, acquire new sources of information
- Library of Congress Country Studies
- Oxford Analytica Daily Briefs
- AP TV to Defense Intelligence Network
- British Library Document Supply Center (for gray literature acquisition)

Evaluation of resources
- surveys
- online questionnaires
- focus groups
- statistical analysis 

Liaison with:
- Crime & Narcotics Center O.S. WG
- IC Librarians' Committee
- Interagency Gray Literature WG
- Non-Proliferation Center Eval. Task Force
- International Aquisition Workshop
- IC Open Source Training Courses

International Access

- Foreign governments interested in links to OSIS: Australia - Germany - Norway- Canada - Israel - UK
- Steering Committee approved linkage with Canada, Australia, and UK
- International Working Group formed to discuss linkage 

Criteria for joining
1. Indemnify others and hold harmless from copyright infringement
2. Strong firewalls & user authentication for official use only
3. Make the Enterprise larger (i.e., Burden Sharing)
4. Certain holdings might be subject of bilateral negotiations


What is OSIS?

 - The Open Source Information System (OSIS) is a network of interconnected agencies that allows access to and sharing of UNCLASSIFIED US. Government information and other Open Source information.

OSIS Description
- Open Source Information System
- Internet architecture (WEB Based)
- Virtual private network
- Firewall protected
- Access to Internet resources

Advantages of OSIS
- Access to many U.S. Government databases quickly and easily
- Ability to retrieve information remotely
- Access to the Internet
- Opportunity to share technologies with other U.S. Government agencies

OSIS Architecture

Current OSIS Offerings
- Electronic Early Bird - AIA
- Library of Congress Country Studies - COSPO
- Brown Commission Report - COSPO
- GAO Report on Computer Hacking - COSPO
- Foreign Language Training Materials - CALL
- Oxford Analytica (DIA)
- ProQuest periodicals - DIA
- Jane's Electronic Library - DIA
- Landmines and Demining: A Global Problem (NGIC)
- Medical capabilities reports (AFMIC)
- Radiation Handbook (NGIC)
- Machine Translation (NAIC)
- General Intelligence Requirements Handbook (MCIA)
- FBIS Reports
- CIRC Database (NAIC)
- Controlled Image Base (DMA)
- Digital Terrain Elevation Data (DMA)

Comming Soon
- British Library Document Supply Center Access to conference proceedings (DTIC)
- Janes Products
- Infrastructure database


COSPO Vision Statement

Provide users access to all available information relevant to their jobs -- highly classified intelligence or open source -- and the means to exploit that information.

1995 Strategic Goals

1. Apply proven commercial and government technology
2. Provide an unclassified Community OS architecture
3. Determine and prioritize Community OS requirements
4. Take advantage of larger amounts of OS information
5. Ensure timely and easy use by the user and customer
6. Provide capabilities for discovery, retrieval, and exploitation
7. Ensure a protected operating user environment
8. Allow for OS information sharing outside the Community
9. Provide Community guidance for efficient use of resources
10. Preserve a sound base to meet the challenges of the future

1. Apply proven commercial and government technology

- Monitor technology developments in government and commercial world for COTS and GOTS solutions
- Restrict open source funds to integration of proven new enabling technologies, rather than in R&D for the technologies
- Encourage appropriate NFIP R&D investment through judicious application of Community resources in areas where the current pace or scope of development is not in line with Community open source needs
- Retain agility in adhering to commercial practices and adopting government standards for Community open source activity

- Deployed a COTS firewall and had it evaluated by NSA
- Web access to data declared preferred methodology
- Evaluated COTS switch and had it approved for CIA use
- Developed one way transfer prototype
- MITRE produced technology roadmap
- Evaluated and selected COTS encrypting router for OSIS
- Invested in CIRC move from mainframe to server-based technology.
- Investment in FBIS Bureau of the Future development
- Investment in CIA IC-ROSE development off shelf software.
- Hosted "Industry Day" to monitor commercial technologies, products and services

2. Provide an unclassified Community Os architecture

- Establish and expand the Open Source Information System (OSIS) as the backbone architecture for accessing, aquiring, processing, distributing, and manipulating open source information
- Establish, operate, and enhance system capabilities on nodes at Community OSIS sites
- Coordinate the development of OSIS and Intelink as compatible capabilities

- Access for: AFIWC, NIC, US Mission to UN, NJMIC, Balkans Task Force, Reserve Units, JWAC, INSCOM, Naval War College, etc.
- New communications architecture, increasing accessibility & reducing costs
- Established JAC node as European theater hub
- Provided Web access to nodes
- Designated a network management agent (NSA)
- Coordinated OSIS & INTELINK architecture models
- Provided technical assistance to node sites for development/deployment
- Revalidated resources for communications network with ANS
- Revalidated MITRE for systems engineering support contract

3. Determine and prioritize Community OS requirements

- Identify national policy maker, war-fighter, and Community user information needs that can be met by open source
- Maintain, aggregate, and update existing open source requirements in Community components
- Create am agile, interactive process for users and customers to specify information needs and requirements
- Develop an evaluation and appraisal system to determine how well open source information has satisfied needs.

- Funded NAIC IRMS system
- Produced over 20 open source capability statements on regions and topics of customer interest
- Sponsored Pacific Rim Open Source Conference, Crime and Narcotics Conference, International Acquisitions Conference, and NGO Conference
- Provided OS assessment as input to classified studies of capabilities and gaps
- Represented OS capabilities and participated in the 1995 DCI Intelligence Priorities Working Group.
- Studied OS requirements management in support of the National Needs Process and completed a concept paper for an Information Requirements Management System (IRMS).
- JAC Support and Evaluation

4. Take advantage of larger amounts of OS information

- Maintain a robust worldwide collection.
- Expand access to and acquisition of hard and soft copy media, including gray information, in response to user needs; emphasize soft copy
- Encourage innovative techniques to access and acquire information
- Leverage existing global communications to enhance access to and coverage of open source information
- Replace and upgrade outmoded access and collection equipment and procedures
- Access and acquire open source information in ways that optimize the value of expenditure of Community resources, broaden its availability throughout the Community, and ensure full compliance with copyright and other constraints
- Expand use of non-Community agents for accessing and collecting information to supplement or substitute for traditional Community efforts.

- Provided "secure" access to the Internet for data access
- Augmented FBIS maps and publications procurement
- Funded a Hardcopy Procurement Study & IC-ROSE data purchase
- Reached agreement with US military reserve units to provide OSIS connectivity for twenty-eight reserve intelligence training sites around the country
- Funded DTIC initiative to identify and access conference proceedings held by the British Library Document Supply Center and to catalogue gray literature held by the Library of Congress.
- Digitized over 60 Country Studies Handbooks produced under Library of Congress auspices
- In cooperation with DIA, acquired Oxford Analytica and Jane's Electronic Library for Community use on OSIS

5. Ensure timely and easy use by the user and customer

- Integrate proven technology in Community open source data transformation activities (conversion, translation, and formatting) in processing text, graphics, images, sound, and motion video in response to user requirements
- Encourage development of processing operations that work on foreign language as well as English-language materials
- Increase investment in both machine and human translation capabilities

- Provided access to NAIC machine translation tools
- Developed multimedia capabilities in response to CDA
- Provided demonstration CD-ROMs to Congress
- Demining CD/ROM distributed and used widely in Bosnia
- Assisted in the development of NAIC machine translation tools and provided OSIS access
- Studied open source availability for Latin America and sub-Saharan Africa
- Arranged to have language materials from the Center for the Advancement of Language Learning (CALL) and Defense Language Institute distributed via OSIS.

6. Provide capabilities for discovery, retrieval, and exploitation

- Provide capabilities which enable users to access, scan, browse, search, and retrieve from the expanding volume of open source information in multiple media and languages
- Develop means for users to control the flow of open source information to them
- Enable users to recall open source information from their own and remote archives
- Emphasize capabilities and techniques which increase precision and accuracy in retrieving open source information
- Insert enabling technologies for near-term improvements of exploitation at the workstation level
- Give users better knowledge of and the capability to apply exploitation tools developed outside their immediate environments

- Developed RFP for OSIS web based homepage, and web browsing capability
- Conducted open source training needs assessment and developed SOW for development of on-line open source training materials.
- Provided COSPO/OSIS input for recurring "Open Source Options" course sponsored by ClA's Collection Requirements and Evaluation Staff (CRES).
- Provided WAIS access to OSIS and Internet data
- Evaluated PATHFINDER tool for OSIS use
- Provided guidance for SENTINEL evaluation and integration into OSIS System engineering at MITRE

7. Ensure a protected operating user environment

- Support information architecture features that allow users from their work enviromment, whatever their respective security configurations are, to use open source information distributed throughout the Community and elsewhere
- Protect Community-held, networked, unclassified data holdings from inadvertent manipulation, deliberate intrusion from outside elements, or unauthorized use of intellectual property
-Adopt procedures to safeguard against the flow of classified information into the unclassified environment

- Modified S-KEY software throughout the network to meet NSA security criteria
- Implemented TESSERA technology for secure password dissemination
- RFP developed for security development through DISA contract
- Funded for NSA Springtide development

 8. Allow for OS information sharing outside the Community

- Support U. S. Government efforts to determine how to share data holdings while protecting intellectual property rights
- Rely on non-Community government intermediaries to make non-OUO, copyrighted information available to the private sector

- Developed a flexible architecture that could accommodate non-lC elements
- Funding for two NTIS officers on COSPO staff to assist in coordination between NTIS and COSPO
- Assisted FBIS and NTIS in creaking the World News Connection, which made selected FBIS product available to US Government and the public
- Helped NSA transfer government-owned code to other government agencies Compiled a list and periodic updates listing of Web sites covering East Asia for users at Pacific Command and other OSIS sites, including Australia Coordinated open source initiatives with several international partners. both bilaterally and multilaterally with Commonwealth partners

 9. Provide Community guidance for efficient use of resources

- Identify all open source activities in Community agencies and delineate the interactive roles of Community open source components Invest in tools, technology, data, and people to meet user needs
- Strive for better resource distribution for accessing, acquiring, processing, distributing, and exploiting information to best meet user needs
- Provide guidance for and review and evaluate individual NFIP programmatic activities that affect the Community's open source efforts
- Use cost-benefit analyses of open source activities to maximize investment
- Identify departmental lead agents to carry out Community open source goals and objectives, and hold them accountable to Community oversight
- Staff the Community Open Source Program Office with department representatives who are dedicated to implementing Community approved goals and objectives

- COSPO coordinated Open Source CBJB inputs throughout the Community.
- COSPO invested approximately 80% of budget into Community activities (OSIS, data purchase, hosting community conferences, and system development)
- COSPO coordinated Community guidance for the Open Source Program.
- Enhanced Community manning of key COSPO positions.
- OIR designated as Executive Agent for OCR technology
- NAIC designated Executive Agent for machine translation technology

10. Preserve a sound base to meet the challenges of the future

- Maintain a solid programmatic infrastructure and resource base of personnel, structures, organizations, relationships, and human and technological capital investment within the NFIP open source program
- Within the NFIP open source program build strategic innovative solutions to achieve program growth on the basis of the resources required to meet the explicit needs of the Community
- Champion the funding of Community infrastructure efforts upon which Community open source activities depend
- Advocate the Community Open Source Program before Commumit,v managers, the Executive Branch and Congress on all program and budget/DCID issues related to or affecting open source activity

- COSPO coordinated strong community support to preserve as much as possible of the DCI Open Source Enhancements
- Open Source program represented at IPRG and other community resource forums (??)
- COSPO drafted, coordinated and finalized the Open Source inputs into CBJE Volumes.
- Coordinated the community responses to Congressional queries.