May 30, 2005

Next Generation Cyberinfrastructure Tools
With Applications to Complex Behavior of Organizations and Individuals

Researchers in the social and behavioral sciences and computer and information sciences have many important synergistic relationships.One way in which this is manifest is in the development and utilization of data. On the one hand, social and behavioral scientists find new ways to create and analyze data in their endeavors to describe human and organizational behavior. On the other hand, computer and information scientists conduct research that yields new ways to improve both domain-specific and general-purpose tools to analyze and visualize scientific data -- such as improving processing power, enhanced interoperability of data from different sources, data mining, data integration, information indexing and data confidentiality protection - or what we have termed cyberinfrastructure tools.

This solicitation invites proposals for "information infrastructure testbeds", each of which would include the development of the next generation of cyberinfrastructure tools applied to data from various sources collected in two areas of research fundamental to social and behavioral scientists: organizations and individuals. The tools that are developed on these platforms must not only change ways in which social and behavioral scientists research the behavior of organizations and individuals, but also serve sciences more broadly.

It is envisioned that proposals for the "organization information testbed" will address three specific components:

• the development of tools that facilitate the integration of qualitative and quantitative information from heterogeneous sources, multiple media, and/or multiple modes;
• investment in basic research that addresses the protection of the confidentiality of respondents in computerized, widely accessible databases; and
• the development of incentives, standards and policies for collecting, storing, archiving, accessing, and publishing research results using organization-relevant information.

It is envisioned that proposals for the "individual information testbed" should concern cyberinfrastructure tools that can be applied to both large scale and distributed data-sets. Proposals should address cyberinfrastructure tools that facilitate automatic collection, integration, annotation, archiving, accessing, and analyzing of

• existing distributed data sets and/or
• extensive audio and video recordings and details of physical artifacts, while paying special attention to
• the protection of the confidentiality of participant identity in widely accessible, computerized databases.

Breakthroughs in the social and behavioral sciences have been enabled by and increasingly depend on advances in cybertechnologies. The impact of social and behavioral research has proven to be substantial – leading, for example, to new insights into core national challenges such as the creation of wealth and jobs, as well as to fundamental understanding of behavioral phenomena at scales ranging from the molecular to the social. The special technical needs of the social and behavioral scientists have also posed numerous research challenges to cyberinfrastucture experts and contributed to substantial advances in the cybertechnologies of other sciences: for example, geographic information systems applications used in engineering, the geosciences and biology, and statistical software packages developed for social and behavioral science applications but now widely used across the health sciences. Yet, the range of opportunities and the full benefits that can be derived from the application and use of large-scale scientific infrastructure, currently called "cyberinfrastructure" at NSF, have yet to be fully explored and realized by social and behavioral scientists.

Stimulated by the potential contribution of the social and behavioral sciences to such national challenges, as well as by the potential advances that can be generated by responding to the special needs of social and behavioral scientists, the Directorate for Social, Behavioral and Economic Sciences (SBE) together with the Directorate for Computer and Information Science and Engineering (CISE) announces a competition designed to develop new cyberinfrastructure that can be shared across the social and behavioral sciences as well as with other sciences. Research will be supported within two "testbeds", or platforms on which experimental cyberinfrastructure tools and products will be deployed with the aim of identifying successful tools and products for further development. Examples of such cyberinfrastructure tools might be shared domain-specific and general-purpose tools to analyze and visualize scientific data -- such as improving processing power, the interoperability of data from different sources, data mining, data integration, information indexing and data confidentiality protection.

These "testbeds" are:

I. information collected on organizations from a variety of heterogeneous, independently developed data sources, such as administrative and survey data, temporal, spatial and image data or textual data. The goal is to free users from having to locate the data sources, interact with each data source in isolation, and manually combine data from multiple formats and multiple sources. This could be achieved through the creation of new and more accurate and efficient ways to collect, code and analyze qualitative information from case studies, and other sources, and to enable the linking of this information with repositories of quantitative data, while protecting fundamental privacy and confidentiality concerns. The research should be designed to show how appropriate cyberinfrastructure tools can lead to multiple advances in the empirical understanding of how organizations emerge, develop, thrive or weaken.

II information collected on individuals such as human cognition, activities and artifacts. The focus will be on projects to develop cyberinfrastructure tools that can be used to better acquire, annotate, archive, access and analyze data in the form of audio and video recordings, records of physical artifacts, changes in images across time, etc., and that boost levels of volume and complexity tractable for analysis on both large-scale and distributed datasets. These proposals should show how appropriate cyberinfrastructure tools can lead to multiple advances in the detailed empirical understanding of individual behavior and/or the integration of previously isolated distributed databases.

Advances in both "testbeds" must have applicability across and beyond the various social and behavioral sciences. They must also have the demonstrated potential to be scalable across time, geographies, institutions, and/or disciplines. Finally, they must exploit state-of-the-art knowledge to ensure sustained relevance and usability.

Additional information on cyberinfrastructure and the social, behavioral, and economic sciences can be found in "Revolutionizing Science and Engineering through Cyberinfrastructure: Report on the National Science Foundation Blue-Ribbon Advisory Panel on Cyberinfrastruture, January 2003, Daniel E. Atkins, Chair"

https://www.nsf.gov/cise/sci/reports/toc.jsp

and the “ACLS Commission on Cyberinfrastructure for the Humanities & Social Sciences,”

http://www.acls.org/cyberinfrastructure/cyber_meeting_notes_august.htm

This competition is viewed as an initial step toward meeting the goal of innovatively developing cyberinfrastructure tools that are applied to two "testbeds" and yet have broad applicability across the social, behavioral, economic and other sciences and engineering. Investigators are strongly encouraged to contact the cognizant program officers.

Proposals must address the protection of data providers from identification, exploitation, and other misuses of personal or organizational information. Such misuses present a perpetual challenge to the melding of data and media of different types in a tool for widespread use. Proposals in response to this solicitation must show a sophisticated understanding of this sociotechnical problem and must propose to advance fundamental knowledge of effective privacy protections during the development of the analytical tools and in their later use by various research communities.

Proposals must demonstrate potential long-term sustainability, usability, and impact. This could be achieved for the organizational "testbed", for example, by documenting proposed collaboration with firms in an industry, attracting support from foundations or developing replicable incentive-compatible policies for collecting, storing, accessing, and disseminating data while continuing to utilize and advance relevant cybertechnology. It could be achieved for both "testbeds" by, for example, capitalizing on existing infrastructure, such as research centers or supercomputer centers.

Research supported under this solicitation must involve medium-scale groups of senior personnel representing multiple disciplines, including social and behavioral science as well as computer science. Teams must include individuals with a demonstrated capacity to identify and develop the appropriate cyberinfrastructure tools -- which is what we term methodologies and tools for the representation and manipulation of large volumes of data for heterogeneous, multimodal sources, on either organizations or individuals. All proposals should indicate how the research will provide opportunities for training, education and broadening participation.

Examples of topics the cyberinfrastructure tools could be applied to are listed below; followed by a description of the "testbeds" to which the tools should be applied. It is desirable that tools within a "testbed" are well integrated with each other.

Examples of Cyberinfrastructure Tools

A. Social and Behavioral Informatics:

• Social and Behavioral Data Models and Systems. Implementable theoretical foundations for the representation and manipulation of advanced data types (e.g., temporal, spatial and image data, textual data, administrative or survey data); data/knowledge calibration and validation. Systems issues include system extensibility; development of user-transparent, multi-level storage management; multi-media data indexing; partial match retrieval algorithms; archiving; and version control. Research in this area must consider the special data and information characteristics associated with the social and behavioral sciences "testbeds" described below.
• Analysis of Organizational and Individual Databases and Information Resources. Topics span computing environment transparency; establishing baseline patterns, data examination, selection, analysis and manipulation of temporally or spatially related data; knowledge discovery algorithms; information extraction (e.g., from abstracts of publications), citation analysis, visualization; parallel model execution and cross-validation on large volumes of data; automated knowledge acquisition; incorporation of new knowledge into a system; and audit trail provisions including data provenance. The research in this area must be done in connection with the "testbeds" described below.
• Analysis of Social and Behavioral Multimodal and Multimedia. A key research challenge in many research problems is to derive measurements or abstract features from 2-D, 3-D and multispectral images as well as well as other audio and other sensory data and to use this derived information for generating or evaluating hypotheses.
• Shared Resources Environments. The construction of shared, archived, and documented data, publication, or software resources that can accelerate the rate of scientific discovery.

B. Information Integration:

The goal of information integration should be to integrate many different, disparate and possibly distributed sources; support automated discovery of new data sources and information within them; facilitate their configuration, management and system maintenance; incorporate structured, semi-structured, text, image, video, time-series, 3D images, citations, graphs, speech and other data streams; and provide flexible querying of the sources and the data.

Some of the specific challenges include:

• Unifying Data Models and System Descriptions: There is a need to develop stronger theoretical foundations for the representation and integration of information of various types from extant data models (e.g., temporal, spatial and image data, textual data, administrative and survey data) as well as the scientific literature into conceptually coherent views. Specific topics include: metadata management and integration; the automated collection of metadata from instruments and processes that transform data, ontologies and taxonomies; data/knowledge calibration; heterogeneity of data type and format; scale of distributed systems; rapid integration of new information sources. Research in this area must consider the special data characteristics associated with the social and behavioral sciences disciplines and data described in the "testbeds" below.
• Reconciling heterogeneous formats schemas and ontologies: The fundamental problem in any data sharing application is that systems are heterogeneous in many different aspects, such as different ways of representing data and/or knowledge about the world, different representation mechanisms (e.g., relational databases, legacy systems, XML schemas, ontologies), different access methods and policies. In order to share data among heterogeneous sources, approaches to form a semantic mapping of their respective representations are needed to avoid manual intervention in each step of converting and merging data resources.
• Web semantics: Data on the web needs to be defined and linked in a way that it can be used by machines not just for display purposes, but also for automation, integration and reuse of data across various applications. Supported research topics will include frameworks for describing resources, methods of automating inferences about web data and resources, and the development of interoperable ontologies, mark up languages and representations for specific social, behavioral and other scientific domains.
• Decentralized data-sharing: Traditional data integration systems use a centralized mediation approach, in which a centralized mediator, employing a mediated schema, accepts user queries and reformulates them over the schemas of the different sources. However, mediated schemas are often hard to agree upon, construct and maintain. For example, researchers conducting social and behavioral research share their experimental results with each other, but may do it in an ad hoc fashion. A similar scenario is found in data sharing among government agencies. Architectures and protocols that enable large-scale sharing of data with no central control are needed.
• Data-sharing on advanced cyberinfrastructure: Research topics will include models for federating information resources in advanced grid computing and/or Web services, integration and understanding of sensor information, the collection of metadata from sensors including models and tools to cope with the scale, pervasiveness, concurrency and redundancy of sensor data. Effective integration of network management information will be critical to enable basic networking functions such as routing, overlay node placement, denial-of-service detection, and fault recovery. The integration of network management information will facilitate adapting network resources to changing conditions.
• On-the-fly integration: Currently, data integration systems rely on relatively static configurations with a set of long-lived data sources. On-the-fly integration refers to scenarios where one wants to integrate data from a source immediately after discovering it. The challenge is to significantly reduce the time and skill needed to integrate data sources so that scientists can focus on domain problems instead of information technology challenges.

Separate criteria for each "testbed" follow.

"TESTBED" I: ORGANIZATIONS

Research on organizations has historically involved many different paths using different types of methods and data. One research path has relied upon the collection and analysis of large scale, nationally representative datasets on organizations. Another research path has relied upon the development of rich case study information by individual researchers and sets of researchers. Another has been the development of complex models with rich descriptive realism as exemplified by agent-based modelling approaches; and yet another the examination of information collected by observational and ethnographic methods.

Each of these approaches has led to interesting and useful findings. Researchers working with large-scale firm-level datasets, for example, have uncovered important underlying dynamics: much of productivity growth is due to factor reallocation from less productive to more productive firms; a disproportionate amount of economic change is driven by new firms displacing old; and aggregate economic statistics do not capture the enormous amount of turbulence uncovered by an examination of firm level micro-data. However, the quantitative data are not rich on non-economic indicators, such as intra-organizational networks, personnel policies and the status of women and minorities. In contrast, case study researchers have described rich contextual variation in organizational decision-making, as well as how hard-to-quantify phenomena like culture and management affect organizational performance. Case studies have often uncovered important forces that are later measured in larger-scale work. But case study research has often suffered from the criticism that that work is neither generalizable nor replicable. Similarly, researchers in the agency-based modelling literature who use real-world databases to both calibrate and validate theoretical models as well as generate new data, provide fresh new perspectives on describing organizations - particularly the medium-term dynamics. Yet they need new approaches to organize and analyze data from a variety of sources in order that their models more closely approximate reality . All sets of researchers could gain from the development of cyberinfrastructure tools that permitted the integration of the different sources of data.

Advances in cyber-infrastructure now make it possible to fundamentally change the way in which social scientists collect, create, and analyze data on organizations and people so that each research path can gain from knowledge derived from others. This competition aims to create tools that facilitate the combination of information from a variety of sources -- such as text, documents, video, audio and maps -- using data on organizations as a test-bed. This will involve creating a set of tools that permit the creation, standardization and searchability of a data archive – although the specific context in this proposal is to inform and enrich the empirical analysis of organizations.

Proposed projects should combine elements of each of the following three areas, although consideration will be given to outstanding proposals that address one or two of the areas.

• The development of tools that combine information on organizations from multiple media – both qualitative and quantitative
• Basic research that addresses the protection of the confidentiality of individual and organizational data providers and survey respondents, and
• The development of incentives, standards and policies for collecting, storing, accessing, and publishing research results using the data and information, as well as including appropriate protections of confidentiality and security.

Proposals should focus specifically on the development of tools and methodology for one set of organizations, such as an industry, or group of industries, but the tools developed in the proposed pilot project should be demonstrably scalable to additional industries and organizations (such as unions, voluntary organizations schools, governments and political groups). The tools should have the potential to meet needs in other sciences. The proposed project should document the potential for sustainability. Expressions of interest in the research and the "testbed" area from the private sectors such as, for example, letters from private foundations or potential industrial partners is one way to document such potential. The case for sustainability might also be made by documenting how the proposed tools could be used to develop a database that addresses some of the following research questions or questions of similar national or scientific importance:

• The factors and processes that contribute to the emergence and growth of businesses and successful entrepreneurship - particularly job and wealth creation
• The factors and processes that contribute to the success or failure of mergers and acquisitions
• The organizational response to globalization, such as outsourcing, and the consequences for job creation and business competitiveness
• The factors and processes that contribute to employees’ satisfaction or dissatisfaction with their jobs
• The existence and function of internal and boundary spanning networks
• The implications of legal and administrative regulation for customer or employee relationship as well as for the pace or shape of firm development
• The hiring, training and retention of different types of workers: for example, older or disabled individuals

"TESTBED" II: INDIVIDUALS.

Research on human behavior has traditionally involved summary data that reduce the complexity of detail available for analysis and subsequent reanalysis, such as summary data that omits details of change across time or variability in individual responses. Additionally, other data collected on small scales would have greater scientific value if used in larger scale studies that combined many data sets. This competition encourages the development of cyberinfrastructure tools that make such data comprehensible, while protecting privacy and confidentiality. Proposals are invited that address the complex behavior of individuals. For example, proposals that integrate multiple data types, such as voice, video, images, or physical artifacts, and that acquire, annotate, archive, access and analyze these and boost levels of volume and complexity up to and including large-scale and distributed datasets.

For example, real time observations of human behavior can generate vast data bases that currently exist in multiple locations on and off the web. Or consider legislation that mandates anthropological mitigation during construction projects, resulting in the excavation of millions of individual objects that lack integration into a single data-base. A similar story can be told for data on children's development and social phenomena as well as remote sensing images that inform work in geography and the regional sciences. In addition there are huge numbers of language data sets and knowledge bases, variously sized, that are currently available on the Web or that will soon come online.

As another example, archaeological artifacts, fossils and skeletal material, and historical records all provide threads of information about past human actions and behavior. The human migrations that populated the world, the rise and fall of civilizations, the origin and spread of innovative behaviors such as agriculture are all represented by data from which many insights could be gained if standardization and interoperability could be applied to extant data collections. In many cases a means to incorporate data and analytical techniques from a wide variety of other disciplines (e.g. geographic, geological, genetic, linguistic) would greatly enhance the utility of all of the constituent parts. The ability to incorporate ethnographic case material would enrich large, cross-sectional databases and add substance and validity.

These "testbed" descriptions are applicable across many disciplines in the behavioral, cognitive, anthropological and geographic sciences, which have amassed large numbers and varieties of data sets and analyses of them. The key to solving the interoperability problem is to develop concepts, semantics and mechanisms that can be used to store and analyze data. Using the data accumulated by linguists as an example, it is clear that these resources are currently in a mix of formats and states of annotation including (a) raw or partially analyzed text files in a variety of formats, (b) idiosyncratically prepared and analyzed dictionaries, word lists, sketch grammars, cross-linguistic typological comparisons,in a variety of formats, flat or relational databases, and other files, (c) for example, collections of annotated text and lexicons in standard "best practice" XML formats, and (d) audio and audio-video recordings of speech, ideally but not necessarily aligned with one or more transcriptions (phonetic, standard orthography), and possibly also with sociolinguistic and/or grammatical annotations.

Data abound, but, even assuming that these resources are discoverable, they are still not interoperable, i.e. comparable for purposes of systematically carrying out research across them, or for enriching them. Achieving interoperability will require that at least the following three components be developed, in addition to the development of procedures and protocols to protect confidentiality and privacy as well as to prevent copyright violations:

• Standard methods for converting legacy and other non-best practice data collections to best practice.
• A universally accessible knowledge base that provides uniform and agreed upon rules for the integrating and interpreting of the data, both domain-specific (e.g. linguistic, social psychology, or geography, etc.) or across domains.
• A set of methods for linking the best-practice formatted data to the standard knowledge base. These involve the creation of term sets and standard protocols for linking them both to the documents that use them and to the general knowledge base.

Parallel issues emerge across many disciplines in the behavioral, cognitive, anthropological and geographic sciences, as well as other sciences. There exist now unlinked data resources requiring the terabyte scale of storage, and analyses from around the world, the terra scale, that, with attention to interoperability, will answer longstanding questions about the complexity of human behavior and generate increasingly complex questions. The use of data that are in the form of visual images, audio and visual records, and physical objects is essential, but not unique to the behavioral cognitive sciences. Thus these fields offer an excellent testbed for the development of cyberinfrastructure for amassing and analyzing this sort of data, but the developing infrastructure will have uses across the sciences broadly. These tools necessarily raise ethical issues including questions of confidentiality, privacy, and long term access that must be addressed in any proposal.

The National Science Foundation expects to make one standard or continuing grant in each "testbed", of approximately $2 million each, for a total of $4 million, to cover these activities, subject to the availability of funds. NSF support may be requested for professional staff since such participation is likely to be necessary.

The following instructions supplement the GPG guidelines.

• The project summary must identify whether the proposer is submitting for "testbed" I or "testbed" II.
• The project description may be up to 15 pages. Of these, up to 5 pages should be spent presenting an outreach and dissemination plan, a coordination plan (described below) and an indication that the research will have some teaching and education component.
• The proposal must contain a coordination plan, which includes 1) the specific roles of the PI, co-PIs, other senior personnel and paid consultants at all institutions involved, 2) how the project will be managed across institutions and disciplines, 3) identification of the specific coordination mechanisms that will enable cross-institution and/or cross-discipline scientific integration (e.g., yearly workshops, graduate student exchange, project meetings at conferences, use of the grid for videoconferences, software repositories, etc.), and 4) pointers to the budget line items that support these coordination mechanisms. If budget cuts are necessary, NSF staff will make every effort not to reduce the budget for project coordination other than proportionally.
• Proposals may refer to supporting materials, such as the datasets or methodology that may be used, previous related work and other directly relevant information, posted on investigators' publicly available websites. Reviewers are not, however, obligated to view supporting materials.
• Project teams must consist of medium-scale groups led by senior personnel from multiple disciplines, including social and behavioral science as well as computer science.

May 30, 2005