NSF 08-529: Petascale Computing Resource Allocations (PRAC)

Program Solicitation NSF 08-529

National Science Foundation Office of Cyberinfrastructure

Full Proposal Deadline(s) (due by 5 p.m. proposer's local time):

March 31, 2008

May 20, 2009

March 17, 2010

June 30, 2011

June 30th, Annually thereafter

Important Information And Revision Notes

A revised version of the NSF Proposal & Award Policies & Procedures Guide (PAPPG), NSF 11-1, was issued on October 1, 2010 and is effective for proposals submitted, or due, on or after January 18, 2011. Please be advised that the guidelines contained in NSF 11-1 apply to proposals submitted in response to this funding opportunity. Proposers who opt to submit prior to January 18, 2011, must also follow the guidelines contained in NSF 11-1.

Cost Sharing: The PAPPG has been revised to implement the National Science Board's recommendations regarding cost sharing. Inclusion of voluntary committed cost sharing is prohibited. In order to assess the scope of the project, all organizational resources necessary for the project must be described in the Facilities, Equipment and Other Resources section of the proposal. The description should be narrative in nature and must not include any quantifiable financial information. Mandatory cost sharing will only be required when explicitly authorized by the NSF Director. See the PAPP Guide Part I: Grant Proposal Guide (GPG) Chapter II.C.2.g(xi) for further information about the implementation of these recommendations.

Data Management Plan: The PAPPG contains a clarification of NSF's long standing data policy. All proposals must describe plans for data management and sharing of the products of research, or assert the absence of the need for such plans. FastLane will not permit submission of a proposal that is missing a Data Management Plan. The Data Management Plan will be reviewed as part of the intellectual merit or broader impacts of the proposal, or both, as appropriate. Links to data management requirements and plans relevant to specific Directorates, Offices, Divisions, Programs, or other NSF units are available on the NSF website at: https://www.nsf.gov/bfa/dias/policy/dmp.jsp. See
Chapter II.C.2.j of the GPG for further information about the implementation of this requirement.

Postdoctoral Researcher Mentoring Plan: As a reminder, each proposal that requests funding to support postdoctoral researchers must include, as a supplementary document, a description of the mentoring activities that will be provided for such individuals. Please be advised that if required, FastLane will not permit submission of a proposal that is missing a Postdoctoral Researcher Mentoring Plan. See Chapter II.C.2.j of the GPG for further information about the implementation of this requirement.

Summary Of Program Requirements

General Information

Program Title:

Petascale Computing Resource Allocations (PRAC)

Synopsis of Program:

In 2012, a new NSF-funded petascale computing system, Blue Waters, will go online at the University of Illinois. The goal of this facility is to open up new possibilities in science and engineering by providing computational capability that makes it possible for investigators to tackle much larger and more complex research challenges across a wide spectrum of domains. The purpose of this solicitation is to invite research groups that have a compelling science or engineering challenge that will require petascale computing resources to submit requests for allocations of resources on the Blue Waters system. Proposers must be prepared to demonstrate that they have a science or engineering research problem that requires and can effectively exploit the petascale computing capabilities offered by Blue Waters. Proposals from or including junior researchers are encouraged as one of the goals of this solicitation is to build a community capable of using petascale computing.

Cognizant Program Officer(s):

Please note that the following information is current at the time of publishing. See program website for any updates to the points of contact.

• Irene Qualters, telephone: (703) 292-2339, email: iqualter@nsf.gov

Applicable Catalog of Federal Domestic Assistance (CFDA) Number(s):

• 47.080 --- Office of Cyberinfrastructure

Award Information

Anticipated Type of Award: Standard Grant

Estimated Number of Awards: 10 to 12

Anticipated Funding Amount: $400,000 to $500,000 annually, pending availability of funds.

Eligibility Information

Organization Limit:

None Specified

PI Limit:

None Specified

Limit on Number of Proposals per Organization:

None Specified

Limit on Number of Proposals per PI: 1

An individual may be the PI on no more than one proposal that responds to this solicitation. There is no limit on the number of proposals with which an individual may be associated in other capacities such as co-PI or other senior personnel.

Proposal Preparation and Submission Instructions

A. Proposal Preparation Instructions

• Letters of Intent: Not Applicable

• Preliminary Proposal Submission: Not Applicable

• Full Proposals:
  • Full Proposals submitted via FastLane: NSF Proposal and Award Policies and Procedures Guide, Part I: Grant Proposal Guide (GPG) Guidelines apply. The complete text of the GPG is available electronically on the NSF website at: https://www.nsf.gov/publications/pub_summ.jsp?ods_key=gpg.
  • Full Proposals submitted via Grants.gov: NSF Grants.gov Application Guide: A Guide for the Preparation and Submission of NSF Applications via Grants.gov Guidelines apply (Note: The NSF Grants.gov Application Guide is available on the Grants.gov website and on the NSF website at: https://www.nsf.gov/publications/pub_summ.jsp?ods_key=grantsgovguide)

B. Budgetary Information

• Cost Sharing Requirements: Inclusion of voluntary committed cost sharing is prohibited.

• Indirect Cost (F&A) Limitations: Not Applicable

• Other Budgetary Limitations: Other budgetary limitations apply. Please see the full text of this solicitation for further information.

C. Due Dates

• Full Proposal Deadline(s) (due by 5 p.m. proposer's local time):

  March 31, 2008

  May 20, 2009

  March 17, 2010

  June 30, 2011

  June 30th, Annually thereafter

Proposal Review Information Criteria

Merit Review Criteria: National Science Board approved criteria. Additional merit review considerations apply. Please see the full text of this solicitation for further information.

Award Administration Information

Award Conditions: Additional award conditions apply. Please see the full text of this solicitation for further information.

Reporting Requirements: Standard NSF reporting requirements apply.

Agency Contacts

Other Information

I. Introduction

In recent decades, the development of powerful computing resources has been driven by the need to solve challenging questions in science and engineering with significant impacts on knowledge about the natural world, on industrial competitiveness, and on national security. At the same time, as researchers in more and more areas of science and engineering have developed techniques for using computation to understand their grand challenges, there has been a steady expansion in the breadth of the research frontier at which large-scale computation has become an essential tool. As we approach the second decade of the twenty-first century, researchers are preparing to apply new computational resources, an order of magnitude more powerful than those presently available, to an extremely varied range of problems. This range includes materials science, nano-engineering, fluid dynamics, climate and earth system dynamics, cosmology and astrophysics, chemistry and biochemistry, economics and social science, neuroinformatics and bioinformatics, as well as many different topics within physics and engineering.

In 2012, researchers will have access to a powerful next-generation computing system capable of delivering sustained performance in excess of one petaflop/s on a broad range of types of calculations. This system, named, Blue Waters, is funded by NSF and will be deployed at the University of Illinois at Urbana-Champaign.

Because of the expense of acquiring and operating Blue Waters, the allocation of time on Blue Waters to a research group to solve a challenging problem in science or engineering represents a considerable investment by NSF. The purpose of this solicitation is to elicit requests for resource allocations on the Blue Waters system so that these requests can be reviewed by the scientific community in a timely fashion and so that there is sufficient time for research groups to prepare to make optimal use of the Blue Waters system. Successful proposals will receive preliminary allocations of access to the system to support the research that they have planned, together with limited travel funds to support technical coordination with the Blue Waters project team. This program does not provide funds for the research itself or for the development of models or analysis tools.

II. Program Description

This solicitation seeks proposals to make use of Blue Waters for breakthrough research in any domain supported by the National Science Foundation or another federal agency.

The Blue Waters system consists of a large number of powerful SMP nodes connected by a very high-bandwidth, low latency interconnect and can be programmed using a MPI or hybrid MPI-OpenMP model. The system offers a large memory, connected to the processors at very high bandwidth, that also can be accessed using a global shared memory protocol. The system incorporates a tightly integrated I/O subsystem and is suitable for computations with very large input and/or output requirements. The system is also intended to support visualization of large-scale datasets produced by computations that use the system. A large amount of archival storage is associated with the system. The system design responds to input from researchers in a broad range of science and engineering disciplines. Non-proprietary details of the system design may be obtained from http://ncsa.uiuc.edu/BlueWaters.

Trends in HPC architecture are such that, at the highest end, production systems in the next two to three years are expected to consist of tens of thousands to a few hundred thousand processors, with each processor containing multiple cores, each core capable of executing multiple threads, and, often, arithmetic units that support small vector instructions. These features present a programmer with a number of different ways to exploit different types of parallelism within algorithms. Optimizing performance involves a number of challenges, including discovering and exploiting parallelism within codes and overlapping different types of operations. Multi-level caches, local and remote main memory, intra-nodal and inter-nodal communication networks and parallel I/O interfaces offer an increasingly deep hierarchy of latency within computing systems. In addition, other types of commercial HPC system designs are emerging, including hybrid systems in which general purpose processors are coupled with specialized co-processors, either on-chip or separate. Several recent developments can simplify the challenge of developing scientific and engineering computer codes that scale.

To effectively use computation at sustained rates of a petaflop/s or more, with memory-resident data of order one petabyte and input-output datasets is a considerable computational science challenge in itself. Some algorithms readily scale across large numbers of processing elements. In general, though, the design and implementation of computing codes that can harness all of the resources of a system like Blue Waters to address complex science and engineering problems that are not readily amenable to attack by other means is not trivial. It is anticipated that research groups may require several years of preparation before being ready to exploit the first sustained petaflop/s systems. A number of research groups are already being funded by federal agencies and/or industry to develop petascale applications. The purpose of this solicitation is to identify groups who plan to use petascale computing for ground-breaking science or engineering research, who have a need for the unique resource that Blue Waters represents, and that are likely to be ready to use Blue Waters effectively when it comes fully online in 2012. Because of the intrinsic value of the Blue Waters resources, a research group will only be granted significant access to the production system after its request for a resource allocation has been successful in a competitive, merit review managed by NSF. This solicitation represents the first phase of that merit review process. Successful proposers to this solicitation will be granted a pre-allocation of Blue Waters resources together with a small amount of funds to support travel costs. The Blue Waters project team will be able to offer consulting support and assistance to a relatively small number of projects prior to Blue Waters entering operations. This consulting support includes performance analysis and prediction, together with access to system simulators. It is anticipated that the Blue Waters team will give highest priority for consulting help to awardees under the current solicitation and under future solicitations of this type.

In the months prior to Blue Waters entering production in mid-2012, groups receiving awards under the current competition will be asked to submit a request for a final allocation. Reviewers of the final allocation requests will be asked to review progress and readiness.

III. Award Information

Anticipated Type of Award: Standard Grant

Estimated Number of Awards: 10 to 12

Anticipated Funding Amount: $400,000 to $500,000 annually, pending availability of funds.

Estimated program budget, number of awards and average award size/duration are subject to the availability of funds. The primary purpose of this solicitation is to identify research projects that are candidates for large allocations of Blue Waters computing resources. This solicitation does not provide direct funding support for the research associated with such projects. Instead it is intended to provide indirect support for research projects requiring petascale computing resources by paving the way for a project to receive a resource allocation on the Blue Waters system. For any project that is successful in the current competition, the receipt of an actual resource allocation is subject to a demonstration, when Blue Waters becomes available for use, of the readiness of the project to use the resources. It is anticipated that the Blue Waters team will provide consulting support to projects that are successful under this solicitation to help those projects prepare to make effective use of the Blue Waters system. To facilitate a project's interaction with the Blue Waters team, funding of up to $40,000 will be provided to support travel by members of the project team and/or the Blue Waters team.

IV. Eligibility Information

Organization Limit:

None Specified

PI Limit:

None Specified

Limit on Number of Proposals per Organization:

None Specified

Limit on Number of Proposals per PI: 1

An individual may be the PI on no more than one proposal that responds to this solicitation. There is no limit on the number of proposals with which an individual may be associated in other capacities such as co-PI or other senior personnel.

V. Proposal Preparation And Submission Instructions

A. Proposal Preparation Instructions

Full Proposal Preparation Instructions: Proposers may opt to submit proposals in response to this Program Solicitation via Grants.gov or via the NSF FastLane system.

• Full proposals submitted via FastLane: Proposals submitted in response to this program solicitation should be prepared and submitted in accordance with the general guidelines contained in the NSF Grant Proposal Guide (GPG). The complete text of the GPG is available electronically on the NSF website at: https://www.nsf.gov/publications/pub_summ.jsp?ods_key=gpg. Paper copies of the GPG may be obtained from the NSF Publications Clearinghouse, telephone (703) 292-7827 or by e-mail from nsfpubs@nsf.gov. Proposers are reminded to identify this program solicitation number in the program solicitation block on the NSF Cover Sheet For Proposal to the National Science Foundation. Compliance with this requirement is critical to determining the relevant proposal processing guidelines. Failure to submit this information may delay processing.

• Full proposals submitted via Grants.gov: Proposals submitted in response to this program solicitation via Grants.gov should be prepared and submitted in accordance with the NSF Grants.gov Application Guide: A Guide for the Preparation and Submission of NSF Applications via Grants.gov. The complete text of the NSF Grants.gov Application Guide is available on the Grants.gov website and on the NSF website at: (https://www.nsf.gov/publications/pub_summ.jsp?ods_key=grantsgovguide). To obtain copies of the Application Guide and Application Forms Package, click on the Apply tab on the Grants.gov site, then click on the Apply Step 1: Download a Grant Application Package and Application Instructions link and enter the funding opportunity number, (the program solicitation number without the NSF prefix) and press the Download Package button. Paper copies of the Grants.gov Application Guide also may be obtained from the NSF Publications Clearinghouse, telephone (703) 292-7827 or by e-mail from nsfpubs@nsf.gov.

In determining which method to utilize in the electronic preparation and submission of the proposal, please note the following:

Collaborative Proposals. All collaborative proposals submitted as separate submissions from multiple organizations must be submitted via the NSF FastLane system. Chapter II, Section D.4 of the Grant Proposal Guide provides additional information on collaborative proposals.

PROJECT DESCRIPTION

Proposers should note that the Project Description section of proposals is limited to be no more than 15 pages in length.

In addition to any sections required in the Grant Proposal Guide, the Project Description should include the following seven sections:

• Target Problem
• Intellectual Merit
• Broader Impacts
• Description of the Computational Codes to be used
• Development Plan
• Source(s) of Development Funding
• Resources Required

Target Problem: A detailed description of one or more specific research questions that the resources requested will be used to answer. Include an explanation of why a petascale resource of the leading-edge capability that Blue Waters represents is necessary to address this research. If you believe that this research will be transformative, please describe why it is transformative.

Intellectual Merit: A description of the intellectual merit associated with the research associated with the target problem. If there is substantial intellectual merit associated with the work to be done preparing to use the Blue Waters system effectively, this may also be described.

Broader Impacts: The broader impacts of this work where broader impacts are defined in the description of NSF’s second primary review criterion. (See Section VI.A, below.)

Description of the computational code(s) to be used: Describe the structure of the application codes that you intend to use. These may either currently exist, but might require enhancement, or they may be in development. Please include details about the algorithms involved and the approach that you intend to use to ensure that the code scales effectively on the Blue Waters architecture. Please include descriptions of how your code(s) will use each of the major system elements: the memory hierarchy, the communications network, the computational elements, and the I/O subsystem. Identify which system element(s) is/are likely to be the main bottlenecks and how the design of your application minimizes the impact of these bottlenecks. Describe how you intend to analyze the output resulting from your use of Blue Waters. IMPORTANT: Please describe any run-time libraries or special system software or program development environment features that you will require and the types of graphics support that you would find most useful.

Development plan: Describe the current state of readiness of the application codes that you intend to use and your plans for developing these to the point where they are ready to run in production mode on the Blue Waters system. Estimate the type of access that you will need, for development purposes, to Blue Waters system and processor simulators, to systems that are smaller than Blue Waters but that are still relatively large-scale systems, such as NSF’s “Track 2” HPC systems, and to consulting help from the Blue Waters project team. (Proposers are encouraged to contact the Blue Waters project team, before submitting a proposal, to obtain a better understanding of the type of help that it can provide.)

Source of Development Funding: If you are already funded to develop your petascale application, please identify the source, amount and duration of that funding. If not, describe how you intend to support the development work needed, if any.

Resources Required: Describe the Blue Waters resources required to complete research on the Target Problem. This description should include the number of system nodes needed for your runs, the anticipated actual memory usage, the expected numbers of each major class of arithmetic and logical operation, the expected numbers of local and remote memory accesses, the total number of node-hours required, the anticipated input and output requirements, the amount of data that you anticipate transferring to or from the Blue Waters enclave, and any other system resource needs that you anticipate.

B. Budgetary Information

Cost Sharing: Inclusion of voluntary committed cost sharing is prohibited

Other Budgetary Limitations: To facilitate a project's interaction with the Blue Waters team, funding of up to $40,000 will be provided to support travel by members of the project team and/or the Blue Waters team. Funding requests must not exceed $40,000.  Funds may only be requested in the following two budget categories: Travel (members of the project team) and/or Participant Support Costs (travel of the Blue Waters team).

C. Due Dates

• Full Proposal Deadline(s) (due by 5 p.m. proposer's local time):

  March 31, 2008

  May 20, 2009

  March 17, 2010

  June 30, 2011

  June 30th, Annually thereafter

D. FastLane/Grants.gov Requirements

• For Proposals Submitted Via FastLane:

  Detailed technical instructions regarding the technical aspects of preparation and submission via FastLane are available at: https://www.fastlane.nsf.gov/a1/newstan.htm. For FastLane user support, call the FastLane Help Desk at 1-800-673-6188 or e-mail fastlane@nsf.gov. The FastLane Help Desk answers general technical questions related to the use of the FastLane system. Specific questions related to this program solicitation should be referred to the NSF program staff contact(s) listed in Section VIII of this funding opportunity.

  Submission of Electronically Signed Cover Sheets. The Authorized Organizational Representative (AOR) must electronically sign the proposal Cover Sheet to submit the required proposal certifications (see Chapter II, Section C of the Grant Proposal Guide for a listing of the certifications). The AOR must provide the required electronic certifications within five working days following the electronic submission of the proposal. Further instructions regarding this process are available on the FastLane Website at: https://www.fastlane.nsf.gov/fastlane.jsp.

• For Proposals Submitted Via Grants.gov:

  Before using Grants.gov for the first time, each organization must register to create an institutional profile. Once registered, the applicant's organization can then apply for any federal grant on the Grants.gov website. Comprehensive information about using Grants.gov is available on the Grants.gov Applicant Resources webpage: http://www07.grants.gov/applicants/app_help_reso.jsp. In addition, the NSF Grants.gov Application Guide provides additional technical guidance regarding preparation of proposals via Grants.gov. For Grants.gov user support, contact the Grants.gov Contact Center at 1-800-518-4726 or by email: support@grants.gov. The Grants.gov Contact Center answers general technical questions related to the use of Grants.gov. Specific questions related to this program solicitation should be referred to the NSF program staff contact(s) listed in Section VIII of this solicitation.

  Submitting the Proposal: Once all documents have been completed, the Authorized Organizational Representative (AOR) must submit the application to Grants.gov and verify the desired funding opportunity and agency to which the application is submitted. The AOR must then sign and submit the application to Grants.gov. The completed application will be transferred to the NSF FastLane system for further processing.

VI. NSF Proposal Processing And Review Procedures

Proposals received by NSF are assigned to the appropriate NSF program where they will be reviewed if they meet NSF proposal preparation requirements. All proposals are carefully reviewed by a scientist, engineer, or educator serving as an NSF Program Officer, and usually by three to ten other persons outside NSF who are experts in the particular fields represented by the proposal. These reviewers are selected by Program Officers charged with the oversight of the review process. Proposers are invited to suggest names of persons they believe are especially well qualified to review the proposal and/or persons they would prefer not review the proposal. These suggestions may serve as one source in the reviewer selection process at the Program Officer's discretion. Submission of such names, however, is optional. Care is taken to ensure that reviewers have no conflicts of interest with the proposal.

A. NSF Merit Review Criteria

All NSF proposals are evaluated through use of the two National Science Board (NSB)-approved merit review criteria: intellectual merit and the broader impacts of the proposed effort. In some instances, however, NSF will employ additional criteria as required to highlight the specific objectives of certain programs and activities.

The two NSB-approved merit review criteria are listed below. The criteria include considerations that help define them. These considerations are suggestions and not all will apply to any given proposal. While proposers must address both merit review criteria, reviewers will be asked to address only those considerations that are relevant to the proposal being considered and for which the reviewer is qualified to make judgements.

What is the intellectual merit of the proposed activity?
How important is the proposed activity to advancing knowledge and understanding within its own field or across different fields? How well qualified is the proposer (individual or team) to conduct the project? (If appropriate, the reviewer will comment on the quality of the prior work.) To what extent does the proposed activity suggest and explore creative, original, or potentially transformative concepts? How well conceived and organized is the proposed activity? Is there sufficient access to resources?

What are the broader impacts of the proposed activity?
How well does the activity advance discovery and understanding while promoting teaching, training, and learning? How well does the proposed activity broaden the participation of underrepresented groups (e.g., gender, ethnicity, disability, geographic, etc.)? To what extent will it enhance the infrastructure for research and education, such as facilities, instrumentation, networks, and partnerships? Will the results be disseminated broadly to enhance scientific and technological understanding? What may be the benefits of the proposed activity to society?

Examples illustrating activities likely to demonstrate broader impacts are available electronically on the NSF website at: https://www.nsf.gov/pubs/gpg/broaderimpacts.pdf.

Mentoring activities provided to postdoctoral researchers supported on the project, as described in a one-page supplementary document, will be evaluated under the Broader Impacts criterion.