January 14, 2005

ROLE Letter of Intent (for 2005 competition only)

March 31, annually

EREC Letter of Intent

December 11, annually

ROLE Letter of Intent due December 11, 2005 and annually thereafter.

March 04, 2005

ROLE Full proposal (for 2005 competition only)

May 15, annually

EREC Full Proposal

January 10, annually

ROLE Full proposal for 2006, and annually thereafter.

Evaluative Research and Evaluation Capacity Building (EREC) And Research on Learning and Education (ROLE)
A solicitation of the Division of Research, Evaluation, and Communication (REC)

Evaluative Research and Capacity Building (EREC)

The EREC program seeks proposals that offer unique approaches to evaluation practice in the generation of knowledge for the science, technology, engineering, and mathematics (STEM) education community and for broad policymaking within the research and education enterprise. Successful proposals may focus on one or more STEM education programs or projects in order to examine major issues in STEM education and/or may focus on the development of capacity within the education evaluation field.

Research on Learning and Education (ROLE)

The ROLE program seeks to capitalize on important developments across a wide range of fields related to human learning and to STEM education. It supports research across a continuum that includes (1) the biological basis of human learning; (2) behavioral, cognitive, affective and social aspects of STEM learning; (3) STEM learning in formal and informal educational settings; (4) STEM Policy research; and (5) The diffusion of STEM innovations. The ROLE Program aims to advance the knowledge base within and across the intersections of these multidisciplinary areas. It encourages projects that reconcile and integrate basic research and educational practice, and generate hypotheses from one disciplinary area that can be tested and refined in another.

About NSF and EHR

The National Science Foundation (NSF) is charged with promoting the health and vitality of the Nation's scientific and engineering research and education enterprises. As one part of that mission, the Directorate for Education and Human Resources (EHR) has primary responsibility for NSF's efforts to provide national and research-based leadership in science, technology, engineering, and mathematics (STEM) education. EHR maintains four goals in fulfilling this responsibility:

1. Prepare the next generation of STEM professionals and attract more Americans to STEM careers.
2. Increase the technological and scientific literacy of all Americans so that they can exercise responsible citizenship in an increasingly technological society and acquire knowledge of science, mathematics and technology that is appropriate to the development of workforce skills and life-long career opportunities.
3. Broaden participation (diversity) and achievement in STEM.
4. Attend to critical workforce needs requiring significant math and science skills and knowledge, both by attracting new people to these STEM careers and by support for the development and retooling of the current STEM workforce.

To reach these goals, the Directorate sponsors programs in the Divisions of Elementary, Secondary, and Informal Education (ESIE), Undergraduate Education (DUE), Graduate Education (DGE), Human Resource Development (HRD), the Experimental Program to Stimulate Competitive Research (EPSCoR), and the Division of Research, Evaluation, and Communication (REC). The Directorate also supports the Math and Science Partnership (MSP).

About REC

In pursuit of the four EHR goals above, the Division of Research, Evaluation, and Communication (REC) (http://www.ehr.nsf.gov/rec) seeks to:

1. advance research on science, technology, engineering, and mathematics education and improve evaluative research on STEM education programs;
2. increase the capacity of the field to conduct high-quality, innovative, useful, and credible STEM education evaluation and research studies; and
3. increase the capacity of STEM education researchers and evaluators to communicate the results of their research.

The principal grant programs of REC are Research on Learning and Education (ROLE), Evaluative Research and Capacity Building (EREC), the Interagency Education Research Initiative (IERI), and Faculty Early Career Development (CAREER). The EREC and ROLE programs are the subjects of this program solicitation.

About Human Capacity Development in REC

Like other REC programs, EREC and ROLE seek proposals that help to stimulate quality preparation of the STEM educational researchers and evaluators. Such projects may address the needs of undergraduate and graduate students and/or early- and mid-career researchers or evaluators, especially those who wish to transition from other science and engineering disciplines or those who wish to expand upon existing skills and knowledge.

Proposers are reminded of the second NSF merit review criterion. Proposals should address how the research activity will advance discovery and understanding while promoting teaching, training, and learning. Additionally, the proposed activity should broaden participation of underrepresented groups (be they researchers, graduate students, undergraduate student, K-12 students, or practitioners).

Proposers interested in undergraduate and graduate education should examine opportunities available through the Research Experiences for Undergraduates (REU) program and the Integrative Graduate and Education Research Traineeship (IGERT) program.

In addition, non-academic institutions are strongly encouraged to develop collaborative arrangements with academic institutions for the purpose of human resource development as appropriate to the goals of the proposed project.

A. EVALUATIVE RESEARCH and EVALUATION CAPACITY BUILDING (EREC)

Evaluation has gained currency throughout government and within the education enterprise as a part of a move toward greater accountability, oversight, and management of public resources. Until this time, however, evaluation has not been fully used as a research approach to generate knowledge about effective programmatic and policy features and strategies. While the community of evaluators who focus on STEM education remains small, the demands on it are growing. The EREC program is designed to support compelling evaluative studies that build the knowledge base about effective STEM education policy and practice, and to increase the size and capacity of the evaluation community to respond to evolving challenges in STEM education.

REC expects to support EREC projects with the intent that knowledge will be built through the diversity of disciplinary perspectives, methods, and approaches to these problems. Therefore, representation of investigators with expertise in evaluation and/or education research, social science, and other sciences and engineering is strongly encouraged.

In general, the EREC program does not support projects focused on the development of assessments of student learning. Investigators interested in this topic are encouraged to consult a program director in the ROLE program; the Interagency Education Research Initiative (IERI), the Teacher Professional Continuum (TPC) program; or the Course, Curriculum, and Laboratory Improvement program (the latter two programs are referenced in section IX of this solicitation).

EREC proposals may address topics in evaluative research, evaluation capacity building, or both.

a. Evaluative Research Studies

The EREC program seeks proposals that offer unique approaches to evaluation practice in the generation of knowledge for the science, technology, engineering, and mathematics (STEM) education community and for broad policymaking within the research and education enterprise. Successful proposals may focus on one or more STEM education programs or projects in order to examine major issues in STEM education.

The objective of the evaluative research proposals should be to expand understanding of educational practices, policies, procedures, and outcomes that can make a strategic contribution to STEM educational improvement, to policymaking, and to the improvement of evaluation theory and methodology. REC expects that these studies will serve as innovative and exemplary models for the STEM education research and evaluation communities. Therefore, investigators are strongly encouraged to focus on topics of national importance with the expectation that results may generalize beyond the scope of the research context and may be broadly useful for STEM educational improvement and the development of the evaluation field in general.

b. Evaluation Capacity Building

The EREC program supports projects that increase the capacity of the field to conduct high quality, innovative, useful, and credible STEM education evaluation studies. There are two eligible topic areas for evaluation capacity building proposals: enhancing capability and infrastructure and advancing the state-of-the-art in evaluation. (Further opportunities in support of capacity building can be found in section C of this solicitation).

Enhancing Capability and Infrastructure

REC will support projects designed to enhance the capability and infrastructure of the education field to conduct evaluations through education and training, the development of evaluation knowledge and skills, and through the creation of evaluation resources useful for the field in general. The following broad examples are provided for illustrative purposes only. Applicants are encouraged to develop focused projects in these or possibly other areas of relevancy to this topic.

• The development of professional communities focused on specific innovative evaluation approaches and practices via workshops, electronic networks, or by other means;
• The pre-service and in-service education and training of evaluators, with special emphasis on preparation of those groups underrepresented in science, technology, engineering and mathematics;
• The provision of training in evaluation to appropriate audiences such as STEM education program administrators, instructors, policymakers, and others.
• The planning, organization, and initiation of high quality undergraduate, graduate, and professional STEM education evaluation experiences, degree and certificate programs, and degree concentrations;
• The provision of education and training to researchers from other disciplines who wish to refocus their professional careers on STEM evaluation through such means as postdoctoral or midcareer fellowships and specialized programs;
• The creation or enhancement of infrastructures to support the practice of evaluation, such as regional professional groups or education and training consortia; and
• The compilation, critique, and dissemination of resources useful for evaluation practice.

Advancing The State-of-The-Art In Evaluation

Public demands on improving quality and access to STEM education are requiring new evaluation approaches and methods. REC will support projects designed to advance the state-of-the-art of evaluation by developing innovative tools, models, theories, and techniques that will assist the field in addressing questions of complex causality, attribution of cause and effect, and the impacts of various educational interventions on educational systems and learning environments. The following broad examples are provided for illustrative purposes only. Applicants are encouraged to develop focused projects in these or possibly other areas of relevancy to this topic.

• The synthesis of existing evaluation and research results from multidisciplinary perspectives, the development of meta-analyses, and the organization of conferences to seek clarity and consensus among disparate bodies of literature on methods for evaluating STEM education activities;
• The development of effective new mixed, quantitative-qualitative methodologies derived from multiple disciplinary traditions;
• The development of methods that might increase the validity and reliability of measures, address issues of complex causality, and/or enhance the ability of evaluators to make causal or attributional statements;
• The development or refinement of conceptual or theoretical frameworks for innovative evaluation designs of STEM education programs;
• The development of cost-effective approaches to evaluation or approaches that reduce the time required to obtain credible and reliable preliminary results;
• The creation of new models and approaches for disseminating STEM evaluation findings and methods to various stakeholder audiences; and
• The development of new methods for evaluating complex programs in STEM, including the use of mathematical models, qualitative or multidisciplinary methods, and measurement techniques.

B. RESEARCH ON LEARNING AND EDUCATION (ROLE)

Advances in many related fields have transformed research on learning and education in recent years. These developments have contributed to an emerging, multidisciplinary science of learning that bears directly on the educational and research goals of the National Science Foundation (NSF). New opportunities promise to improve educational research and practice, and to increase the level, quality, and accessibility of STEM education.

The ROLE program supports strategically important research studies that promote progress toward the four EHR goals appearing in the Introduction. The ROLE program seeks to understand how to produce significant improvements in STEM learning through a comprehensive approach that contributes to research frontiers in both human learning and in the educational environments and systems that are structured to support STEM teaching and learning. It also seeks to increase the capacity and breadth of the research communities contributing to these frontiers.

The ROLE program supports research across five emphasis areas (EA’s) to include:

1. The biological basis of human learning;

2. Behavioral, cognitive, affective, and social aspects of STEM learning;

3. STEM learning in formal and informal educational settings; and

4. STEM policy research, and

5. The diffusion of STEM innovations.

Research funded by ROLE will enable the community to effectively integrate research on learning into educational contexts and to build and strengthen cross-disciplinary communities of research. Each of these emphasis areas (EA’s) comprises broad research topics with their own distinct characteristics and historical foundations. The ROLE program aims to advance the knowledge base within and across the intersections of these multidisciplinary areas. It encourages projects that reconcile basic research and educational practice, and that generate hypotheses from one disciplinary area that can be tested and refined in another in the support of content learning in the STEM disciplines. These bridging features, connecting research and practice, and connecting disciplinary traditions and approaches, are distinctive characteristics of the ROLE program. ROLE seeks a balance of both “within-emphasis-area” and “across-emphasis-area” research in its portfolio.

Emphasis Area I. The Biological Basis of Human Learning

The effort to understand the relationships among learning, intelligence, and the human brain is one of the most fundamental and profound journeys of basic science. Converging lines of research have begun to reveal how learning affects the brain's structure, activity, and organization, from infant development through adulthood. Fundamental aspects of visual and spatial cognition, language, and mathematics are beginning to be understood in terms of neural processes and biological context. Discoveries in these and other areas are influencing our understanding of behavior, cognition, and the nature of human learning.

ROLE will support studies focused on human learning drawing on a wide range of theoretical approaches and empirical techniques, including but not limited to biological neural networks, computational neuroscience, cognitive neuroscience, functional imaging, neuroplasticity, and adaptive systems. An important aspect of these activities is to build capacity in neuroscience related to complex human learning and education, and to identify trajectories by which multidisciplinary research anchored in the biological basis of human learning can inform educational practice. Therefore, submissions in this emphasis area will necessarily demonstrate credible potential connections between their specific proposed activities and current research issues in STEM education as part of the scientific justification for seeking ROLE support.

Emphasis Area II. Fundamental Research on Behavioral, Cognitive, Affective, and Social Aspects of STEM Learning

The goal of this emphasis area is to enhance the multidisciplinary understanding of the foundations of human learning in the STEM disciplines. Models of general learning need to be tailored in service of content learning. The specific content should include science (including computer science), engineering or mathematics. Moreover, NSF seeks proposals that formulate compelling and innovative bridges from cognitive science either to biological basis of human learning (EA I) or to research on STEM learning in educational settings (EA III). ROLE strongly encourages multidiscipline, institutional, and researcher and educator collaborations to support the development of learning theories that explicitly consider the central and special place of scientific content in the theories of learning that are being developed.

Emphasis Area III. Research on STEM Learning in Educational Settings

Many educational approaches, curriculum materials, assessments, and technological tools have been developed to mediate the learning process without the benefit of a strong research foundation. In some instances, this is because the appropriate research does not exist. In other cases, this is because of insufficient exchange of information and knowledge among research, development and implementation communities.

All submissions to emphsasis area III should identify critical, practice-derived research questions and should provide a means for interacting significantly and in partnership with STEM educational practitioners. ROLE will fund studies that are exploratory (theory development and elaboration) and confirmatory (theory testing) in nature.

A principal expectation for research related to this emphasis area is to provide a stronger evidentiary base to support sustained improvement in STEM educational practice both in formal classroom settings and in informal learning sites (including the home). Consequently, whether the proposed work is exploratory or confirmatory in focus, the importance of the research question, the innovativeness of the research and methodological fit will weigh in the review. (Please see the methodology comments in Section C).

Emphasis Area IV. STEM Policy Research

ROLE emphasis area IV will fund studies that focus attention on policy questions pertinent to STEM student learning that occur at and across the various levels of educational systems: international, national, state, district, school and higher education, for example. Investigators should attend to interactions across levels where pertinent. STEM policy researchers will actively engage in building usable knowledge. Researchers are encouraged to investigate the causes, consequences, and performance of STEM-oriented public policies and interventions (or programs) that link to knowledge drawn from emphasis areas I through III.

C. COMMON THEMES AND ISSUES FOR THE EREC and ROLE PROGRAMS

Methodology

Proposals should reflect relevant advances in quantitative, qualitative, and mixed-methods research and evaluation methodologies and provide a compelling argument about how the methodologies proposed are appropriately matched with strategic research and evaluation questions. All proposals should develop and defend a logical chain of reasoning from evidence to theory and back again that is coherent, shareable, and persuasive to a skeptical reader (Shavelson & Towne, 2002). Finally the proposal should demonstrate how the methods chosen would result in rigorous, cumulative, reproducible, and usable findings.

Technology

EREC and ROLE seek proposals that advance our understanding of how a broad range of technology can enhance learning or help create more effective and efficient educational systems. ROLE recognizes that in order to study the effects of emerging and existing technology, developmental efforts are necessary. However, the development activities must be subordinate to the research on the efficacy of those technologies.

Knowledge Transfer

EREC and ROLE seek to accelerate the integration of high-quality research and evaluation findings into STEM educational practice and into the knowledge base. Proposals should discuss how the work would contribute to productive public or scholarly debate. As appropriate, proposals should describe mechanisms to effectively and efficiently transfer findings into educational practice. Requests for the preparation of critical literature reviews, workshops to develop new research networks and collaborations, and other forums to communicate results among appropriate constituencies are encouraged. In addition, proposals that focus on the potential utility of research and evaluation findings and their transfer into practice or use by other researchers and policymakers are encouraged.

Organizational Capacity Development

As appropriate to the goals of the project, EREC and ROLE seek proposals that establish and/or build on existing collaborative arrangements among graduate-degree granting institutions, two- and four-year predominantly undergraduate institutions, Historically Black Colleges and Universities (HBCUs), Hispanic Serving Institutions (HSIs), Tribal Colleges, and all institutions in EPSCOR states. Proposals should describe the nature of the collaboration, the anticipated effects of the collaboration and how it is likely to enhance the capacity of the collaborating institution. The proposal should include letters of support from the collaborator that outlines the role and value of the collaboration to the participating institutions.

Human Capacity Development

As noted in the introduction, EREC and ROLE seek proposals that help to stimulate quality preparation of the STEM educational researchers and evaluators. Such projects may address the needs of undergraduate and graduate students and/or early- and mid-career researchers or evaluators, especially those who wish to transition from other science and engineering disciplines or those who wish to expand upon existing skills and knowledge.

Proposers are reminded of the second NSF merit review criterion. Proposals should address how the research activity will advance discovery and understanding while promoting teaching, training, and learning. Additionally, the proposed activity should broaden participation of underrepresented groups (be they researchers, graduate students, undergraduate student, K-12 students, or practitioners).

Proposers interested in undergraduate and graduate education should examine opportunities available through the Research Experiences for Undergraduates (REU) program and the Integrative Graduate Education and Research Traineeship (IGERT) program.

In addition, non-academic institutions are strongly encouraged to develop collaborative arrangements with academic institutions for the purpose of human resource development as appropriate to the goals of the proposed project.

High-Risk Research and Evaluation

EREC and ROLE welcome innovative and groundbreaking proposals that address critical issues in STEM learning and educational research and evaluation, particularly in areas where the knowledge base is underdeveloped. Such proposals should contain a rationale and appropriate evidence for why the proposed research work is innovative or groundbreaking within the discussion of intellectual merit and broader impact in the one-page summary (and elsewhere in the proposal as appropriate). While innovative and groundbreaking research is encouraged in general, proposers should also examine section V.A. of this program solicitation and the NSF's Grant Proposal Guide for a description of the eligibility requirements for the Small Grants for Exploratory Research activity.

Adult Learning

ROLE seeks proposals that promise to build a stronger research base in adult workplace STEM learning and in other educational settings, such as e learning or distributed environments. ROLE adult learning should:

• increase knowledge through research that examines critical issues that affect program success,
• improve the research base to support changes in practice where appropriate
• provide leadership by bringing a research base and perspective to improving policy and practice.

Through an initiative jointly sponsored by the Division of Graduate Education (DGE) and REC, proposals on STEM graduate education are encouraged in the February 2005 ROLE competition (see the DGE and REC Dear Colleague Letter for more information).

ROLE seeks proposals that integrate research on STEM learning with areas of educational practice associated with programs in the EHR Divisions and especially welcomes research proposals that include collaborations with projects funded under such programs.

A catalog of current EHR programs and abstracts of funded awards appears at http://www.ehr.nsf.gov/award.asp.

Useful References:

Bransford, J., Brown, A. & Cocking, R. (1999). How People Learn. Washington DC: National Academy Press.

Kilpatrick, J., J. Swafford and B. Findell (Eds.) (2001) Adding It Up: Helping Children Learn Mathematics. Washington DC: National Academy Press.

Patton, M. (1986) Utilization-Focused Evaluation. Newbury Park: Sage. (Note: useful discussion of the definitions of evaluation and evaluation research).

Pellegrino, J. , N. Chudowsky and R. Glaser (2001). Knowing What Students Know: The Science and Design of Educational Assessment. Washington DC: National Academy Press.

President's Council of Advisors on Science and Technology (PCAST) (1997). Report to the President on the Use of Technology to Strengthen K-12 Education in the United States. Washington DC: Office of Science and Technology Policy, Executive Office of the President.

President's Information Technology Advisory Committee (PITAC) (2001). Using Information Technology to Transform the Way We Learn. Washington DC: Office of Science and Technology Policy, Executive Office of the President.

Shavelson, R. and L. Towne (Eds) (2002). Scientific Inquiry in Education. Washington DC: National Academy Press.

• Anticipated Type of Award: Standard or Continuing Grant
• Estimated Number of Awards: 15 to 30 (5-10 for the EREC annual competition, 10-20 for the ROLE competition
• Anticipated Funding Amount: Pending the availability of funds, $4 million for EREC; $12 million for the ROLE competition.

Estimated program budget, number of awards and average award size/duration are subject to the availability of funds.

Both the EREC and ROLE programs will fund studies up to a maximum of $1,350,000 over a 36 month period.

Where appropriate, both programs are willing to fund well-designed longitudinal studies for a period of 48 months at a maximum of $1,600,000.

Letters of intent should be submitted via FastLane.  The letters of intent should not be longer than one page and should include the following:

• Proposed project title
• The research questions to be addressed
• The STEM content to the investigation
• The population to be examined
• List of Pi's and Co-Pi's and their institutional affiliations

January 14, 2005
ROLE Letter of Intent (for 2005 competition only)

March 31, annually
EREC Letter of Intent

December 11, annually
ROLE Letter of Intent due December 11, 2005 and annually thereafter.

March 04, 2005
ROLE Full proposal (for 2005 competition only)

May 15, annually
EREC Full Proposal

January 10, annually
ROLE Full proposal for 2006, and annually thereafter.

If the deadline falls on a weekend or holiday, the proposal is due by 5 p.m. proposer's local time on the first business day thereafter.

The following programs may be of particular interest to EREC and ROLE proposers:

• Advanced Learning Technologies (ALT)
• Advanced Technological Education (ATE)
• Course, Curriculum, and Laboratory Improvement (CCLI), especially the Assessment of Student Achievement program (ASA)
• Human Social Dynamics (HSD)
• Innovation and organization Change (IOC)
• Instructional Material Development (IMD)
• Integrative Graduate Education and Research Traineeship (IGERT)
• Interagency Education Research Initiative Program (IERI)
• Research Experiences for Undergraduates (REU)
• Teacher Professional Continuum (TPC)
• Science, Technology, Engineering and Mathematics Talent Expansion Program (STEP)

See https://www.nsf.gov for the most recent program solicitations for these programs.