Evaluation policy (April 2023)
About This Policy
NSF’s Evaluation Policy outlines the key principles that guide evaluation activities conducted or supported by NSF. It is aligned with NSF values of scientific leadership, diversity and inclusion, integrity and excellence, public service, and innovation and collaboration (NSF Strategic Plan, 2022) and informed by legislation and guidance regarding federal evaluation and performance management activities, including the "Foundations for Evidence-Based Policymaking Act of 2018," the "Data Quality Act of 2001," and Office of Management and Budget guidance (OMB M-20-12).
About the National Science Foundation (NSF)
NSF was created “to promote the progress of science; to advance the national health, prosperity, and welfare; to secure the national defense” (1950, as amended).
NSF seeks to achieve these goals through an integrated strategy that advances the frontiers of knowledge; cultivates a world-class, broadly inclusive science and engineering workforce; expands the scientific literacy of all citizens; builds the nation’s research capabilities through investments in advanced instrumentation and facilities; and supports excellence in science and engineering research and education.
NSF is committed to evaluating the efficacy and efficiency of its strategy, leveraging evaluation to help the agency achieve its mission.
Scope of Evaluation Activities
An evaluation is “an assessment using systematic data collection and analysis” (OMB M-20-12). Assessments may have different targets—an intervention, program, policy, and so on—and serve different purposes, such as monitoring progress, guiding improvement efforts, or determining the effectiveness or efficiency of the target intervention, program, or policy.
The principles described here therefore apply to a wide range of activities conducted as part of efforts to generate evidence useful for decision making. These include policy studies, performance measurement, and descriptive and exploratory analysis (also known as “foundational fact-finding,” OMB M-20-12).
Throughout the NSF’s Evaluation Policy, the term “evaluations” refers to all of these evidence-building efforts and “evaluators” and “researchers” refer to individuals involved in these efforts.
Evaluation Principles
Six principles are presented in NSF’s Evaluation Policy:
1. Relevance
2. High Quality and Rigor
3. Independence
4. Transparency
5. Ethics
6. Equity
These principles align with and affirm NSF’s commitment to federal evaluation standards and leading research practices.
Selected enabling practices are included to showcase examples of NSF practices that foster implementation of these principles.
Researchers from inside and outside of the foundation who are participating in NSF’s evidence-building activities are expected to adopt these principles in order for findings from their work to be useful for stakeholders. They are also expected to balance decisions judiciously in addressing tensions or conflicts among principles, if any arise.
Enabling practice:
To ensure relevance and utility of evidence-building activities, evaluators must collaborate with stakeholders in different phases of the work, as appropriate. These stakeholders include leadership supporting the effort, program officers who may use findings to inform decisions, and others.
Evaluations must produce credible evidence—findings that NSF and its stakeholders can confidently rely upon. To produce such evidence while balancing constraints like time and resources, evaluations must rely on the most appropriate designs and methods to answer key questions; provide clear explanations of limitations; ensure appropriate execution of the study design with quality controls and regular risk monitoring; and promote proper interpretation of findings.
Enabling practice:
To develop high quality evaluations, when appropriate, NSF-supported studies shall begin by developing a logic model (based on a theory of change), which provides a framework to understand and communicate how a program or intervention is hypothesized to yield expected results
Enabling practice:
NSF evaluators and collaborators shall follow NSF procedures to identify and prevent conflicts of interest and engage in peer-review and quality control processes at appropriate stages of the life cycle of an evaluation.
NSF promotes transparency in the planning, implementation, and reporting phases of evaluation activities to promote dialogue that enhances quality, enables accountability, and prevents tailoring that influences findings. Transparency is crucial to support reproducibility and contribute to advancing knowledge. Whenever possible, completed evaluations will be released in a timely manner and with sufficient detail to support use of findings (including comparability to the existing literature) and replication.
Enabling practice:
To the extent possible, decisions regarding the specification of the evaluation—such as objectives, design and methods, and dissemination of findings—will be documented ahead of time, taking into consideration and abiding by any legal, ethical, national security, or other constraints to disclosing information.
Evaluations supported by NSF will be conducted with the highest ethical standards to protect the public and maintain public trust in the agency’s evidence-building efforts. Evaluations shall be planned and implemented to safeguard the dignity, rights, safety, and privacy of participants and other stakeholders and affected entities, abiding by regulations governing research involving human subjects. Evaluations will be equitable, fair, just, and consider contextual factors that could influence the findings, their interpretation, or their use.
Enabling practice:
Researchers engaged in studies supported by NSF shall submit their work for quality review before use or release to ensure ethical and other principles are upheld.
NSF is committed to supporting evaluations that contribute to the common good and advance an equitable and just society. To this end, evaluators must seek to include the perspectives of diverse populations, particularly from populations that are underrepresented and/or underserved. Relevant data collection efforts should be rigorous, inclusive, and culturally appropriate. Appropriate methods should be used to produce credible findings applicable to broad and specific populations, whenever possible.
Enabling practice:
NSF evaluators and collaborators shall include diverse perspectives during all phases of the work, including in the interpretation of data and dissemination of results. NSF evaluators and collaborators shall seek to understand the effect of the program or policy being evaluated on different populations.
