1

Writing Research and Evaluation Plans for NSF Grants: How are they similar and different?*

by

Kavita Mittapalli, Ph.D.

In general, several National Science Foundation (NSF) solicitations in the Directorate for

Education & Human Resources including:

may require separate research and evaluation plans. It is important to be able to distinguish

between the two and write both sections well for a competitive proposal.

Research vs. Evaluation

Several fellow researchers and evaluators have done a tremendous job of explaining the

fundamental differences between the two including: John LaVelle (AEA365 blog post ); Sandra

Mathison’s book chapter; and Patricia Rogers’ Better Evaluation blog post.

Research asks: What’s so? How does it work?

Typically, results are reported at the end of the research project and are meant to inform the

field of study.

The evaluation aspect of an NSF grant particularizes as it is designed to improve the program

itself.

Evaluation assesses the value or merit of the program and asks the So what? and How well does

2

Let’s dissect a typical research plan.

A research plan typically comprises:

Hypothesis: The hypothesis can examine the project’s/intervention’s outcomes to measure

participants’ achievements in STEM courses, STEM teaching and learning, engagement, training,

If/Then)

Research Question: The research question answers questions such as:

the feasibility of undertaking a complex study, time, and budget among other factors.

Analytical techniques: Analytical techniques comprise the types of analyses the researcher will

undertake to derive findings from the study. Qualitative data collected via interviews, focus

groups and other means would require coding the data to develop common themes to build

narratives. Quantitative data can be analyzed using descriptive or predictive analyses which

would depend on the data quality and quantity (e.g., sample size) and outcome measures that

may require more complex analytical techniques.

Let’s dissect a typical evaluation plan.

3

Evaluation Approach/Design: Among other options, approaches include formative and

summative evaluations.

the attributes of the program such as adherence, delivery, and quality as proposed and

provides just-in-time feedback to help improve the program in its formative stages of planning

and implementation.

Summative evaluation, usually completed in the latter years of the grant, focuses on the

type and number of outputs or products as well as if the intended outcomes were achieved and

how well. It also assesses the model of the program—its merit and broader impacts insofar as

its viability to sustain beyond its funding cycle and/or scale up.

Data Collection & Analytical Techniques: Data collection sources as well as analytical

Always plan to engage a researcher and an evaluator early and often!

In some cases/tracks, principal investigators (PI) may select to have an advisory team/board,

instead of an external evaluator, conduct the project evaluation in addition to serving in an

advisory role.

Let’s talk about the selection process, roles, and functions of an advisory team/board when that

o Who will serve on the team/board? (e.g., community members, faculty, K-12 teachers,

o Why do you need an advisory team/board instead of or in addition to an evaluator?

4

o What are their roles/functions in the project? (e.g., purely advisory, rubber stamping

o What is in it for them and you? (e.g., what is the mutual benefit of engaging the

o What is the nature of work the team/board will do? (e.g., clearly defining their roles,

Once the advisory team/board is selected, outline their specific roles and expectations and

stipends/compensation clearly in the budget narrative.

In cases when there is a separate advisory team/board and an external evaluator, it is

important to understand their different roles, functions, and contributions. External evaluators

and advisory board members generally contribute to the accountability component of project

management and evaluation(s). Both parties may be used to provide the project team with

feedback (on both as-needed and annual bases) on a range of issues, including the project’s

implementation, research/evaluation methods and analysis, dissemination processes, how to

improve engagement with a project’s target audiences, and how to effectively bridge

*An earlier version of this piece appeared in the August issue of HSISTEMHub at New Mexico

State University and t can be accessed here: https://hsistemhub.org/portfolio-item/august-

Useful evaluation resources:

W.K. Kellogg Foundation Logic Model Development Guide,

https://www.aacu.org/sites/default/files/LogicModel.pdf

Reimagining S-STEM Project Assessment and Knowledge Generation, webinar presented by Kavita

https://www.sstem.aaas.org/s/November-2020-S-STEM-Webinar.pdf

5

very diverse backgrounds, academic training, and professional experiences to MNA. To date,

MNA has evaluated 32 NSF grants in 16 tracks (e.g., ATE, AISL, CAREER, Conference, CREST, ECR-

CORE, ERN, HBCU-UP, HIS, IUSE, INCLUDES/DDLP, ITEST, MSP/STEM C, Noyce, PREM, and S-

STEM) in addition to evaluating several medium to large grants funded by other agencies (e.g.,