Home   |   Join the NSPA   |   Sign In


Start Up Program Management Program Enhancements  Accountability


Toolkit Welcome

Program Evaluation Overview

Reporting to Stakeholders

Starting a Program Evaluation

Data Tracking

Tutorial on Assessment Plans

Program Renewal Through SWOT Analysis

Reporting and Records Retention


Program Evaluation Overview

Program evaluation is the process of collecting and analyzing data to assess the effectiveness and impact of program offerings and to measure progress toward achieving programmatic goals. Evaluation can be conducted while a program is operating to inform adjustments and improvements. This type of evaluation is called formative evaluation because it helps shape and refine program design. Summative evaluation is performed at the conclusion of activities or programmatic cycles to measure overall program impacts, and perhaps to determine if the model could be expanded or replicated. Conducting evaluations of their services helps scholarship programs demonstrate the impact of student financial assistance to donors and potential donors, thus building the case for more support. Evaluation also helps scholarship providers adjust programs to better meet students’ needs as they learn about the effectiveness of their efforts, which maximizes the return on each dollar invested in the program.

Defining Success

Success can be measured only if it is clearly defined. Before beginning an evaluation, program planners must detail what success means to them by setting goals. What does the organization intend to achieve by providing scholarships? If the organization has not already answered this question, it should convene a group of key stakeholders, including staff and board leaders, and possibly community members and students as well, to consider the question together. 

Developing a Logic Model and Evaluation Plan

Once the goals for the scholarship program have been clearly defined, it is helpful to think about how the program produces results. Developing a logic model will help clarify the relationship between what a program does and what it intends to achieve. A logic model is a graphic representation of how the resources and activities of a program are related to results. Logic models can take many forms, but a basic framework is shown below: 




Short-term Outcomes

Long-term Outcomes/Impact

What resources are necessary to operate the program?

What will be done with those resources to achieve the program goals?

What are the quantifiable deliverables or products of those activities?

What short-term changes occur as a result of the program activities?

What long-term changes occur as a result of the program activities


When developing a logic model for a scholarship program, providers should engage a group of stakeholders to build the model together. Ideally, this is done during program planning, but it can also be done retroactively. The group should begin by agreeing on a timeframe that the logic model will represent to establish parameters for the scale and scope of responses to the following key questions:

  • What resources are necessary to operate the program? When describing program investments, consider who is involved, what financial resources are necessary, where the work will be done, and what tools will be used.
  • What will be done to achieve the program goals? When detailing the strategies and action steps, consider what services will be offered, what key functions the program will carry out, and what processes will be conducted.
  • What are the quantifiable deliverables of those activities? When listing the program’s results or products, consider the quantity and frequency of services or tangibles to be delivered. For example, the provider might measure this by the number of participants who have been served or the number of hours of support provided by staff and volunteers.
  • What short-term and long-term outcomes or changes occur as a result of the program activities? When outlining the program outcomes, consider the changes that will occur as a result of the program. What will the program achieve? What will the impact be?

Each component of the logic model should be connected with the rest of the framework. Outputs should be produced with specific activities and resources, and they should lead logically to certain outcomes. When completed, a logic model should read as a series of “if … then” statements in which the resources, activities, outputs and outcomes are logically related. If the necessary resources are available to conduct program activities, then the program will deliver services that result in the desired changes and accomplishments.

Developing an Evaluation Plan

Once the logic model is completed, it will serve as the foundation for developing a plan to determine if the program is achieving its intended outcomes, identify potential improvements, and measure impact. An evaluation plan outlines key questions about the program and the strategies for assembling evidence to address the questions.

Tracking and Analyzing Data

The ability to measure the progress and impact of a scholarship program depends on the quality of data available. The reliability of the evaluation efforts will depend on the accuracy and integrity of data used in analyses. Therefore, meaningful evaluation efforts must begin with carefully maintained student records. The evaluation plan should help clarify the information needed to answer key questions about the program. Once it is established what information should be tracked, providers can create standard procedures for collecting information and entering data into a database to help maintain consistency in student records. Consistent data will allow providers to perform meaningful analyses, comparing apples to apples.

See the article on database creation in the start-up section or the article on data tracking from the College Access Foundation.

Source: Louise Myrland, Scholarship Program Director, Denver Scholarship Foundation



 TGThe Scholarship Program Toolkit is made possible by a TG Public Benefit Grant.

Thank you to the following organizations for their generous support!

Membership Software Powered by YourMembership  ::  Legal