Program Evaluation - Board Recommendations and Contracts

Program Evaluation - Planning For Success
OR
How Do You Know if What You are Doing is Working?

Without evidence, it is impossible to know if your efforts are creating the type of  change you hope to see.  The Office of Shared Accountability hopes to provide guidance to assist you in this process.

It is important to select an initiative or program that is evidence-based, implement it in a way that can be studied, establish goals to determine if you have achieved success, and monitor your efforts to know if what you are doing is working.


What is Program Evaluation?

Program evaluation is a systematic method for collecting, analyzing, and using information to answer questions about projects, policies and programs, particularly about their effectiveness and efficiency.

What's the Purpose of Program Evaluation?
  • To provide information for improvement
  • Guide in decision making
  • Support accountability
  • Support effective practices 
Types of Program Evaluation

Formative (or Process) Evaluations
  • Assess to what extent were the various program activities delivered as planned.  This type of evaluations are geared to understanding how a program works.  
Summative (or Outcome) Evaluations
  • Assess impacts/benefits/changes to program participants (as a result program activities) during and/or after their participation.  The primary question being addressed by this type of evaluation is geared to understanding to what extent are the program activities yielding expected changes in participants.
Other Types of Evaluations 
  • Needs Assessments
  • Cost/benefit Analysis

For more information, please feel free to click here:  Program Evaluation 101


Program Evaluation - Planning For Success

Select an initiative or program with evidence in mind

Establishing evidence of need
  • Do a needs assessment.
  • Find data to support a need (for example, students in need of additional services, appropriate indicators, and where to find these services).
Where can evidence/research be found?
  • National research on strategies
  • Evidence from other schools or districts
  • Publishers of the approach
  • Instructional research or theory
  • Business literature 
Building a case
  • Pick a strategy and build a plan.
  • Plan to monitor implementation.
  • Establish indicators (Click here if you need ideas about indicators).
  • Determine the evidence you have that indicates the program or initiative is working.
How will you know when your project is successful?

Where are you going? How will you know when you get there?
  • Use tools such as the Logic Model (see blank template below) to help determine your desired outcomes.
  • Determine existing indicators of short and long term outcomes and the sources of each one. 
  • Consider a pilot with a strong evaluation design to evaluate outcomes - consult the OSA if you need help! 
Preparing your monitoring plan
Guiding questions
  • Who is the evaluation for?
  • What do we need to find out and why?
  • When will the findings be needed?
  • Where should we gather information?
  • How will the results be interpreted and used?
Content of A Monitoring (or Evaluation) Plan
Purpose and Scope of the Evaluation
    • What questions are being answered by the evaluation
    • Type(s) of evaluation to be undertaken
Methodology
    • Types of data/information that will be gathered.
    • How data/information will be gathered?  List of all instruments that will be used. 
    • How data/information will be analyzed? 
    • How is program success (i.e., attainment of the program goals/objectives) determined? 
Schedule of Evaluation Activities
    • Primary evaluation activities with completion dates or period
    • Evaluation activities must be are closely aligned with program activities
Evaluation Reports/Submittals
    • Identify reports, briefs, and other submittal that will be produced as a result of the evaluation.  Include due dates
    • Identify (i.e., use) of each Evaluation Information/Report 
    • Identify who will receive each evaluation report/submittal 
Dissemination of Evaluation Information/Reports
    • Describe how Evaluation Information/Reports will be disseminated to all stakeholders.
Designation/Qualification of Evaluators 
    • Skill and experience of the evaluation team
    • Experience/familiarity of the evaluation team with program being evaluated 
Formalize your monitoring plan
  • Following and tracking the participants in your program 
  • What are your key questions?
  • What are the key data sources you'll use
  • What tools are you going to use to monitor your success beyond the existing data you have (such as collecting surveys, checklists, reviewing pertinent documents or other records, observations, participation in the programs, timelines and responsibilities)

Still need help? Check out the links below. 

Presentations 
Division Heads - 2/9/17
Using Data to Drive Improvement by Genelle M(2/9/17)

Links for research



Related Files