Assess the success and impact of a program or project
Program planning and evaluation are a significant component of any type of outreach effort, especially if one’s role is the director of the outreach initiative. Individuals with the role of Science Outreach Director can work in a variety of settings e.g. nonprofit, academe or government.
Focusing on evaluation or assessment part of the planning process is important, because it is critical to know if the goals and objectives of the program are being achieved. Thus, each activity or event must be planned with a mission statement lens. For example, if one goal is to train scientists in the method and practice of teaching, several program activities such as a workshop should be geared towards this mission. The knowledge gained through evaluation also aids in acquiring resources through the development of a pitch and summary of impact. Such resources are volunteers and/or collaborators, equipment and/or monies in the form of donations or grants. Therefore, the tasks explored here involve skills that are useful to develop, and have the potential to profoundly effect on the success of a project/program.
Note: The in-depth program evaluation explored here is usually done by a professional evaluator. The evaluator is not typically program staff, but may represent a stakeholder.
The process below includes categories of information and activities that are necessary to plan and evaluate the effectiveness of a program. Steps 1-5 may be represented in a logic model. A logic model or framework is a tool used for the planning, implementation and/or evaluation of a project/program.
Program X partners local scientists and teachers to teach natural or physical science modules in kindergarten through 6th grade classrooms. Modules must includes hands-on lessons that are active, investigative and meet the Next Generation Science Standards (NGSS). The main goals and/or objectives of Program X are as follows:
Program X is funded for an additional two years and, you must make sure that the goals of the program are being met. The school district is especially interested in this evaluation; since, there is an opportunity to include additional schools in the upcoming academic year.
For this exercise, you are the Executive Director of the Science Education Consortium that serves several elementary school districts and your task is to evaluate Program X.
|Task 1: Critique of a logic model
Critique of a logic model
A logic model, program matrix or logical framework is a graphical depiction of the relationships between program elements. These elements include inputs, outputs and outcomes. Logic models are used for planning, executing and properly assessing a project/program. Before starting this task, familiarize yourself with examples of logic models from informalscience.org, such as “A Start to Developing a Logic Model” and “Logic Model for Agricultural Literacy Programming“. Below is a simplified logic model for Program X containing program inputs, outputs, outcomes and evidence. The model does not includes the program goals/objectives (see “The exercise”), a summary of external factors or any assumptions.Your job is to correct the logical model and provide a rationale for your edits. Critique the model by answering the following questions:
Inputs and Resources Outputs Outcomes Evidence
After completing this task, check your corrected logic model with a sample corrected logical model provided. How do they differ? Logical models will vary depending on the author and place of employment there’s no right or wrong answer.
|Task 2: Evaluate data for Program X
Evaluate Data for Program X
During the previous school year, Program X partnered 37 scientists with 18 teachers in 15 teams that served approximately 600 students. Evaluation data for Program X is available below:
Keeping the program goals in mind (from “The Exercise”), examine the data and answer the following questions:
Write a summary that contains the information above. This summary would be presented in a meeting to your staff that implemented Program X. This task represents steps 7 and 8 of the process outlined above. Step 6 has been hypothetically completed and provided for the purpose of this simulation.
|Task 3: Prepare a report
Using your results from Tasks 1 & 2, create a report that includes:
This report is what you would present to the school district or the sponsors. This task represents step 8 and 9 in the process outlined above.
For Task 1, edit the logical model and write a few sentences explaining your changes. This logical model would be discussed with your staff and, once finalized, become part of a grant application.
Task 2 requires a written summary but, you may choose to present the data in a drawing, graph, etc. as appropriate. This summary should contain all of the data and, any accordant commentary.
Write a brief report for Task 3. Remember, your audience is the school district,potential collaborators, and sponsors. The report should be broad, clear and concise.
Full reports can be lengthy, for the purposes of this simulation, the report should be no more than 2-3 pages and include the following:
All of these tasks can be completed in a word processor.
General resources to help you get going:
Also, view this example of program evaluation from the viewpoint of a funding agency.
For this task:
Needed in this career:
A professional in the field of regulatory affairs may also perform these activities:
Simulation author: Corin White, PhD
Simulation vetted by professionals in the Bay Area