Successfully Evaluating Educational Programs

The demand for the evaluation of vocational education programs arises from several sources.

• The youth and adults who enroll in vocational education programs;
• The parents of those youth;
• Those at the local, state, and federal levels whose dollars support the programs; and
• The employers who hire program graduates.

This article focuses on locally directed and product-oriented program evaluation. This article is designed to give you skill in planning and conducting vocational education program evaluation efforts.

The Nature of Program Evaluation

Program evaluation is the task of making judgments about the worth or value of a program.

Reasons for Program Evaluation

The fundamental reason for evaluating vocational-technical programs is to seek program improvements.

Who Should Evaluate?

Vocational-technical programs are continually evaluated by the people they serve. Evaluations are made by educators, taxpayers, parents, students, and other people affected by the vocational education program. By using a structured approach to gather evaluation data from those affected by and involved with the educational program—instructors, administrators, supervisors, counselors, students, parents, alumni, employers, and employees—you can ensure that you get an accurate overall picture of program strengths and weaknesses.

Involving Key Groups

The first group to involve is the school or college administration. In post-secondary institutions, the group could include the department chairperson, the dean of occupational education, and the college president.

A second key group to involve might be called a staff steering committee. Members of the steering committee can provide direction and assistance for the evaluation effort by performing such tasks as the following:

• · Devise the overall evaluation plan
• Develop or update a philosophy statement for the vocational program

Assess Needs

↓

Develop Philosophy

↓

Develop Goals and Objectives

↓

State Criterion Questions

↓

Collect Data

↓

Analyze Data

↓

Formulate Recommendations

↓

Make Decisions

↓

Assess Needs

Assess Needs—A needs assessment should tell us the difference between where we are and where we need to be.

Develop Philosophy—The basic guiding beliefs of the school or college, about vocational-technical education, need to be formally presented in writing.

Develop Goals and Objectives—Program goals and objectives should be based on the results of needs assessments, and they should reflect the statement of philosophy.

State Criterion Questions—The development of criterion questions for each objective requires a good deal of time and should involve several people who are keenly interested in the program. A criterion question is one that states the objective in such a way that an answer is called for that would help to measure the attainment of the objective.

Collect Data—The type of demographic data that can usually be obtained by searching existing records are as follows:

• Student demographic data, including grade point average, courses taken, attendance, address, aptitude and intelligence test scores, and placement records.
• Teacher demographic data, including years of occupational and teaching experience, educational preparation, and field experience.
• School or college demographic data, include enrollment in classes, average daily attendance, and programs offered.

Distribution of Final Report

Results of the evaluation effort should first be reported to the school or college administration and the governing board. The school administration and governing board should determine the individuals and other groups to which the final report should be released.

I. Abstract

Occupies one or two pages; identifies the school; and concisely and simply summarizes the objectives, findings, conclusions, and recommendations.

II. Introductory Section

A.Introduction

Vocational Program: This section should give a description of the institution’s vocational program. It may include statements on the institution’s vocational education philosophy and objectives.

B. Objectives of Local School or College Evaluation

This section should include a clear, concise listing of the objectives established within the framework of the evaluation. These objectives give direction to the institution’s evaluation effort.

C. Description of Activities

In this section, a detailed description should be given of each activity included in the evaluation. A time schedule and other materials may also be included.

D. Staff Involvement

This section should present information focusing upon the extent and type of staff involvement. It should address the outcomes, attitudes, type of committee, functions, philosophy changes, successes, failures, and so on.

E. Citizen Involvement

Facts about citizen involvement in the evaluation are presented in this section. It reports types, size and representation or committees, functions, attitudes, outcomes, successes, failures, and so on.

F. Consultant Involvement in Project

This section includes information focusing upon the role of any consultants in the evaluation.

G. Public Relations

This section deals with the type, method, and scope of public relations used in connection with the evaluation.

A. Committee Activities

This section reports the number and type of committees and the scope of activities of each.

B. Committee Recommendations

The recommendations of each committee are presented in this section. Each recommendation should be accompanied by supportive data and other information.

C .Follow-up Studies

If employer surveys and follow-up studies of graduates and dropouts were made, the results and generalizations are reported in this section.

D. Curricular Studies

If studies of present or potential curricula were made, the results are reported in this section.

E. Other Types of Studies

If occupational surveys, attitude studies, or other types of studies were made, the results are reported in this section.

F. Changes Resulting from or Related to Local Program Evaluation

Any changes, innovations, adoptions, or plans already made are reported in this section.

IV. Conclusions and Recommendations

A. Conclusions

The conclusions should be organized according to the objectives that were drawn up for the evaluation.

B. Recommendations

All recommendations and future plans for evaluation should be reported in this section.

V. Appendix

A. Questionnaires

B. Checklists

C. Committee membership lists

D. Public relations materials