Evaluation Skills For Managers And Supervisors.
The case for developing evaluation skills.
If organisations are going to meet the demands of the 21st Century, Managers and Supervisors must be able to make high quality, effective decisions. Perceptions of what is happening at any given time, both to and within an organisation, must be highly accurate.
Without accurate perceptions of the status quo, all communication and decision making is fatally flawed.
Therefore, Managers and Supervisors must be skilled in the use of theoretical frameworks and analytical tools, so they can take a precise picture, a "snapshot" of the work environment.
What is evaluation?
We can think of evaluation as the systematic gathering of information for decision making purposes.
Evaluation uses theoretical frameworks and analytical tools that will give Managers and Supervisors an accurate fix on their team's position in relation to their objectives. It will identify individual and team training needs and reveal the opportunities available for staff development activities.
Evaluation shines a powerful spotlight on things that are going well and things that need to be improved.
In 2002, I carried out an evaluation of training opportunities is a non-profit organisation. At that time the organisation had changed their leadership, their organisation structure and their ways of working. However, despite all these changes, they were still not achieving their objectives.
Evaluation revealed the need for soft skills, communication, teamwork and problem solving. However, none of the training opportunities available contained elements for soft skills development. In short, there was a "big hole" in their training program.
In this particular case, my theoretical framework was built primarily on knowledge about voluntary organisations and volunteers. The analytical tool used to collect qualitative data was a semi-structured interview with five main questions.
The results of this evaluation were so powerful, my perception of evaluation was instantly transformed. Evaluation was no longer an academic debate. It is a very powerful and highly effective tool for getting to the very heart of what is really going on in an organisation.
A few years ago I worked in local government, evaluating community based education programs for children, young people and their families. There were about thirty projects, running dozens of programs, for which I designed an overarching framework of evaluation based on the CIRO framework developed by Warr, Bird and Rackham in the '70's to evaluate Management Training.
One of the main reasons I like the CIRO framework is the way it begins with Context evaluation.
These four elements ask the basic questions:
- Where are we now and where are we going?
- What are we doing to get there?
- How are people reacting to what we are doing?
- How are things changing over time?
When organisations are investing time, money and resources into training and/or the change process, we will also need to ask the question:
- What is the cost/benefit ratio of our investment in the training and/or change process?
The days of doing things because they seem like a good idea, should be well and truly over. However, in my opinion, the amount of resources allocated to evaluation efforts are still woefully inadequate.
The failure to evaluate effectively is costing us not only time, money and resources. It is costing us our health and well-being, our wellness.
The failure to evaluate effectively results in frustration at least and despair at worst.
"Change Fatigue" and constant "Fire-Fighting" are a direct result of what I am talking about here.
Building theoretical frameworks.
Building a theoretical framework is rather like drawing a map. The more detailed the map, the better we can plan our journey. Once we have a highly detailed map then we can plot our current position, our intended destination and decide on the best route to get there.
Here's a list of subject areas I have found invaluable:
- Organisation Design, Organisation Behaviour and Organisation Culture.
- Scientific Management and Human Relations Movement.
- Change Management.
- Work-related Behavioural Styles.
- Teaching Strategies and Learning Styles.
- Stimulus-Response Conditioning and Cognitive Development.
- Perception and Gestalt Psychology.
- Self-talk, Self Fulfilling Prophecy and Learned Helplessness.
- Communication, Teamwork and Problem Solving.
- Staff Development and Staff Appraisal.
- Management and Leadership Styles.
- Goal Setting.
- Quality Systems and Quality Management.
By studying these subjects, together with those directly related to your organisation and it's overall purpose, it's possible to a highly detailed "picture" of the theory pertaining to your organisation's structure and ways of working.
Theoretical frameworks give us insights into what should and what should not be happening for optimal organisation performance.
The next step now is to identify what is happening, that is, we need to collect information that will give us a "snapshot" of what the current situation is. We will plot our current position.
Using analytical tools.
The typical way of collecting data is a questionnaire. If your are familiar the principles connected with questionnaire design and need to collect information from a large number of people quickly, questionnaires, especially those hosted "on line" can be extremely useful.
However, over the last few years I have found that conducting interviews and facilitating small group workshops has proved to be an extremely effective way of collecting very high quality (extremely useful) information. Ideally, interviews and workshops lead to participants experiencing an "aha!" moment.
This has happened with me many times now, such that I have come to say, "information is neither created nor destroyed, it is waiting to be collected."
For me personally, the greatest benefit of conducting interviews and facilitating workshops is that it builds relationship between people, individuals become "connected". I have found this to be true even when working with groups made up of individuals with conflicting ideas and interests.
Here is the outline of five goal related questions I use in interviews:
- These are the goals you are working towards, what do these goals mean to you?
- What needs to happen within the organisation for these goals to be accomplished?
- What knowledge and skills are you using to reach these goals?
- What problems and difficulties are you encountering?
- How will the training opportunities available to you help you reach the goals?
Can you see how answers to these questions will feed directly into staff development and staff appraisal (performance reviews) and give you far better insight into the experiences and thinking of your team members?
A well designed workshop will help individuals find common ground, those things upon which they can agree upon. It will provide a way for them to move forward together.
At this time I will not go into the specifics of my workshop design. This will need to be another article (or better still a video so that you can see how they work in practice).
The workshops I have used in the past have been built around the "Empowerment Evaluation" process developed by Dr. David Fetterman from Stanford University.
The subject matter of these workshops will vary according to the kind of outcome desired. Here are some subjects to consider using in "data collection" workshops:
- Working with core values.
- The Nuffield Partnership Assessment.
- The LEADS framework.
- Team/group/organisation objectives.
The basic premise of my workshops is as follows:
- Where are we now in relation to a specific behaviour/objective?
- Where do we want to be with this in 6 months time (set goals).
- What will we do over the next 6 months to achieve our goals (who, what, when, where, how?)
- In 6 months we will conduct this workshop again to assess our results and set new goals.
I have conducted many workshops along these lines and I have to say this; all the results were good and somewhere totally astounding. I have designed and facilitated some workshops, particularly amongst those with high levels of power and authority, with a great deal of fear and trembling. However, it was those workshops I feared the most that gave the best results.
I want to finish up this article, however, with a word of warning.
A word of warning.
Any Manager or Supervisor that wants to become a skilled evaluator will encounter "the system".
The system causes these things to occur:
- The people with the most to gain from the evaluation process will probably be the least likely to want to get involved or allow it to happen.
- People engaged in the evaluation process need time to learn and grow (produce results), there will be fuss. But the system always want results quickly and without any fuss.
- The system usually lacks the will and always lacks the resources (the knowledge, skills and attitudes) required for change. (This holds true for any change but particularly where evaluation is concerned).
I say these things because many people at the top and the bottom resist evaluation efforts. the people at the top don't understand it and the people at the bottom fear it.
Therefore, the Manager or Supervisor who wants to develop and use evaluation skills must became a champion for the evaluation cause.