EU projects in the eyes of external evaluator

Evaluation model

The session was opened by Anita Sandahl who described evaluation as the dilemma of two aspects. The process of evaluation was summarised as:

"An art to sign value, not simple and straight forward, finding out what is the meaning of the events."

She meant that an evaluator has to decide how far into the project he or she can go without affecting the actions of the participants too much. The answer to this question has to take the purpose of the evaluation into consideration. The purpose could for example be to spread the knowledge gained in the project, to help the participants to gain insight about the project or perhaps to influence the project in a certain way during the process. This is all connected to the role of the evaluator and the preferred distance between the project and the evaluator. Anita Sandahl concluded by referring to Aristotle’s’ concept of practical wisdom and suggested that evaluation is very much a practical wisdom.

The design of this project evaluation model is a bit different compared to the traditional model. A unit of analysis, i.e. an answer to the question: Whom do we do this evaluation for? was chosen – the project participants. A description and evaluation of the project from their point of view would probably be useful to most stakeholders.

The aim of the responsible for evaluation o this project is to describe what happens during the process and the values assigned to the process. The goals formulated in the beginning of the project are interesting since they are part of the process, but so are also any new goals formulated during or after the project.

• There were several interesting questions raised during the sessions. Some participants agreed with the description summarised above and suggested for example that evaluation against old goals in a process that involves learning is useless. A suggestion like that implies that a process involving learning will most likely lead to the development of new objectives. Other discussions were directing to the issue of whether evaluation is possible at all. This is probably dependent on the definition of evaluation. It was concluded that depending on both the evaluators and the interpreter or reader of the evaluation document, the evaluation of the same project can take many different forms. A project is most likely to look different from different perspectives.
• Other participants during the session were more sceptical: If I just look at a process – do I then evaluate? We explained that we do some of the things suggested in the traditional model also. We do this because we think it is of interest for the participants as well as for the European Social Fund, but we also complement these kind of descriptions of the project with a description of the process, since we think that this is even more crucial for understanding the project. More important, we do not just document the process; we also try to document the values assigned by the participants to the process. Someone concluded that measurable goals are important on some levels and that detailed process descriptions are important on other levels of an organisation.
• There were also discussions regarding the role of the evaluator. Someone suggested that the evaluator should stay objective and interfere as little as possible. We argued that it is not possible to even watch something without affecting what is happening.

• Some project participants present said that it is all right for the evaluator to interfere to a certain degree. There were also discussions about objectivity as such, as well as about usefulness of process documentation in business firms.