Project results
The leaflet Nr. 1 of the “Reflective Evaluation” project can be downloaded here
Introduction
The Reflective Evaluation project has the overall aim of developing and testing tools for supporting self evaluation by teachers and trainers. Such self evaluation should lead to empowerment by teachers and should improve the quality of continuing vocational training. The project brings together parters from five different countries. One crucial theoretical underpinning idea for the project is that of mutual learning, that each parter can learn from the others in terms of processes and practices in different countries.
In order for such mutual learning to take place we need a common understanding of the concepts and ideas which we are developing through the project. We are aware that these concepts reflect different cultural backgrounds and social and historical developments in different countries. Even when we use common words - such as competence - the understandings and meanings of these words may differ greatly in different European societies and education and training systems.
Therefore the first task for the Reflective Evaluation project - and the first group of outcomes - has been to develop a common concept of self evaluation quality (assurance) and empowerment evaluation.
Each parter has produced a short paper elaborating their understanding of the concepts of quality, self evaluation and empowerment evaluation. These papers are available for download on this page.
Papers
Title: First National Report, Denmark: Quality through Self-evaluation and Reflection
Authors: Jørgen Ole Larsen and Lisbeth Højdal, DEL
Abstract:The concept of quality is both manifold and dynamic. The assessment of what to label as “good quality” is both linked to the general political objectives, which obviously are constantly changing and more specifically to the nature and aims of the individual education programme. This additionally subject to changes in the course of time and in response to changes in demands from society. Finally the perception of good quality is depending on available resources.
The focus on quality has been high on the Danish political agenda during the last 10 years seen both from a centralised and decentralised perspective. It is in this respect characteristic for the Danish system that the quality concept hasn’t been classified by general definitions but rather encircled by indicators and approaches to maintain and develop quality.
Hence a modern quality concept includes both the process and outcome. Quality in the process depends - in the narrowest sense – to the specific targets for the education, while a more broad quality concept also embrace the outcome of the process including the fulfilling of the general objectives for the education.
Length: 5 pages
Download PDF here.
Title: Quality through self-evaluation and Reflection: The Greek Context
Author: Nikitas Patiniotis, Dr Theodora Asimakou
Abstract:The main challenges a modular system has to address are:
a) The significant role trainers play in making the modular system successful, as it requires changing from didactic teaching towards the organization of student’s learning; the power relations now change, and the trainer is there to support and facilitate learning.This change requires their commitment to the novelty of new pedagogic methods.
b) Trainees need to change the way they see learning in the modular system, since it expects self-direction, self-regulation and active participation. In the Greek context, as presented above, participants in further training programs lack substantial motives to become actively engaged, and are satisfied with the subsidies given by the responsible bodies for participation.
c) Finally, great attention is needed in designing sound quality assurance procedures, since modular systems can lead to fragmentation and incoherence of programs; In the Greek context, the VET system is already fragmented and incoherent in terms of the programs, providers, and supervising bodies, hence this third point presents us with a very likely danger.
Download: RE_FCR_el.pdf
Title: Reflective Quality Development For CVT Teachers and Trainers Through Self-Evaluation
Author: Ira Michel
The paper examines not only the relation between Self Evaluation and Quality Management, looking at Quality Management in the education sector, quality management defining self-evaluation, and concepts of Self evaluation and empowerment it also examines quality development through self evaluation and reflection with a new European self evaluation tool.
Length: 26 pages
Authors: Sandra Santos e Silva and Eduardo Figueira
Abstract:In this report we tried to summarise our understanding about the concepts of quality, self-Evaluation and Empowerment-Evaluation, in order to contribute for the elaboration of the partnership common understanding.
Concerning the education and training system we can define several types of evaluation: Self-Evaluation, training Evaluation, evaluation learning process, evaluation of effects and impacts, evaluation of reactions, diagnosis evaluation, return on investment evaluation, evaluation ex-ante, evaluation ex-post, external evaluation, internal evaluation, on-going evaluation, qualitative evaluation, quantitative evaluation, etc. (INOFOR, 2002)
Having into consideration the definition of evaluation, how can we define Self-Evaluation? We have self-evaluation when the evaluator is at the same time the object of evaluation. Being the evaluator object of evaluation by himself, this process of evaluation as to be much more oriented in a reflexive way.
Self-Evaluation can be defined as following: Self-evaluation is when the author of the action is at the same time the evaluator of that action. (Regnier, 2002).
Length: 15 pages
Download Re-FCR-P.pdf
Title: Second National Report Germany
Authors: Irina Michel and Manfred Sallach
Abstract: At a cursory glance, the project Reflective Evaluation leads to a confusion for those who get across the concept of the newly constructed self reflection instrument. The measurement of quality and result oriented quality objectives is not part of the methodology of this reflective quality development tool.
National and international discussions put a tremendous weight on the aspect of quality assurance. The different characters of the European Quality Assurance Framework (CAQF) are based on the statement of output oriented quality characteristics on the system level and on the level of the VET provider (cf. EC 2004; http://www.qavet.com). The project Reflective Evaluation, however, aims at creating an instrument which will promote self evaluation for quality development based on a set of reflection methods on four levels of reflection (system, institution, interpersonal, individual level) and six quality development areas (Work Conditions, Teaching & Learning, Values & Roles, Assessment & Recognition, Self evaluation & Quality Development, Networking & Collaboration). A new conception of the development of quality was considered necessary for the construction of the self reflection instrument. This reflection instrument is aiming at a reflection oriented design of quality development. It is meant to support teachers and trainers working in CVET to reflect on their teaching/training quality. In other words: The reflection on teaching/training quality is the crystallization field of the instrument for self evaluation and reflection. As a main product of the project Reflective Evaluation the reflective quality development instrument and its new conception of the development of quality is in the focus of this second country report. This report gives a detailed description of the test phase results of the prototype tool and informs about the organisation of the test phase and its methodology.
23 pages
Download: Second German Report
Title: Second National Report Denmark
Authors: JØRGEN OLE LARSEN – OLE DIBBERN ANDERSEN
Abstract: Both the background for establishing the data base and the context of the testing was influenced by a number of trends that are seen as important for the whole evaluation culture in both Denmark and Europe.
The context of the testing of the tool was influenced by the general discussion of how evaluation should be seen and adhered to. On a very general level you can divide it into often contradicting perceptions: One the one side self evaluation is primarily seen as summative and control based focussing on quantitative indicators as part of QA system as highlighted above. This is often combined with benchmarking exercises between school activities. On the other side self evaluation is first and foremost seen as a development tools and process oriented.
The self evaluation was by the persons testing the tools seen as development tool both from an individual and collective perspective.
Our testing started in august 2006 and has been carried out on different kind of vocational schools in Denmark with a wide range of educational activities primarily in the field of health care and transportation. The testing was connected with more general interviews/discussions how the evaluation culture can be improved at the culture of evaluation. This also led to interesting conclusions that seem to be very general for the how self evaluation in Denmark is perceived. The reason for including them in report is that they have to be observed when you want to introduce and implement the tool into a wider context.
19 pages
Download: Danish national report
Title: Second National Report Greece
Authors: NIKITAS PATINIOTIS AND THEODORA ASIMAKOU
Abstract: In the course of the Leonardo da Vinci project “Reflective Quality Development for the CVT teachers and trainers” a computer-based tool has been developed; the tool aims at the supporting practitioners (esp. teachers, trainers, and teachers’ trainers) in the process of self-evaluation and developing further the quality of their work. The tool is based on the assumption that people need the time to think of issues related to their work life; thus, the tool sets questions that intrigue the user to consider various aspects of their work situations and possible ways for improving them. The tool does not aim to act as an informative guide or test of knowledge, but clearly to stimulate people to think on issues that they consider important. It is thus, an interactive tool, which gives users the possibility to choose which questions they would like to reflect on, and most importantly, to develop their own questions –hence customize it to their own needs. This documents reports on the testing of the tool.
14 pages
Download: Greek national report
Title: Second National Report Portugal
Authors: GUIDA LOURENÇO & EDUARDO FIGUEIRA
Abstract: This second report is based on the tool which was built accordingly with purposes of Reflektive Evaluation Project. This has the aim of helping teachers and trainers to reflect about their practices and futhermore improve the development of quality in CVET.
The tool is constituted by six dimensions or working areas: valores & Roles, Teaching & Learning, Self-evaluation & Quality Development, work conditions, Assessment & Recognition, Teaching & Learning and each these dimension it is divided in four levels: individual, group/team, institutional and system.
So this report pretends summarize the results of testing phase made to the teachers/trainers and also to the responsible that belong to our partner institutions. Therefore this report is the evaluation of the testing phase, it has comments and opinions expressed by the interviewed and what they recommend for improving in the tool and to became more optimised.
23 pages
Download: Portugal National Report
Title: Romania National Report
Author: LILIANA VOICU
Abstract: The testing was carried out in different steps and different contexts.
The first step of the test was dedicated to the trial of the actual usability of the tool and the impact of the structure on potential users. Three trainers from the staff of AxA Consulting, that were familiarised with the project’s idea, and three trainers from commercial training companies that were introduced to the general approach of Reflective evaluation tested voluntarily the trial version of the tool, the test being carried out for all six in AxA’s premises. They were not given special instructions, just kindly asked to see if the tool works and to tell their impressions on it.
The comments were intentionally left unstructured, the idea being to see their spontaneous reaction without any outside influence. A larger discussion followed, on their general idea of self-evaluation, including potential themes and modes of interaction that could be used by such an instrument that would initiate a reflection process. The objective of this part of testing was to identify contents and types of questions that could correspond to trainers’ needs and motivate them to use the tool.
The second test was more extended and more structured, being based on the first variant of the tool, with a limited number of questions. Two different groups were used for this phase.
25 pages
Download: Romania National Report
Title: UK National Report
Author: Jenny Hughes
Abstract: The tool was piloted with two groups. The first was a group of trainee Design and Technology teachers on the Post Graduate Certificate in Education course at the University of Newport. The second group was less homogenous and consisted of teacher trainers from a range of establishments with responsibility for initial and continuing education.
The reason for choosing these groups was threefold.
Firstly, self reflection and self evaluation is a core element of all initial teacher training in Wales and one which many students find difficult or of little value. We felt that The Reflective Evaluation tool may be useful in this context and wanted feedback from the students themselves and also from the teacher trainers. Secondly, we had easy access to the groups which was entirely pragmatic. Thirdly ‘CVET’ teachers are not really an identifiable group in Wales.
The reality is that teachers in VET schools are responsible primarily for IVET. Most VET schools will offer short CVET courses targeted at employees in a particular sector but will, in the majority of cases, use the staff whose primary role is IVET. Neither term is in common use in the UK – very few VET teachers or managers would use the term and most would not understand it as there is no formal distinction between the two.
In the case of the staff group, individual subjects were given a brief explanation about the history and rationale of the project, given the url and asked to browse the site. They were told that they did not actually have to fill in ‘answers’ unless they thought this was necessary to inform their feedback.
11 pages
Download: UK National report
