Assessing the Unassessed or the unassessible: Issues of School Evaluation TESOL GREECE, March 2014.
Assessing the Unassessed or the unassessible: Issues of School Evaluation TESOL GREECE, March 2014
Reply
Assessing the Unassessed or the unassessible: Issues of School Evaluation TESOL GREECE, March 2014
This summary of the session appears in Pattison, Tania (ed), IATEFL Selections 2013, UK, 2014
Assessing the unsassessed or the unassessible: issues of school evaluation
Maria-Araxi Sachpazian, Input on Education, Thessaloniki, Greece.
The debate regarding school assessment
If seen superficially, any discussion on School Assessment (SA) seems unnecessary as professionals know the strengths and weaknesses of the schools where they teach. Quite contrary to this emotional take on the issue systematic assessment of our practices, which can render quantifiable data, is the only means available to us in order to initiate change and improve the work of the school.
Unfortunately, there are many obstacles standing between SA and the formation of clear conclusions related with the efficacy of the school. First of all, assessing schools is not straightforward as schools are complicated organisations (White et al. 2008: 26-27), with many of their features remaining either unassessed, because they slip under the assessment radar, or unassessible because they are too qualitative to be quantified. Secondly, defining what a good teacher is or what makes a good lesson is hard to standardise (Coleman et al. 2005: 149). Even the tools used in SAs (observations, questionnaires, interviews) are highly subjective and can easily give us a misleading idea of what actually happens in the school after the SA is over. Finally, when SA is carried out by external inspection teams, educational issues cannot be fully understood in the limited time available. An internal assessment which might solve this problem, may stumble on a worse one: lack of objectivity.
Confusion made worse
Many think that through SA we can manage to identify policies of excellence, which can be applied by other schools in an effort to create ‘’recipes’’ for effective schools. While that sounds promising, only indications of what has worked in other schools can be given, not exact practices that can be duplicated. (Harris 2002:7)
Others have related SA solely with accountability and the upward or downward movement of the staff. This has discouraged the employees from actually investing on the SA and has created a contradiction: although the assessment is supposed to take place for the benefit of the school, the staff tends to be less than excited about it. Relating SAs closely with accountability means that the element of change, which is what teachers hope to get out of an in-depth assessment of the school practices, is removed. Therefore, the SA seems to be irrelevant to the real issues that touch upon the teaching reality of that school, which explains why the staff members do not invest emotionally on it.
Another area of confusion is that of relating the success of the students with the efficacy of the school. Contrary to the opinion of the non-experts, student achievement cannot account for the efficacy of the school, as the student intake is carefully scrutinized so that only the high scorers are accepted (Coleman et al. 2005: 149). Student achievement is an indicator of school efficacy but we cannot rely on it to evaluate the actual process of teaching. The area that might render useful data might be that of the actual effect the school has had on the lives and personalities of the learners but this is far too difficult to study. (Coleman et al. 2005:136-137,139)
Towards a self-regulated model of SA
So far we have seen SA as externally imposed and top-down. Undoubtedly, this mode of SA is significant but hopefully in the near future SA will be more closely associated with continuous professional development and be considered as agent of change, rather than a means of firing or promoting. In that case SA ought to become part of the reflective practice of the school with the staff determining what is assessed and the criteria to assess it. Ideally, the only real way of assessing whether such a SA has been successful would be to check the extent of sustainable change that it has inspired and the progress the entire school has made.
Conclusions
The obvious benefit of this self-regulated model of SA is that the staff would feel more connected to the SA, since it would emerge as a need from within. Therefore, it would be more closely related with the real issues that matter to the people of the school and it would have more chances of being sustained. Moreover, comparing the findings of the external and internal SA would enable the school administration to rate data that has so far remained either unassessed or was considered unassessible which would help administration reach more reliable conclusions regarding the kind of change that needs to be initiated.
sachpazian@input.edu.gr
References
Harris, A. 2002. School Improvement. What’s in it for schools? New York: Routledge
White, R. and Hockley, A. and Van der Horst Jensen, J. and Laughner, M. 2008. From Teacher to Manager. Managing Language Teaching Organisations. Cambridge: Cambridge University Press
Coleman, M. and Earley, P. 2005. Leadership and Management in Education: Cultures, Change and Context. Oxford: Oxford University Press
If you want to view the slides: http://www.scribd.com/doc/213061796/Assessing-the-Unassessed-or-the-Unassessible-TESOL-GREECE-2014
Link
A Xmas Activity with The snowman by Raymond Brigs
A Xmas Activity with The snowman by Raymond Brigs
English: Christmas lights Nederlands: Kerstverlichting (Photo credit: Wikipedia)
Photo found at http://www.eckhobristol.com
The snowman and me
I remember finding ”The snowman” in the library of the first Foreign Language School where I had a job teaching mainly junior classes. Although I was fascinated by it, I could not (and still can’t) put my finger on what exactly impressed me more. Perhaps the reflection of silver white snow, the great music or the lack of script which makes this Christmas story perfect to use with any level and to focus on any skills the teacher wishes to enhance.
Not having worked with it for years, I decided to use The snowman once again this year with my C1 Class and I must say that I was pleasantly surprised that all my teenage learners were engaged with the story.
Choices to be made
Since I did not want to spend too much class time watching the video and since we had to watch it quite a few times, I opted not to work with the full version, which lasts about 26 minutes. Instead, I chose the part that shows the snowman and the boy flying and is beautifully accompanied by the song ”Walking in the air.” This version lasts for about 4 minutes and is more flexible when used in the classroom.
First watching
The first time we watched the video, the learners were given no information about what they would be watching. There was no worksheet or task. Their sole task was to imagine what happened before this scene and how the boy must be feeling. These two questions were an introduction to the story and also a springboard for discussion on the short video clip.
The second viewing and the worksheet.
The second viewing was the one on which we devoted more class time. The first task was to listen only to the song and write the words missing. This task, apart from being relaxing and pleasant, helps learners to focus on the song which interprets the story and provides the only script available. When looking at the worksheet (to be uploaded later on) one can easily notice that I did not remove many words. This was done on purpose as I did not wish to stress the learners and turn this into a listening test. Learners were also given the option to pair up with another classmate during this activity. After checking answers, learners had the chance to discuss the title ”Walking in the air” and how appropriate it is. Once again the issue of flying and being free and weightless were brought up.
The next activity is a sequencing one. The learners had to put the places over which our two characters fly in order of appearance. To be honest, I expected that this activity would tire learners or start a range of disagreements but fortunately it worked out perfectly and gave learners a firm graps of the storyline. The questions that follow on the worksheet are less focused on comprehension ones and more more aimed at making learners visualise the feelings of the two characters, which will help them make their writing more realistic.
The final activity is a writing one. This year I opted for a text and after setting it up in class, I’ve asked learners to do it as homework.So hopefully I will be able to uplooad some work samples after the holidays. Despite the fact that short stories and poems are not genres candidates might encounted at a C1 or C2 exam, I feel that getting a chance to try them out gives learners a better understanding of the appropriate style for non-factual, creative writing.
The main gains from The snowman.
This lovely cartoon story is versitile enough to work in different classrooms and because of its versatility it engages learners of different learning styles. Therefore, my way of using it this year is by no means exhaustive. One main gain from using it is that learners can do an amusing Christmas activity with true lignuistic gain which also allows them to work on their skills.
This year the writing activity was more or less guided but for next year I am planning to add some guided visualisation which will allow learners to see themselves flying over their own city and write about their personal experience. Antoher writing idea is to have learners tell the story from the aspect of the people the snowman and the boy meet on their journey in the sky. This will give them the chance to create about four or five entirely different short stories which can be the different parts of a mosaic.
Conclusion
The more I ”play” with The snowman the more mesmerised I feel and the more teaching possibilities I find in it. Hopefully, there will be many more classes and many more Christmases so that my learners and I can try working with different aspects of The snowman
Link
This is the slideshow from the session in PDF format.
Common Mistakes School Owners Make
Aside
| When School Owners make their personal self-analysis of the way they manage their schools, they practise the third conditional a lot! As all teachers know the third conditional refers to something in the past that cannot be undone and it can often express regret. Therefore, school owners very often find themselves thinking: If I had had more money, I would have found a better place. If my competition had not had that great spot on the main street, they wouldn’t have been so successful! If my staff had been better, I would have had more students. Obviously, there is nothing to be done since things are said in third conditional. But perhaps the mistakes Foreign Language School Owners make are not related with money, their competition or their staff but with simpler, easier to locate areas. | ||
 In the last twenty years we have been discussing the VAK learning styles at lenght. Perhaps, it’s time we started classifying School Owner mistakes as Visual (related with their eyes), Auditory (related with their hearing), Oral (related with their mouth) and Intellectual /Emotional (related with their brain and soul). |
||
| VISUAL | ||
| Question 1 ”Do you actually look at your school?” | ||
| I know it might come as a surprise but when people have had a business for a considerable amount of time, they tend to lose objectivity. Not looking, means not noticing. Mistakes are positive and they can become stepping-stones to learning if they are noticed. If they go unnoticed, they harm our school. For this reason, it is always refreshing to ask a consultant or a trusted colleague to take a look at the school in order to comment. | ||
| Question 2: ”Do you look at your services?” | ||
| Foreign Language schools in Greece have a hard time identifying their Unique Selling Point (USP) mainly because the owners themselves do not actually know which aspects of the school sell or don’t sell. This can be mainly related to lack of performance monitoring and data collecting without which owners cannot gain an in-depth knowledge of their school. | ||
| To go back to question number 2, if you don’t look at your services, who does? If nobody does, then the school ends up circulating a brochure that has nothing to do with the actual school. To make matters worse, a significant number of School Owners think that once clients are drawn to the school because of the great advertisement, they will stay. This has made owners think that marketing the school is even more important than developing its academic aspects and supporting teachers in their work. The truth of the matter is that FLS need marketing to promote the great work they do. We advertise what we have, not what we asprire to have. If a school owner wants to advertise greatness, s/he will have to create it first.Marketing which promotes attributes the school doesn’t have is doomed to fail. The clients who came drawn by Photoshop pictures or false promises will evaporate as quickly as they came. | ||
| Another mistake related with not actually looking at our school is venturing out to offer services we think the school budget/building/staff can support without making a thorough cost analysis. Very often school owners take on classes at a cost so reduced that the school doesn’t profit. At other times they continue offering German or French classes which are funded by the income of the English classes, for fear of losing clients or ground in the local market. Unfortunately, maintaining something that doesn’t work, has never managed to change the situation to the best. All these prove that lack of advanced planning, thorough budgeting and data analysis blur the manager’s vision and prevent him/her from running a healthy business. | ||
| Question number 3:Do you let clients look at your school? | ||
| Recently when I discuss registration policies with School Owners, I get this far-off look from them and a sigh: ”But parents don’t care. They only care about the price.” The price is a valid concern but parents surely care about the environment in which their offspring will learn. If they don’t, we can make them interested. After all, we are the experts! Having parents look at the classrooms means that we get a chance to comment on the reasons why desks are placed they way they are, therefore we get to talk about pair work and group work and the need for learners to speak in L2 and be exposed to it. So, what seems to be marketing (showing the school to clients), indirectly sends messages about the quality of our academic services and makes our USP more pronounced to the clients. | ||
| Question 4: ”Do you maintain your vision for the future?” | ||
| As managers School Owners ought to be thinking of the future of the school all the time. Unfortunately, what School Owners think about is the school as an empty shell (with walls and furniture) and themselves as providers for their family. These are valid concerns, but this way of looking into the future puts forward the idea that the school can progress without its staff. Perhaps this is a Greek International First because usually all sorts of organisations and corporations develop based on the skills and abilities of the people who work there. It is necessary for the owner to have a vision of how the staff will contribute to the development of the school and how the school will play a part in the Continuous Professional Development of the Staff. | ||
| Auditory | ||
| Question 1:”Do you listen to your clients/your staff?” | ||
| Listening is not easy. More often than not when others speak, we don’t listen intently. We listen to them but at the same time we are busy thinking of our reply. Some other times we are so prejudiced about what other people are saying, we don’t actually let them finish or we listen to them defensively cutting them off with a ”yes, but…” every other sentence. This can be a serious mistake. Even if our clients do not have the best interest of our business at heart, they can give us useful pointers. Let us not forget that they are the fresh pair of eyes we were talking about before. On the other hand, our staff is and should be every bit as interested in the school as we are. If they aren’t, it’s time to re-examine the school’s hiring, training and firing policies. The staff come in contact with our clients every single day. They see how the procedures and policies work and the effect they have on the learners. Listening to them should be a School Owner’s number one priority. In fact, I would be seriously puzzled if members of staff had no input to contribute. | ||
| Question 2: Do you listen to the same voices all the time? | ||
| We all have friends or colleagues who share our passion and see things the same way. We all have our pet consultants, professional or not. Unfortunately, listening to the same advisors & friends all the time, can be quite limiting. Much like we need a fresh pair of eyes, we also need new voices speaking to us. Even professional consultancy isn’t a panacea which can last for ever. | ||
| Question 3: ”Do you listen to feedback?” | ||
| In order to listen to feedback, there should actually be some feedback to listen to and we are not big on that. There are, of course, the usual questionnaires which ask questions about the surface and get surface answers but most Foreign Language Schools in Greece do not invest in examining what their internal and external clients think. This means that we have no data and therefore we base our next move onto what we ”think” or ”feel” which can be misleading. | ||
| Oral | ||
| Question 1: Is your communication effective? | ||
| Organisations choose channels of communication based on their services/ products, audience and on their budget, or so they should. Choosing a means of communication the school cannot afford can put the school in serious danger. Another problem is that schools stick to given channels of communication (letters, notes, announcement, phone calls at predictable occasions) and they do not try out new ones (Facebook groups, blogs etc). | ||
| Question 2: Does what you say cause friction? | ||
| As managers, School Owners should be very careful not to alienate clients or staff. Usually, with clients diplomacy works because we all know we need them. Sometimes, diplomacy works too well and clients are not told what they should have been told: the truth! It is usually with the staff that diplomacy doesn’t work. It is natural for any manager to have favourite employees. Personally, I wouldn’t like to work with anyone who wasn’t my favourite employee, simply because I wouldn’t be able to do justice to this person or motivate him/her. Still, having pet staff members is fine as long as this is not obvious to the others. Talking to staff, even when it is about discussing sensitive issues or reprimanding them, should be done peacefully in a non-threatening environment, in which any staff member can open up and admit a mistake s/he has made. | ||
| Question 3: Is what you say said at an appropriate time? | ||
| Holding back on what you have to tell clients or staff, from a simple ”thank you” to a precious ”well done”, might make all the difference. Some other times waiting for people to cool off and become more receptive can be a useful strategy. Therefore, when to open our mouth and how to phrase what we need to say are issues to consider depending on the situation and the people involved in it. | ||
| Intellectual & Emotional | ||
| Running one’s own business is all about thinking and feeling. At the same time if one increases it is usually at the expense of the other and this spells trouble as we either become oversentimental and underthink things or we think to much and forget all about emotions. Once again to see how the school ”feels” we need to use our eyes. | ||
| Question 1: ”Does the school feel like home?” | ||
| Now, this question has two sides. The positive one is related with how cozy and safe our school feels for our learners who came to us when they were little and will leave when they are teenagers. It is also related with how our staff perceives our school. Is it a second home for them? Is it a learning hub where they grow professionally? All these positive aspects are valid as long as they don’t evoke any of the negative ones. For examples, if the school owner works with members of his/her family, it would be a good idea if the school did not to feel too much like home because in the professional setting the relationships would have to change. Secondly, schools are schools and they should be decorated in a professional way. Professional & cozy is completely different from home-cozy and the moment the two styles are confused clients cannot see our business as a place where business is done. | ||
| Question 2: ”Does your school feel great?” | ||
| The difference between a ”great” school and a ”small” one is not the number of students or the size of the building, it is actually the vision the owner has for that school. If school owners manage their schools with the same consistency as supposedly ”big” schools are run, then clients are forced to see these schools with different eyes. The question that complements this one is how this greatness is expressed. Because if greatness come with aloofness and a diva attitude to staff and clients, sales will not go up. | ||
| Question 3: ”Do you brainstorm solutions?” | ||
| All foreign language schools have staff meetings. Unfortunately, these lengthy affairs do not produced the desired results. The reason for that is firstly because of ineffective communication (which has already been brought up) and secondly because we spend too much time talking about the problem, not brainstorming solutions. This is also related with the visual/ hearing issues we have discussed. Even if ideas are put forward, is the school owner ready to see the school under this new light and listen to this advice? | ||
| Question 4: Do you think of the entirety of the school or just the parts? | ||
| I have discussed this issue at length in an article that appeared in ELT News in January 2012. It is a complicated issue so I will just try to touch the tip of the iceberg here. Nothing in a school is unrelated to all its other aspects. What we perceive as ”unrelated” is only seemingly so. Therefore, when we make one slight change, odds are this slight change will influence everything. This is why rushing into decisions and making changes without thinking things through might be detrimental for the school. | ||
| Question 5: ”Do you know your people?” | ||
| To motivate means to know the other person well and teachers know that. It’s a pity that we don’t transfer skills we know from teaching into our managing. In order to know the staff, channels of communication need to be open and there should be occasions in which staff and administration can exchange opinions and get to know one another. Yet another reason why staff appraisal sessions should be held and why performance monitoring might come in useful. | ||
| Offering a unique experience | ||
| What schools need to create is a unique experience for their clients and their staff. This unique experience is not out there nor is it one for all schools. It can be identified by analysing what the school USPs are and by carefully examining what the school can offer. Offering this unique experience should be effortless because only then it will be natural. Schools that would like to offer this experience ought to avoid: | ||
|
||
|
||
|
||
|
||
|
||
| Conclusion | ||
| Beauty is on the beholder’s eye, we say. Well, greatness is also on the beholder’s eye. The biggest mistake a school owner can make is to see his/her business as a small, limited one and be influenced by this when making decisions. No staff and no client wants to associate themselves with the small and the limited. Valuing our business does not make us conceited, it makes us inspired. And everyone wants to hang out with those who are inspired. | ||
| 3rd Foreign Language Schools Conference | ||
| Organised by ELT NEWS | ||
| The Grand Palace Hotel | ||
| Thessaloniki | ||
| Special thanks to Ms Tasoula Spyropoulou. |
Link
Slideshow from my session at 47th IATEFL CONFERENCE in Liverpool.
Educational effectiveness can be defined as the effect a school has on its students’ development. Assessing the effectiveness of a school is not a straightforward issue because it touches upon many issues. Moreover, most of the data is hard to quantify.
Firstly, the aim behind the evaluation must be specified. If we believe that the outcomes of the evaluation will lead to the isolation of good practices, we accept that generalisations about schools can always be made. If we claim that evaluating schools will lead to the avoidance of wrong practices, then school assessment is directly related with change. Therefore, effective evaluation should mean assessing whether change has occurred.
Another issue is the importance of student assessment and whether its results can be presented as proof of school efficacy. Although, according to Educational Research[1] schools that give their students ‘’added value’’ are those which make a difference, we have to examine whether this can be attributed to the school or whether students have been ‘’screened’’ to fit a certain profile. Finally, school assessment delves into classroom based issues (i.e. what effective teaching is, how high expectations can be linked with increased student performance), which very often impedes teaching by forcing teachers to teach in an accepted ‘’observer’’ mode. Sadly, teachers are often poorly supported for this.
This presentation aims to examine the theories behind school evaluation and to present an overview of the issues related with it. Secondly, it will point out the relationships between assessment, change and staff development.
[1] Macbeath and Mortimor, 2001 in Coleman Marianne & Earley Peter (eds), (2005), Leadership and Management in Education: Cultures, Change and Context. Oxford University Press, page 139
