Second reflective meeting

I met again with the team, this time with the slightly added pressure of Ofsted being in for a full inspection. Initially, I was a bit wary of holding the meeting, but after some consideration I realised that if Ofsted were to walk into our meeting it could only be a good thing; after all, it would show the students working in partnership with a teacher, discussing an issue which is highly relevant!

Anyways, this time we watched a Year 8 interview back and considered key themes and ideas that arose. I will briefly outline their thoughts below:

- Year 8s appear to really like the 2 star and wish policy, especially the praise element it contains. However, the student researchers suggested that it could be suggested that Year 8 students run the risk of becoming 'too reliant' on the stars. One of the student researchers suggested that Year 11s are less likely to focus on the stars and more on the wish, because of the critical element it implies. He said that in his opinion, he felt 'disappointed if [they] don't get criticism'. He went further to suggest that, in his opinion, 'Year 11 want the criticism'. However, he did suggest that if they receive 2 stars and a wish and one of the stars is blank, this can be negative, because it makes them think that they haven't done enough to warrant praise.

The student researchers also suggested that the Year 8s are happy with 2 star and a wish because it is 'the only method they know'. This is quite true, because the policy has only been used in my school since our last Ofsted inspection, two years ago.

- They too are less comfortable with self assessment, due to 'bias' (which we discussed could mean both positive and negative bias)

- I brought up the idea of verbal feedback, as I felt it hadn't come up much in the interviews. As such, I asked the team what they thought about verbal feedback; they felt that subjects like music and drama make the most consistent use of verbal feedback. However, the team of students did suggest that verbal feedback is highly valued, and often the most effective form of feedback, though it doesn't get used as much as they'd like, or sometimes it can be forgotten.

- The best way to improve feedback is to ensure that it is consistent across subjects and teachers; if the whole school follows and enforces a consistent policy, then feedback can be used well to help students improve

Discussing the interviews

This morning I met with the student research team and we watched one of the Year 11 interviews back, discussing ideas and themes which appeared. I didn't share any of my emerging themes or codes with the students because I wanted our discussion to act as a form of triangulation; through our discussion, key themes and ideas would hopefully appear, which could or could not correspond with my own interpretations.

After watching, the students seemed to (unknowingly) support the themes which I had picked out, especially when it came to 'good' and 'bad' feedback criteria. Bad feedback, in their opinion, was when teachers gave out the examiners grade criteria; while they understood why teachers did this, they found the language too vague and not helpful; specifically, the called it 'teacher speak'. They said that they would prefer if grade criteria was made more accessible in 'student speak'. Good feedback was when teachers were consistent and regular in providing evidence of 'current levels' and offered advice on how to get to higher levels.

Focus was also given to ideas of 'right and wrong' and 'improving'. Overall, they tended to agree with much of what the interview students suggested, focusing on a desire to receive grades, criteria and examples all at once to help ensure greater chances of successful progress. They suggested that both grades and comments are needed together, because the grade provides them with a solid understanding of where they are and the comments help them to get to where they want to be. To quote one student specifically, he said that grades acted as a 'stepping stone' to progress using comments to help. Expanded ideas included providing students with clear checklists of success criteria and level ladders written in 'student speak'.

One interesting point which they developed further than the interview students was this idea of teachers marking 'for Ofsted'. They were quite articulate in suggesting that they could clearly perceive when teachers were marking because they wanted students to progress and when they were marking purely for Ofsted. One boy specifically said that he disliked it when marking was 'not for our benefit', especially when teachers made them do tasks which specifically related to Ofsted, such as 'tidying our folders' or 'responding to feedback in a different colour so it's clear to Ofsted' (direct quotes used). One phrase that stood out to me particularly was what they referred to as 'teacher panic'. The students expressed a lot of displeasure with what they clearly saw as teachers panicking about the arrival of Ofsted. It was something that they felt negatively affected them; when I asked them how it made them feel, two girls told me that it made them, in turn, feel panic themselves.

The conversation also took a reflective turn, with students beginning to discuss how they would do things differently, if given more time and opportunities. All of the students felt that the Year 11 respondents were more open and honest than the Year 8s. They felt that the Year 8s were more likely to give responses that would 'please the teacher' and that they tended not to disagree with each other or offer alternative viewpoints. Shyness and an unwillingness to stand out or be different were offered as reasons why this could be. Another student also suggested that the very setting of the interviews had a negative effect on the students; because the interviews were conducted in a classroom, they felt that the respondents made more 'teacher pleasing' comments. One student went so far as to suggest that if they were able to conduct the interviews again, they'd try to find a more neutral environment.

They also suggested that interviewing the students a second time, later in the summer term, asking the same questions, would be a good way to check to see if their opinions remained the same. However, due to time constraints, this won't be a possibility.

These are my initial thoughts to the meeting. We will be meeting again on Thursday, looking at some Year 8 interviews and discussing.

Transcribing and initial reflections

Hello! It feels as though it's been a long time since I've written anything, but I can assure you I haven't forgotten my studies! Over the Easter holiday I typed up and coded the two Year 11 interviews, posting everything to the Google Classroom website so that the student researchers could have a look. Surprisingly, some of them actually went on and engaged with the research, which was amazing! Some brief discussions were had via posting comments, which will be helpful for us later.

Due to mock exams and other commitments, I wasn't able to meet with the team again until this morning. Since I had previously scheduled the Year 8 interview subjects to come, it was a bit of a whirl-wind morning, with the student researchers arriving and then immediately having to rush off to greet the Year 8s.

I did manage to speak to two of the girls before they left for first lesson though, and we briefly explored some issues arising regarding student dominance during the discussions. I also gave them a paper copy of the transcript of their Year 11 interview and asked them to read through it, jotting notes and summaries in the margin (by way of beginning to code, which I explained to them). They managed to get a bit of work done before the bell went, which I will outline below.

First of all, I suppose I should outline the emerging code list that I've generated.

Revised Codes Year 11 Interviews

To improve
               achievement
              strengths vs weaknesses

Right vs wrong

Grades > comments

Consistency
               Subject

Format

Bad feedback
               ticks and crosses
               vague
               general/generic
               impersonal
               lack of effort/engagement by teacher
               negative
               unclear

Good feedback
               combination
               constructive
               personal (implied)
               mark scheme
               peer

Working with targets

Confidence

Effort
               teacher
               student 

Of these codes, the two girls complimented many of them with their own, writing things like 'Strengths v Weaknesses', 'lack of consistency', 'grades and comments', 'peer assessment = good', and 'confidence'. 

I am going to meet with the entire team next week, and get them to read through and annotate all of the transcripts, in order to finalise and triangulate the codes. 

Luckily, I was also able to type of the transcripts for the Year 8 interviews, coding these only a few moments ago. Many of the same themes are appearing, although from slightly different angles. For example, it appears that the Year 11s notice a lack of consistency in feedback across the different subjects. A few key subjects were identified by Year 11 students as being 'good' at giving consistent feedback, but on the whole it seems like they don't receive feedback consistently. This isn't to say that the WAY feedback is given is inconsistent - they all made reference to the two star and wish policy used by the school. Instead, lack of consistency here means that feedback appears to be subject specific; some subjects give good feedback, and give it regularly, while others rarely give it at all.

The picture painted by the Year 8s, however, was much different. While certain subjects were repeated as being 'good' at providing regular feedback, there were additional subjects included, beyond the very few cited by Year 11 students. This suggests that those departments that are doing it well are doing it across the board, but that some subjects haven't implemented a system that is equal across year groups or key stages. 

A few sub-codes were also added to my list, after typing up the Year 8 interviews. For example, under the 'to improve' category, I have thought about adding 'steps/progress' and 'mistakes recognised', as these terms came up a few times, across the interview subjects. Two other sub-codes for bad feedback also include 'disappointment' (the feedback made them feel sad about forgetting something)  and 'confusing' (which can be linked to 'unclear'). Overall though, many of the same themes were appearing, which is encouraging. It makes my life a lot easier, at any rate! 

Something that I do find interesting though is that both Year 11 and Year 8 students have expressed a lack of confidence or comfort in self-assessment. It seems to me that they aren't given enough opportunities to do self-assessment, and if they do, it's under poor guidance. Students are unsure of how to properly mark their own work, and struggle with being 'too harsh' or 'too biased'. This is an area that I'd really like to pursue next year, as part of my own CPD. I think self-assessment is such a powerful tool, if used properly, and it's a shame that the students don't feel it's useful. 

Anyways, that's all I'm going to rant on about for now. I'll provide an update next week, after I've met with the team again. 

- KC

Initial impressions

Well, the team of student researchers have completed their first set of interviews. I have uploaded the video data to a private Google Classroom page (access is invite-only, and requires a private password), and left instructions for the team to try and watch the videos over the half term break. I've also started the ball rolling by writing up my own comments/thoughts about the videos underneath each one. My hope is that the students will watch the videos and then engage with my comments and each other.

The reason I did this was to encourage critical engagement with the research process; I had given each student a notebook to write in, but when I collected these in, only two of the five students had used them. Those that did use the books had only written a few notes on the first page. This was a bit disheartening, as I was hoping to have a second data source from the notebooks. However, I cannot force the students to use them, as I want the experience to be natural for them; I want my input and direction to be minimal, so that they feel a sense of responsibility and ownership.

As I've mentioned, I have already watched the videos once through. It was an interesting experience, because when I piloted the project I conducted the interviews myself. Watching other people conduct interviews that you've worked together to plan is quite an odd experience; I really have felt a certain loss of control. There were a few moments where I wish I'd been able to delve a bit deeper into what's been said. However, the students have been pretty good at using prompt questions to tease out information.

It's interesting to me to see which students are taking a leadership role in the interview process. One male student, who is academically the weakest in the group, is actually engaging really well with the process. He recognised, on more than one occasion, where the responses had been vague, and pushed the students to explain themselves in more detail. The higher ability girls were especially keen to get the interview process started, though I think they were more likely to follow the interview schedule exactly, whereas the boys tended to be able to think 'on the spot' a bit better.

I am eager to start the Year 8 interviews after half term, so that I can see if any of these patterns crop up again (in terms of how the student research team works).

As for what was said in the interviews, the following are points of interest:

- most students recognised that feedback was used to address both strengths and weaknesses
- English and Maths were mentioned as examples of good feedback by all students - Science was suggested being less effective (might be an interesting thing to explore in follow-up interviews)
- negative comments are viewed as unhelpful, but these are rarely given
- a 'bad marker' is a teacher who uses ticks and crosses
- generic feedback is not helpful - it helps when it feels personal
- constructive feedback, which links clearly to a mark scheme, is most helpful
- self-assessment isn't viewed as helpful, because students feel they 'lack a clear perspective' on their own work
- most students recognise that when work is set, it will usually be marked
- however, if students begin to realise that their teacher doesn't regularly mark their work, they lose motivation to do it - this suggests that small, regular bits of work and feedback promote a better work ethic within the students
-  most students feel that they can recognise when teachers are marking 'for Ofsted' and when teachers are marking because they genuinely want a student to improve

Obviously more information is needed; I think some follow-up interviews might be in order, to help us clarify some of the points that were made. That, or a few more individuals could be interviewed, to see if any of these ideas appear again.

Reflecting on the pilot interview

On Tuesday, three of the student researchers were able to conduct a pilot interview of two Year 9 students. Part of the agreement was that the students would not appear on film, so the students pointed the video-camera at the floor while conducting the interview. They asked the ten questions which we had devised as a group, in roughly the same order that they appeared on the sheet.

Today we met to watch the footage back and discuss what we've learnt from the experience. I also showed them some footage from my own pilot, and we went through each one critically, looking at sound, angle, etc. To start, the students mentioned that the sound quality wasn't very good. You could clearly hear them (or me) asking the questions, but it was sometimes hard to hear the respondents. When I asked them how we could fix this, they suggested that we move the camera closer to the interview subjects. Eventually, we came to the decision that if students don't want to appear on camera we should place the recorder face up on the table. This way, we can clearly hear what they are saying. If students agree to be video-recorded, then we discussed seating the respondents side-by-side and setting the camera clearly in front of them on a tripod. The interviewers themselves don't have to appear on camera, and can be sat to the side of the camera itself.

Another issue that the group brought up was that one of the two interview subjects really dominated the conversation. I am glad this happened in the pilot, as it is a common problem with group interviews. They described using some of the prompting techniques I had taught them, but found that the dominant person still managed to speak for the other student. Again, I asked the students how they could fix this problem. One of the girls suggested knowing the names of the students they were interviewing ahead of time, so that they could personally address questions towards them. We agreed that this was a good technique to use, so I assured students that they would know the names of the students they would be interviewing; I can also use the school behaviour module to show them a picture of the students before-hand, so that they know exactly who is who.

A third issue was that they discovered some of the questions required 'yes/no' responses (which I knew was going to be the case; however, because I wanted the students to devise the questions, I didn't want to influence their choices too much by pointing this out), which didn't always invite much discussion. However, they immediately described using some of the prompt questions to try and dig for more detailed responses.

Similarly, they felt that through the conversation, some of the later questions became repetitive or redundant; because this was their first experience interviewing other people, they stuck to the questions and asked them all, despite this. In our discussion, I told them that they should use their professional judgement in these types of situations; if they think that they've already addressed a question once, they don't need to ask it again. They seemed pleased with this - I think perhaps they were afraid of missing things out if they left questions off, but once I explained that they are essentially in control, that they're the experts once they're in the room and they can use their own discretion, they felt more confident.

Finally, they described how the students seemed to focus on feedback in English lesson. I asked them why they thought this might be. One student suggested that it was because the students knew that I was involved, and that I was an English teacher; their responses may have been focused on English because that's what they thought I wanted to hear. Another suggested that it was because the interviews themselves took place in an English classroom (not my own, but the Head of Departments). Again, this could be an influencing factor, and is one that I hadn't previously considered. The third student suggested that perhaps English was a subject where they received the 'best' feedback, or one where they clearly recognized feedback for what it was. Again, this could be the case, and it will be interesting to see what comes up in the real interviews.

To address these issues, we decided to add a statement at the beginning of the interview informing students that the questions are meant to be about feedback in ALL of their lessons, across the school. Hopefully this will take away any desire to focus on English to please me or due to associations with the setting.

Overall, it was a really good learning experience; if we had more time, I would have them conduct another pilot interview. However, as their time at the school is running out, our first set of interviews will take place next week! How exciting!

Why should teachers engage in educational research?

As part of my MEd session, I was put into a group with a few other MEd students and asked to work as a team to produce a user-friendly guide which helped explain why teachers should engage in educational research. The information that follows is part of our group discussion on the topic.

The Essence of Ethics

By K Carson, J Knight and N Taylor

Guiding Questions

1) What are ethics and why are they important?

Most people think that ethics deal with issues of right and wrong; in research, ethics are used to promote the aims of the research and to ensure that it is conducted in a way that is safe, fair and balanced. 

2) How will ethics inform the way I conduct my research?

No matter what subject you choose to research, you must make sure that you are conducting an ethically sound study. This will ensure that people do not question the way you went about collecting, analysing and reporting your data. Being explicit in your ethical considerations means that you will be transparent (open) in the way you conduct your research.

Key Points

1) Make sure you get informed permission from all parties, including parents if students are being consulted.

2) Researchers must be open and honest at all times.

3) All participants have the right to withdraw their participation at any time, without fear of negative  consequences.

4) Children should be consulted on matters that involve them.

5) Don't give incentives that could affect participant responses (i.e. chocolate or a day trip out).

6) Participants should remain anonymous wherever possible.

7) If any harmful effects are likely to result from the research process, the participants must be duly informed (this usually only effects medical research).

8) Don't falsify or distort your findings!

9) Don't take credit for anyone else's work.

10) Make sure that you communicate your findings in an accessible way.

Methods and Practicalities of Ethics in Research

1) Use consent forms and letters to parents.

2) Ensure open disclosure of the purposes of the research. This means that you should explain why you are undertaking your research, for what purpose(s) and for whom (audience).

3) Make it clear to participants that they can withdraw their support at any time. This should form part of the consent form. This may mean that you will lose participants throughout your project, and this is a problem when researching your own classroom. A good way around this is to have a backup plan or multiple options. You should remain flexible throughout your project.

4) Don't underestimate the power of student voice. Keep the students informed and answer their questions openly and honestly.

5) Change the name of participants to pseudonyms or numbers/letters. You should not be able to identify any participants by reading the finished report, so avoid any specific identifying features.

6) Keep a record of all your data. Research logs, diaries and journals are a good idea, as are keeping recordings and video data.

7) Always reference the work you are using. This means including the author(s) names and the date in which the source was published. For example, you would use (Carson et. al, 2015) if quoting our article.

8) Write in a way that makes your topic sound relatable and clear to others. Stick to the facts.

Key readings

The British Educational Research Association, (2011) 'Ethical Guidelines for Educational Research', London. Available online at https://www.bera.ac.uk/wp-content/uploads/2014/02/BERA-Ethical-Guidelines-2011.pdf 

Hammersley, M. and Traianou, A. (2007) Ethics and Educational Research. London: TLRP. Available online at http://www.tlrp.org/capacity/rm/wt/traianou/

Devising Interview Questions

This is a bit delayed, of which I am a bit sorry to admit; usually I like to type up my thoughts immediately after a sessions. However, last week was a little hectic, so I haven't been able to reflect on my most recent meeting with the student researchers. In our last meeting, we looked at the questions I used during my pilot (adapted from Gamlem & Smith, 2013) and adapted them to suit our own purpose. I was quite struck by a number of the students suggestions, and will briefly outline my thoughts below:

Question 1: When was the last time you got feedback? What was the feedback? Can you give examples? How useful was it?

The SaRs felt that this question was far too long, and suggested that we trim it down and simplify it. They suggested that we leave the first part of the question, but simply ask students to describe the feedback, if they could recall a specific example.

Question 2: What do you find useful when getting feedback from your teacher? Why?

Again, the students suggested that we change this question slightly. They felt that by including the phrase 'from the teacher' was too leading; their argument was that feedback doesn't always come from the teacher, and that by leaving it as simply 'when getting feedback' it left the options of peer and self-assessment there, while not necessarily bringing it up (which would be left specifically for later).

Question 3: What don't you like when getting feedback from your teacher? Why?

We removed the 'from the teacher' part again, for the same reasons as above.

Question 4: What do you usually do with the teacher feedback? How often do you get the opportunity to work with your targets?

This question was changed dramatically. Again, the students felt that the question was too 'leading'. First of all, it implied again that the only type of feedback was teacher feedback. Secondly, they felt that it implied that students DID get the opportunity to work with their targets, which might influence their answers. Instead, we changed the question to 'How do you respond to feedback?'

Question 5: What usually happens when you get feedback? How is feedback followed up?

This question was completely deleted; students felt that it was too similar to the previous question AND suggest that it was (again!) too leading.

Question 6: How do you do peer assessment?

Once again, they identified this was a 'leading' question; it implied that they DID do peer assessment. Instead, we changed the question to be 'Do you ever use peer assessment? If so, how?'

Question 7: Do you self-assess your own work in lessons? How do you do this?

We left this one the same.

Question 8: Are there times when you don't get feedback but you'd like to?

We left this one the same.

Question 9: How are you told about moving from one level to the next?

We changed this one as well, eliminating the 'how' at the beginning and changing 'one level to the next' to simply 'Are you told how to improve your work? If so, how?' The students felt that this left the question more open ended.

Question 10: Do you know what your teacher looks for when assessing your work?

In this question, 'assessing' was changed to 'marking' in order to be less formal.

Finally, we decided to add a question at the very beginning and start with 'What is the main purpose of feedback?'