Why should teachers engage in educational research?

As part of my MEd session, I was put into a group with a few other MEd students and asked to work as a team to produce a user-friendly guide which helped explain why teachers should engage in educational research. The information that follows is part of our group discussion on the topic.

The Essence of Ethics

By K Carson, J Knight and N Taylor

Guiding Questions

1) What are ethics and why are they important?

Most people think that ethics deal with issues of right and wrong; in research, ethics are used to promote the aims of the research and to ensure that it is conducted in a way that is safe, fair and balanced. 

2) How will ethics inform the way I conduct my research?

No matter what subject you choose to research, you must make sure that you are conducting an ethically sound study. This will ensure that people do not question the way you went about collecting, analysing and reporting your data. Being explicit in your ethical considerations means that you will be transparent (open) in the way you conduct your research.

Key Points

1) Make sure you get informed permission from all parties, including parents if students are being consulted.

2) Researchers must be open and honest at all times.

3) All participants have the right to withdraw their participation at any time, without fear of negative  consequences.

4) Children should be consulted on matters that involve them.

5) Don't give incentives that could affect participant responses (i.e. chocolate or a day trip out).

6) Participants should remain anonymous wherever possible.

7) If any harmful effects are likely to result from the research process, the participants must be duly informed (this usually only effects medical research).

8) Don't falsify or distort your findings!

9) Don't take credit for anyone else's work.

10) Make sure that you communicate your findings in an accessible way.

Methods and Practicalities of Ethics in Research

1) Use consent forms and letters to parents.

2) Ensure open disclosure of the purposes of the research. This means that you should explain why you are undertaking your research, for what purpose(s) and for whom (audience).

3) Make it clear to participants that they can withdraw their support at any time. This should form part of the consent form. This may mean that you will lose participants throughout your project, and this is a problem when researching your own classroom. A good way around this is to have a backup plan or multiple options. You should remain flexible throughout your project.

4) Don't underestimate the power of student voice. Keep the students informed and answer their questions openly and honestly.

5) Change the name of participants to pseudonyms or numbers/letters. You should not be able to identify any participants by reading the finished report, so avoid any specific identifying features.

6) Keep a record of all your data. Research logs, diaries and journals are a good idea, as are keeping recordings and video data.

7) Always reference the work you are using. This means including the author(s) names and the date in which the source was published. For example, you would use (Carson et. al, 2015) if quoting our article.

8) Write in a way that makes your topic sound relatable and clear to others. Stick to the facts.

Key readings

The British Educational Research Association, (2011) 'Ethical Guidelines for Educational Research', London. Available online at https://www.bera.ac.uk/wp-content/uploads/2014/02/BERA-Ethical-Guidelines-2011.pdf 

Hammersley, M. and Traianou, A. (2007) Ethics and Educational Research. London: TLRP. Available online at http://www.tlrp.org/capacity/rm/wt/traianou/

Devising Interview Questions

This is a bit delayed, of which I am a bit sorry to admit; usually I like to type up my thoughts immediately after a sessions. However, last week was a little hectic, so I haven't been able to reflect on my most recent meeting with the student researchers. In our last meeting, we looked at the questions I used during my pilot (adapted from Gamlem & Smith, 2013) and adapted them to suit our own purpose. I was quite struck by a number of the students suggestions, and will briefly outline my thoughts below:

Question 1: When was the last time you got feedback? What was the feedback? Can you give examples? How useful was it?

The SaRs felt that this question was far too long, and suggested that we trim it down and simplify it. They suggested that we leave the first part of the question, but simply ask students to describe the feedback, if they could recall a specific example.

Question 2: What do you find useful when getting feedback from your teacher? Why?

Again, the students suggested that we change this question slightly. They felt that by including the phrase 'from the teacher' was too leading; their argument was that feedback doesn't always come from the teacher, and that by leaving it as simply 'when getting feedback' it left the options of peer and self-assessment there, while not necessarily bringing it up (which would be left specifically for later).

Question 3: What don't you like when getting feedback from your teacher? Why?

We removed the 'from the teacher' part again, for the same reasons as above.

Question 4: What do you usually do with the teacher feedback? How often do you get the opportunity to work with your targets?

This question was changed dramatically. Again, the students felt that the question was too 'leading'. First of all, it implied again that the only type of feedback was teacher feedback. Secondly, they felt that it implied that students DID get the opportunity to work with their targets, which might influence their answers. Instead, we changed the question to 'How do you respond to feedback?'

Question 5: What usually happens when you get feedback? How is feedback followed up?

This question was completely deleted; students felt that it was too similar to the previous question AND suggest that it was (again!) too leading.

Question 6: How do you do peer assessment?

Once again, they identified this was a 'leading' question; it implied that they DID do peer assessment. Instead, we changed the question to be 'Do you ever use peer assessment? If so, how?'

Question 7: Do you self-assess your own work in lessons? How do you do this?

We left this one the same.

Question 8: Are there times when you don't get feedback but you'd like to?

We left this one the same.

Question 9: How are you told about moving from one level to the next?

We changed this one as well, eliminating the 'how' at the beginning and changing 'one level to the next' to simply 'Are you told how to improve your work? If so, how?' The students felt that this left the question more open ended.

Question 10: Do you know what your teacher looks for when assessing your work?

In this question, 'assessing' was changed to 'marking' in order to be less formal.

Finally, we decided to add a question at the very beginning and start with 'What is the main purpose of feedback?'

Going through initial questionnaire results with SARs

I'm in a bit of a hurry, but I want to acknowledge some of the things that came up in my meeting today:

- distributed initial pie charts with data from Y8 and Y11 questionnaire
- asked students to look for anything interesting that caught their eye
- students pointed out key differences between Y8 and Y11 emotional reactions to feedback; noticed similarities in some places
- they felt that Y11s were more mature, which accounted for less emotional responses to feedback
- they felt that Y11s valued grades more than comments because they have higher stakes - the results help them make progress for importance GCSEs whereas Y8 don't have the same high stakes
- wanted to question the 'Ofsted' student - felt that it would be wrong to exclude this student's voice - specifically stated that it wouldn't be ethical and it would show bias in the result if we only consulted the students who 'praised' feedback or said 'what the teacher wants to hear' about feedback

Further analysis of the questionnaires

Last week I did some initial data analysis of the quantitative aspect of my questionnaire, so I thought I'd spend some time today looking at the qualitative side of things. Luckily, I didn't set myself too many qualitative responses to go through at this stage, so it wasn't too time consuming. However, I already have spotted some interesting trends that I'll want to explore further.

First of all, I asked students if they could think of a time within the past month where they received feedback. In Year 8, 15 out of the 19 respondents stated that they had, as did 11 out of 14 Year 11s. The next question asked those who responded with 'Yes' to describe what they did with the feedback they received. After analysing all of their responses, I've come up with the following categories:

Year 8
1) they increased their effort;
2) they added to their work;
3) they did nothing; or
4) they recorded their targets, but did not work with them.

Year 11
1) they acted on the target/used it
2) they reviewed their work by recording the target (but didn't necessarily use it);
3) they were given further instruction; or
4) they did nothing or ignored the feedback.

Overall, these seem to be fairly similar responses, with the majority of the students saying that they acted on or used their targets. However, roughly the same amount of students in both year groups also suggested they did nothing with their feedback, which is unfortunate.

Next, I looked at the 'No' students (those who had not received feedback in the past month). Their second question was to state why they thought teachers gave them feedback. Interestingly, all the 'No' students, in both Year 8 and Year 11, stated that feedback was meant to help them improve. No other alternatives or interpretations were given. This may be down to the fact that there were not many 'No' respondents, hence why no other interpretations appear, however I feel that their lack of experience with feedback may be limiting their awareness of its uses. I cannot say with certainty that this is why, but it's my initial reaction.

Interestingly, the 'Yes' students had more ideas about what feedback was for. Their responses tended to fall into the following categories:

Year 8
1) to improve; and
2) to learn from their mistakes

Year 11
1) to improve;
2) to learn from their mistakes;
3) to make you feel bad; and
4) Ofsted

Interestingly, the students who had received feedback felt that not only did it help them to improve, but it also helped them to recognise and learn from their mistakes. This suggests that mistakes are not necessarily seen as a bad thing when feedback is given.

In my last post, I already address the two 'negative' responses re: making you feel bad and Ofsted. I still believe that I would like to include one of these students on one of the interview panels, as I am interested in hearing more about what they think of feedback. I am curious about where their negativity stems from.

I also analysed the results of my Students as Researchers pre-test questionnaire. They had some questions which correspond to the student questionnaire, although I had to make some changes to evaluate the impact of their participation on their own self-awareness and understanding. Overall, their definitions of feedback matched those above (to offer advice and to identify their level of work). One difference was that two of the students (out of 6) stated that feedback could come from anyone, not just the teacher.

All of the student researchers said that they had received feedback in the past month. Their responses to what they did with it fell into two categories, again similar to those given above; they used the feedback or they recorded it (but didn't necessarily use it).

In terms of their understanding of the uses of feedback, they felt that feedback was used to 1) provide them with a grade; 2) help them improve; and 3) help them understand their own ability. Again, many of these themes remain similar across the groups.

Some of the different questions included an opportunity to define what 'research' means. Their responses were either to study something or to gather info (a number of them used the word 'data' in their response, which was nice as it shows a good working knowledge of the language we'll be using throughout the project.

As you can see, there are some interesting points to pursue. I am meeting with the student researchers on Tuesday and Thursday this week to go over the data from the Year 8 and Year 11 questionnaires. I'm not going to share with them my interpretation of the findings at this point, because I'm interested in seeing how they read the results themselves; they might see things that are not clear to me, and I don't want to cloud their judgement with my own.

I will provide an update later. Cheers!

Some initial findings from my questionnaire

Good afternoon! I've just started to get some of my pre-interview questionnaires back, and thought I'd have a quick look at them whilst I had a free moment. Currently, I have thirteen responses (out of 53 sent out) from various Year 11s with me (many more from Year 8 are sat in a folder at home, anxiously awaiting my analysis). I've skimmed through some of the key questions, and have noticed a few early trends.

First of all, only two out of the thirteen respondents said that they have either 'ignored' or 'haven't used' feedback that they recently received. This suggests that most students are working (or engaging) with the targets they receive, which is a good sign. However, when asked to describe what feedback is used for, one candidate wrote that feedback is only used to 'look good when Ofsted check our books'. Another student wrote that feedback is used to 'take the piss out of u with sracstic comment [sic] and call u a god damn idiat [sic]'. I'm definitely going to flag these respondents up for potential interviews, but again I won't make the final decision here until I discuss the results of the questionnaire with my student researchers. Regardless, I find these negative comments fascinating!

The comment about Ofsted suggests that the student has a strong opinion about the effect that Ofsted has on learning; he seems to be implying that we only provide feedback to tick a box, not to help students make progress. While I think some would argue there is an element of truth in this, it takes a lot away from the real potential that feedback should have in helping students to progress. It appears to me that this particular student has become 'jaded' about the system, which could mean he's not valuing feedback as much as he could be. It would be interesting to see where these ideas arose from. I'd also like to know what the other student meant when he said that teachers use feedback to 'take the piss' - is he implying that feedback is used to personally attack students? If so, that's quite troubling!

Interestingly, the same 'Ofsted' comment student ranked comments as being neither helpful nor unhelpful, while ranking grades as being extremely helpful. However, on the whole, the other respondents were quite positive about feedback. The other responses to the uses of feedback tended to follow along the lines of 'providing ways to improve'. When ranking comments and grades, the majority of the thirteen respondents placed comments and grades at about the same level.

Another interesting point that appears to be developing so far is that five of the initial 13 feel that positive and negative comments affect them emotionally; the rest felt neutral about both. This is interesting because much of the literature that I've read so far states that many students feel the emotional effect of positive and negative comments; however, what I've seen so far suggests that this isn't always the case.

Obviously these findings are only just preliminary. I can't say for certain that these trends will continue when the rest of the data comes in, but nevertheless they provide an interesting starting point. More information will follow when I start to do some proper analysis using Numbers.

Our Second Meeting

I met with the student researchers today for the second time. We briefly discussed what they thought about the literature review that I provided them with, although not all of them had completed it. We were able to talk about what they thought about feedback, and I was able to discuss some of my initial findings based on their pre-test questionnaire (more of which I will discuss in detail in a later post). One thing that we all found interesting was that all of them ranked grades as more useful than written comments, which goes against what the literature says is most effective. When I asked them to explain why they felt this way, they said grades provide them with a 'starting point' so that they can see exactly where they are, consult the criteria for the grade above, and make the appropriate changes. 

They also suggested that grades made it easier to compare themselves against other students, which I had assumed (and the literature also suggests) would cause negative feelings; often we're told to discourage students from comparing themselves to others. However, when I asked them to clarify what they meant by comparison, they were quite articulate in describing how they would go to someone with a higher grade for help and guidance, opposed to asking the teacher for an explanation. This is something that I found quite interesting, and would like to perhaps discuss again later.

I pushed them a bit further, and asked them about whether or not the grades ever made them feel bad about themselves. However, they (on the whole) didn't associate negative feelings with grades (which, again, the literature suggests they DO). Overall, the students were interested in exploring their own thoughts, and the thoughts of other students further, against what the literature on feedback says, which is promising regarding their engagement with the rest of the project.

I then showed them a draft of the questionnaire that I planned to use with Year 11 and Year 8 students to gauge their baseline feelings regarding feedback. They were extremely helpful in helping me to re-write some of the questions so that they were more student friendly. We also discussed whether some of the open questions were too 'difficult' for Year 8 students. My initial thoughts were to keep the questionnaire the same for both year groups, but I allowed the students to develop their own thoughts on the idea of having a different, more 'difficult', questionnaire for the older students and an easier one for the younger students. Eventually, they came to the conclusion (fairly unaided by myself) that we shouldn't underestimate the Year 8 students, and that their responses could be 'interesting'. I was very pleased that they came to this conclusion, because it showed that they were starting to think about the ethical implications of using two different questionnaires. 

We then discussed how we would distribute the questionnaire. The students suggested that I compile packs for form groups, which they would then deliver personally during our next meeting slot. They decided as a group that they would speak to individual form tutors first, to let them know what the questionnaire is for, in the hopes that this will encourage form tutors to chase up completion of the questionnaire. They decided to then speak to the entire form, stating that they have chosen random students to complete the questionnaire, and that it's really important that the questions be answered as truthfully as possible. We also discussed how ethical this was, with students suggesting that the students who received the questionnaire might feel singled out, while those who didn't might feel left out... I'm still debating whether or not I issue the questionnaire to ALL Year 11 and 8 students, but then I feel it would move too far away from my focus on PP students. We agreed as a group that it shouldn't be too unethical to single out the PP students, considering that's the aim of our study. They felt that a de-brief with the students, where they explain why they were chosen, might help solve the ethical dilemma. This may be something that we follow up with! 

Overall, it was a really productive meeting, and I've already gleaned some really interesting tidbits from what we've gathered and discussed so far. 

Meeting my student researchers

Hello! It's been awhile since I have posted. This is largely because I've been busy working on my Literature Review. The Christmas holidays also fell into that time period, so you can forgive me for being a bit distracted.

I wanted to post a quick update today to discuss my initial meeting with my student researchers. Initially I invited ten Pupil Premium students to help me with my research; six got back to me with the permission forms, so it was six I arranged to meet with today. Luckily, it worked out that it was 3 boys and 3 girls, so the sample should still be representative. However, one of the boys didn't turn up for the meeting today, resulting in only five students being present. I'm hopeful that he didn't get the message; it'd be a shame if he decided he didn't want to take part.

The meeting itself went quite well. I issued them with a pre-test questionnaire, asking them about what they know about feedback and the research process. My hope is to get them to fill in the same questionnaire at the end of the process, so that I can gauge whether or not they've benefited from being a part of the project.

Following the questionnaire, we discussed some of the key terms associated with research projects, including the Literature Review, ethical issues, types of research, etc. I must admit I rambled on a bit, but then I did allow them time to ask me questions at the end, and we had a good discussion of their expectations and the role they'd play in the research. I am hopeful that they'll still want to take part, and won't be put off my my rambling ways!

We're aiming to meet again on Thursday, so I will keep you posted on the developments.