Well, as you may already recall, I had previously admitted to cocking things up a bit with my second interview; the camera died about half way through. I have ten questions which I ask each group of students and, unbeknownst to me, the camera died after question six. I was gutted when the interview ended and I noticed that I didn't need to turn the camera off; it had decided to do so itself. I had checked the battery prior to beginning (it said it had over an hour of battery life), so I'm going to ensure that I don't use that camera again. Perhaps it shuts itself off after six or so minutes of 'inaction', or maybe it's just a dud. Whatever the reason, I've noted which camera it was and it shall be retired from my collection.
I spent about an hour and a half transcribing the seven-odd minutes of interview I did manage to record, and another half an hour coding it. It was interesting because many of the themes remain the same across both interviews.
The students seem to like constructive feedback which is clearly individualised and shows that the teacher has engaged with their work. In fact, the two students I interviewed went so far as to say that they appreciate the time some teachers take to read their work thoroughly, providing them with targets that can 'stretch' them.
It's the implications of this comment that are troubling; it implies that not all teachers take the time to thoroughly read through a students work, providing them with impersonal targets that are of little relevance; when this happens, it is clear that the students notice it. Both of the interview sessions I've conducted have seen students share their distaste for whole class targets. This is interesting because it implies that students DO engage with their targets, and even share them with their peers, checking to see whether the comments are the same.
Both groups have also suggested that more time is needed to work with targets, and that clear examples, as well as verbal clarification, would be useful. Having clear boundaries and expectations are also of use, according to the students.
That being said, some new themes emerged as well:
* the students suggested that better training is needed to make self and peer-assessment more useful - the students interviewed felt that this type of feedback wasn't useful because they found it hard to mark the work they've just completed, but suggested that more practice doing this type of work would help them become more comfortable doing it
* cues and guidance, in the form of annotations, would be helpful alongside the two stars and a wish
* annotations should be used more often
Anyways, these are my initial impressions so far. Remember, this blog is meant to serve as my 'field log', so a lot of what I write here is more of a 'train of thought' type thing that I'm doing to help me later in the data analysis process. A lot of the literature that I've read so far has suggested keeping a diary or field journal, and the best way for me to do this is digitally.
The point I'm trying to make is that you shouldn't draw too many conclusions from what I've said. I'm just giving some 'fuzzy generalisations' based on the work I've done so far.
Still enjoying the process though, and liking where the journey is taking me.
Next week I'm hoping to have a few student-conducted interviews under my belt. Until then...
KC
Tuesday, 24 June 2014
Monday, 23 June 2014
Interview number two
I conducted my second interview this morning, this time with two year 8 students. It was still interesting, but because both of the interviewees were girls who are achieving higher levels, I feel this affected their experience and understanding with feedback. I won't say much now as I have a full teaching day, but my initial thoughts are:
KC
- Annotated feedback better
- Like two star and wish
- Frequency should be regular
- Personalised comments
- Examples
- Verbal feedback accompanying
- Explanations of how to improve
- Examples
KC
Friday, 20 June 2014
Using Students as Researchers
I've decided that as part of my trial I am going to train 5 members of my form (subject to parental consent and the consent of the students) to be student-researchers, in order to help me to interview some of the younger students, specifically years 7 and 8. I'd like to be able to video record them conducting the interviews without my presence, to see if the responses they get become any more honest or clear when a 'teacher' isn't asking the questions. It will be interesting to compare how the students speak to their peers to how they speak to me. I'll pay particular attention to the body language of the students during the interviews to see if they appear more relaxed and at ease. Using students as researchers has been something I've toyed with for awhile, so I think it would be amiss of me to let this pilot opportunity to pass.
I'll keep you posted on how it goes, but I thought I'd keep you updated on my thought process.
KC
I'll keep you posted on how it goes, but I thought I'd keep you updated on my thought process.
KC
Thursday, 19 June 2014
The Pilot - Transcribing and Coding
Gosh, it's been a busy day for me! I spent the afternoon in faculty discussing Essay 2 with my fellow MEd students. It was an interesting afternoon, and I learnt a lot about the various approaches other people are taking; there are so many intelligent people in the program, I sometimes feel like I'm out of my depth! We then were given some time to think about our thesis project, so I took the opportunity to finish transcribing the interview I conducted this morning and begin some coding.
Transcribing is no easy process. I had 11 minutes and 13 seconds of video interview, which I realise now, reading it back, is brief mainly because I failed to chase things up with the students. I've made myself some notes and I know what I need to push next time. I have added to my list of 'prompts' and made a note about phrasing my 'clarification' questions as open questions opposed to the more closed 'Do you agree?' type question that I seemed to use too often this morning.
THIS is why you do pilot interviews - to work out all the kinks before you are doing your proper thesis. I understand that a lot better now. I have a feeling I'm going to learn LOADS throughout this pilot project, and not just information about feedback.
Anyways, it took me over two hours to transcribe the 11 minute interview, no word of a lie. I had a program installed that I was hoping to make use of, but it turns out I only had the trial version and that had expired. Luckily I work with a Mac, so I imported the video into iMovie and slowed down the timing to 75% so that I could transcribe it better. I went for a 'verbatim transcription' approach, where I didn't include any punctuation (beyond what Word autocorrected for me), simply typing what each person said word for word. I also wrote in actions, such as nods, as I feel that using video allowed me to record and acknowledge these sorts of non-verbal responses (yet another reason why using video recordings is a good idea for future interviews).
I then printed and read through the transcript a few times, highlighting and annotating different themes and ideas that popped up. I think this is evidence of using the grounded theory to code my responses, but I could be wrong. I'm still a bit iffy on this. My thinking was that I didn't want to use any codes that have previously been identified and used because I feel it might cloud my judgement when trying to determine themes; instead, I want to try to identify themes as they emerge to me and then see if any of these themes are repeated across my interviews.
The themes that I have identified so far (and these may need tweaking/re-wording) are:
* instructive feedback
* constructive feedback
* examples
* working with targets
* verbal clarification
* clear boundaries and expectations
* timely feedback
* better access to targets
* SPaG/aesthetics
* specific/individualised feedback
I'm going to take this initial transcript (a fresh, un-coded copy) to a colleague at school who has offered to help me triangulate my coding, just to see if I'm on the right track. I'm hoping that he'll come to similar conclusions, but if not then it will be a valuable learning experience, and will help me to create better codes for future interviews.
Anyways, I feel like I've done more than my fair share of work for today. I am now going to power myself off and go to bed. I've had a long day and I deserve a good rest.
Until next time!
KC
Transcribing is no easy process. I had 11 minutes and 13 seconds of video interview, which I realise now, reading it back, is brief mainly because I failed to chase things up with the students. I've made myself some notes and I know what I need to push next time. I have added to my list of 'prompts' and made a note about phrasing my 'clarification' questions as open questions opposed to the more closed 'Do you agree?' type question that I seemed to use too often this morning.
THIS is why you do pilot interviews - to work out all the kinks before you are doing your proper thesis. I understand that a lot better now. I have a feeling I'm going to learn LOADS throughout this pilot project, and not just information about feedback.
Anyways, it took me over two hours to transcribe the 11 minute interview, no word of a lie. I had a program installed that I was hoping to make use of, but it turns out I only had the trial version and that had expired. Luckily I work with a Mac, so I imported the video into iMovie and slowed down the timing to 75% so that I could transcribe it better. I went for a 'verbatim transcription' approach, where I didn't include any punctuation (beyond what Word autocorrected for me), simply typing what each person said word for word. I also wrote in actions, such as nods, as I feel that using video allowed me to record and acknowledge these sorts of non-verbal responses (yet another reason why using video recordings is a good idea for future interviews).
I then printed and read through the transcript a few times, highlighting and annotating different themes and ideas that popped up. I think this is evidence of using the grounded theory to code my responses, but I could be wrong. I'm still a bit iffy on this. My thinking was that I didn't want to use any codes that have previously been identified and used because I feel it might cloud my judgement when trying to determine themes; instead, I want to try to identify themes as they emerge to me and then see if any of these themes are repeated across my interviews.
The themes that I have identified so far (and these may need tweaking/re-wording) are:
* instructive feedback
* constructive feedback
* examples
* working with targets
* verbal clarification
* clear boundaries and expectations
* timely feedback
* better access to targets
* SPaG/aesthetics
* specific/individualised feedback
I'm going to take this initial transcript (a fresh, un-coded copy) to a colleague at school who has offered to help me triangulate my coding, just to see if I'm on the right track. I'm hoping that he'll come to similar conclusions, but if not then it will be a valuable learning experience, and will help me to create better codes for future interviews.
Anyways, I feel like I've done more than my fair share of work for today. I am now going to power myself off and go to bed. I've had a long day and I deserve a good rest.
Until next time!
KC
The Pilot Interview - After
Wow! I am buzzing! That was such an amazing experience, and it was only my first one. I know I made a few mistakes, but as soon as I made them I knew what I'd done wrong. I will learn from this initial session immensely, in terms of how I ask my questions. I may have led some of their responses, but I think I did it in a fairly accidental way in my quest to find clarity. That is an area I will work on.
Some brief things I found:
- students don't always like having the two stars - some find it pointless
- students weren't always able to work immediately with their feedback to improve, although the ones that did found it very helpful
- some students felt that longer targets would be helpful, especially if they were constructive
- individual targets are better than ones set for the whole class
- students would like to have specific examples or prompts to help them
- students would sometimes prefer more wishes than stars
- students need more clear signposts for what the teacher is looking for when marking - all three felt that teachers only mark for spelling, punctuation and grammar
- they'd like a checklist or grid which suggested up to 5 specific things teachers are looking for on each assessment, as well as having each specific learning goal broken down into 'at level four this looks like', 'at level 5 this looks like'
- not all students felt that peer and self assessment was useful
- all students felt that the two star and a wish policy was an improvement on their previous feedback in English
Obviously these are just my initial thoughts after the interview. I will need to transcribe and then code their actual responses to draw out more specific details. I will then need to interview other students and see what comes up across the different groups.
So far, a lot of what they said has matched the literature, which is interesting. Not many actually mentioned specific levels either, which might be something worth exploring. I wonder if I should include a specific question on what they prefer: constructive feedback or levels. Although perhaps that would be too leading - the questions at the minute allow them to mention levels if they'd like, and they sometimes did, but they never specifically mentioned levels when describing what elements helped them most.
All in all, it was a great first experience and I look forward to learning more in my next interview.
Some brief things I found:
- students don't always like having the two stars - some find it pointless
- students weren't always able to work immediately with their feedback to improve, although the ones that did found it very helpful
- some students felt that longer targets would be helpful, especially if they were constructive
- individual targets are better than ones set for the whole class
- students would like to have specific examples or prompts to help them
- students would sometimes prefer more wishes than stars
- students need more clear signposts for what the teacher is looking for when marking - all three felt that teachers only mark for spelling, punctuation and grammar
- they'd like a checklist or grid which suggested up to 5 specific things teachers are looking for on each assessment, as well as having each specific learning goal broken down into 'at level four this looks like', 'at level 5 this looks like'
- not all students felt that peer and self assessment was useful
- all students felt that the two star and a wish policy was an improvement on their previous feedback in English
Obviously these are just my initial thoughts after the interview. I will need to transcribe and then code their actual responses to draw out more specific details. I will then need to interview other students and see what comes up across the different groups.
So far, a lot of what they said has matched the literature, which is interesting. Not many actually mentioned specific levels either, which might be something worth exploring. I wonder if I should include a specific question on what they prefer: constructive feedback or levels. Although perhaps that would be too leading - the questions at the minute allow them to mention levels if they'd like, and they sometimes did, but they never specifically mentioned levels when describing what elements helped them most.
All in all, it was a great first experience and I look forward to learning more in my next interview.
The Pilot Interview - Before
Well, I'm going to conduct my first pilot interview today, with three year 9 students. I am quite excited and a little bit nervous. I've never interviewed anyone before, much less students. I'm going to ask them questions which are very similar to ones I plan to use for my thesis research, only with a more English specific focus, as the results of this pilot will be used as part of my Performance Management for the year. It will be an interesting process, because I will have to start coding the responses and eventually write a report which I will have to deliver to my head of Department.
Exciting times.
I'm also, in the name of scientific experiment, going to be trying a variety of approaches to recording and capturing the interviews. I've got a video camera set up today to record the interview, and I'll experiment with placement of the camera a few times throughout the process to see which works best. I'll also use audio recording devices, first separately and then perhaps along side video recording, just to see which works best for collecting, using and analysing the data.
I've got some free time this afternoon as well, where I will hopefully be able to write about my experiences this morning. Wish me luck!
Exciting times.
I'm also, in the name of scientific experiment, going to be trying a variety of approaches to recording and capturing the interviews. I've got a video camera set up today to record the interview, and I'll experiment with placement of the camera a few times throughout the process to see which works best. I'll also use audio recording devices, first separately and then perhaps along side video recording, just to see which works best for collecting, using and analysing the data.
I've got some free time this afternoon as well, where I will hopefully be able to write about my experiences this morning. Wish me luck!
Wednesday, 4 June 2014
The First Draft Response
You have no idea how happy I was to complete my first draft only three days after my first blog post. Dutifully, I spent the remainder of that day completing my research (or my research thus far), and then the two following days typing up the paper. I knew as I was typing that I was forgetting things, such as concrete citations for all of the facts about theory I had read earlier in the research process, but I knew that I could add those in later, after a quick skim through my notes.
Overall, I was concerned with getting the 'critical review' part of the paper done; the part that actually required me to think and analyse, interpreting the strengths and weaknesses of a peer reviewed paper, thus proving that I had come to terms with the terminology and language surrounding educational research.
I must admit that I am not 100% there. I still can't confidently tell you about the difference between methodology and methods (which my supervisor pointed out in her feedback - I'm working on it). I know more about doing phenomenological studies, as that's the type of study I want to do. I can also tell you about ethnography, ethics, and methods of qualitative and quantitative data collection. Ask me about triangulation, validity, reliability, and I will totally be able to fill you in. Ask me about trustworthiness though, and I'll fall short (yet another area highlighted by my supervisor, I mighty add).
Clearly I still have some way to go, but I'm not at all intimidated by what remains. I know I can find the information I need. I probably have the sources in my house, having bought many of the texts myself; I'm a pack rat, and I love a good textbook on my shelves. In fact, I've even been known to re-read them purely for pleasure. Yep...I'm a sucker for knowledge.
After submitting the paper to my supervisor, I did a dance around my house. I was extremely pleased with myself, and I wanted everyone to know. My boyfriend got an earful of my whoops and hurrahs when he got home from work, and I'm sure he thought I was insane. I almost felt like cracking open some wine to toast myself, but seeing as I had to attend a fitness camp shortly after submitting, that didn't seem like a good idea. Counter-productive on more than one front, really.
I knew I was going to get feedback on the work, feedback that would highlight my inadequacies, but somehow that didn't bother me because I already knew what they were. When I did receive the paper back, heavily marked up with suggestions and a few brief moments of praise (which I lapped up eagerly), I was both pleased and relieved to see that she had only, as I had thought, picked up on the areas that I already knew I hadn't fleshed out. I was again extremely relieved. In fact, I'm still feeling confident now, which is nice. I hope it lasts.
I've got a one to one supervision tomorrow, where we get to talk through the paper in person. I'm going to spend an hour before hand going over things, adding things in, and generality try and tackle some of her suggestions head on.
Overall, positive things happening at the minute! Nothing more to report, so I shall end here.
KC
Overall, I was concerned with getting the 'critical review' part of the paper done; the part that actually required me to think and analyse, interpreting the strengths and weaknesses of a peer reviewed paper, thus proving that I had come to terms with the terminology and language surrounding educational research.
I must admit that I am not 100% there. I still can't confidently tell you about the difference between methodology and methods (which my supervisor pointed out in her feedback - I'm working on it). I know more about doing phenomenological studies, as that's the type of study I want to do. I can also tell you about ethnography, ethics, and methods of qualitative and quantitative data collection. Ask me about triangulation, validity, reliability, and I will totally be able to fill you in. Ask me about trustworthiness though, and I'll fall short (yet another area highlighted by my supervisor, I mighty add).
Clearly I still have some way to go, but I'm not at all intimidated by what remains. I know I can find the information I need. I probably have the sources in my house, having bought many of the texts myself; I'm a pack rat, and I love a good textbook on my shelves. In fact, I've even been known to re-read them purely for pleasure. Yep...I'm a sucker for knowledge.
After submitting the paper to my supervisor, I did a dance around my house. I was extremely pleased with myself, and I wanted everyone to know. My boyfriend got an earful of my whoops and hurrahs when he got home from work, and I'm sure he thought I was insane. I almost felt like cracking open some wine to toast myself, but seeing as I had to attend a fitness camp shortly after submitting, that didn't seem like a good idea. Counter-productive on more than one front, really.
I knew I was going to get feedback on the work, feedback that would highlight my inadequacies, but somehow that didn't bother me because I already knew what they were. When I did receive the paper back, heavily marked up with suggestions and a few brief moments of praise (which I lapped up eagerly), I was both pleased and relieved to see that she had only, as I had thought, picked up on the areas that I already knew I hadn't fleshed out. I was again extremely relieved. In fact, I'm still feeling confident now, which is nice. I hope it lasts.
I've got a one to one supervision tomorrow, where we get to talk through the paper in person. I'm going to spend an hour before hand going over things, adding things in, and generality try and tackle some of her suggestions head on.
Overall, positive things happening at the minute! Nothing more to report, so I shall end here.
KC
Subscribe to:
Posts (Atom)