Design your own exam question

Photo taken from ELTpics by @eannegrenoble, used under a CC BY-NC 2.0 license

This is a brief post to report on something that I came across on Twitter last year and finally tried out at the final exam a couple of days ago.

There was a tweet – unfortunately, I forget by who and I didn’t bookmark it – that described an intriguing tweak to written exams. Essentially, one of the questions was left blank and students could add any question that hadn’t been asked but they knew the answer or had studied for this. As I understood it, this was entirely optional and was an opportunity to score extra points.

The tweet seemed to garner quite a bit of attention and approval but for all I know the idea isn’t as revolutionary as all that; it might only represent a novel approach in my context, which is not exactly prone to experimentation, especially when it comes to exams. In any case, I knew at once this was something I was going to try in February.

This was the question I added to the last page of the exam paper:

Is there anything else you wish were included in the exam? Something you studied or know the answer to but that question is not in the exam paper? Write down the question(s) and what your answer(s) would be and you may be able to score extra points. Of course, it needs to be related to this course.

Because I’d left it entirely up to the students how many bits of information to include, if any at all, I didn’t settle on how many points they’d actually be able to score. I had this vague idea that the answers might help someone pass if they were short of a few points or get a higher grade if they were pretty close to the cut-off point. As the exam went on, for a short while I thought nobody would take up the option of answering the question and I was sorry the exam had only been scheduled to last an hour because I thought maybe there wasn’t enough time.

Eventually, about half the group did answer. When I read the answers, I realized I’d expected them to refer to the students’ takeaways from the course and say, for instance, things like “I’ve discovered some really effective spell check tools: X and Y” or “I’m much more confident than before about where I could/should use a semi-colon,” along with a sentence to illustrate this.

Instead, they mostly referred to items they’d revised in preparation for the exam, which is not surprising given how the question was phrased. The last unit online included a screencast in which I talked about what they could expect at the exam, so they were (presumably) all aware of what to focus on during revision.

There was a category of answers that clearly didn’t aim at getting a higher grade: one student included suggestions as to how the instructions could be worded more clearly in an exercise and another said they wished they’d been asked to write an essay rather than being tested on a number of discrete points (but then didn’t go on to write an essay, saying that they doubted this would impact on their grade in any way).

Overall, I was able to use those answers where the students had shared what they remembered about the course content to give them a higher grade, so I am counting the experiment as a success, though for some reason my impression is that the person who shared the idea on Twitter was a lot more enthusiastic than me about the results. I think I’ll be using the question again, although probably not on the very next exam date.

Have you used this type of question with your students? What was your experience? If you haven’t, do you think you might? Also, if you have any ideas on how the question could be improved upon (or the link to the original tweet – it might have been phrased much better there), I’d love to hear them. Thanks for reading!

Advertisements

Reflections on reflective writing

Photo taken from ELTpics by Ian James, used under a CC BY-NC 2.0 license

Often when I’m writing a blog post I realize there’s something I could go off on a tangent about and then I vaguely decide I’ll come back to that in another post, which I don’t very often do – I guess this is due to my irregular blogging habits. This is one of those other posts: when I blogged about our introductory campus sessions earlier this semester, it occurred to me that it might be a good idea to say a few words about the learning journal component of the course, or more specifically, about reflecting on learning in an online environment and possible attendant issues.

A learning journal can be very helpful in a semester-long asynchronous course. Apart from giving students an opportunity to think through and reflect critically on the material they’ve covered, it gives the instructor an insight into how everyone is coping in a different way than student assignments do. I might find out, for instance, how students feel about the time they have available to complete tasks, what they find useful about the feedback they receive, or what they think about task types that perhaps aren’t very typical in their offline courses, such as peer review (which I wrote about in more detail in this post). If the course were held on campus, I would probably be able to find much of this out in class.

The course has included this component since I first moved it online and has undergone a couple of tweaks in the meantime. Initially, the students had complete freedom re what they chose to reflect on after each unit, in that there were only some broad suggestions on the type of information they might want to add. The problem with that, it soon transpired, was that although some students clearly did not lack inspiration, there were others who found reflecting challenging, felt they didn’t have much to say or failed to see the point of the activity. Whatever the reason, students in this category wrote exceedingly brief comments whose sole purpose, I suspect, was to tick the “post a reflection in your journal” box.

At first, I tried to address this by posting questions on these students’ entries, hoping they would respond in greater detail, which met with varying degrees of success. After a couple of semesters, I added questions that could serve as writing prompts after each unit. Looking back, I have no idea why it took me so long to do this – I guess it was probably because I thought these were questions students should actually be asking themselves and they needn’t be the same for everyone. I still think so, but learning journals aren’t common practice in the Croatian education system and given that I was aware of this from the start, I’m surprised it didn’t occur to me sooner that students might find model questions useful. The semester I introduced questions student journals became noticeably more focused overall.

Some time after this, I began covering reflective writing in the introductory face-to-face sessions as well, the idea being that this would help students see the learning journal as more than just an afterthought. I begin by explaining what this component entails and show the students a sample journal from an earlier semester, to illustrate what the final product looks like. I choose one at random, although I think I’ll have to start checking with ex-students if I have their consent, on account of GDPR. Afterwards, we take a look at one of those exceedingly brief comments from one of the early iterations of the course, discuss what seems to be lacking at first glance and how each point could be expanded on. This is followed up by a few general good practice suggestions on reflective writing.

What I try to do in the session before this one is set aside 15 minutes for students to answer 2-3 questions of the type they will be addressing in their learning journal entries. This can be at any point during the session. I simply ask them to answer the questions however they think they should best be answered in the next 10 minutes or so. At this point I don’t want students to think about reflective writing as a genre, so there is no guidance nor are there any constraints apart from the time they have available.

I collect these and in the next session, after we’ve talked about how a very general comment can be made more specific, I show them a few examples of how this has been achieved in the pieces of writing they handed in in our last session. These are anonymized but I hope people recognize what they’ve written and that it has a motivating effect. Some terms are marked in red because they are still a little vague and we discuss how these parts of the text could be rendered more specific.

This in combination with the questions to reflect on after each unit generally produces good results. There are still students each semester who struggle with what to write about but after they’ve received personalized feedback on their first reflection, suggesting how they could expand on areas that may be overly general and thus possibly not so useful, their reflections generally become more detailed and specific.

One thing I’m not as happy about is the fact that since the questions have been introduced, the majority of students rely on these and rarely choose other aspects to reflect on, even though the instructions always stress that the questions are only there to provide inspiration and don’t (all) *haveto be answered.  This tends to make the reflections a tad predictable in structure, and to an extent in content. 

Another thing I sometimes feel I could use some help with are the questions themselves. I tweak them most semesters, adding new ones and removing those which don’t seem to have been helpful or produced much engagement. If you know of any resources that provide suggestions on how to structure reflection questions or which aspects of learning to target, they would be much appreciated!

Leveling up

Stefanie L: cat watching tv (CC BY-NC-ND 2.0)

A couple of weeks ago I blogged about H5P and how excited I was to discover this new resource I could make use of in Moodle. In that post I described the process of setting up a drag & drop activity and adding it to my online course. I was sure I wanted to try out a number of other content types – which is what the 40 odd H5P activities are officially called – but I wanted there to be a reason for adding them, apart from novelty and the thrill of experimentation.

There are a few screencasts in the course, which I thought I could use the interactive video content type with. A little bit of background on the screencasts: they started out as presentations I used when I taught the same course offline. Yes, they were PowerPoint, but I didn’t think that was enough to ditch them, especially as they were brief and had been designed to get the students to interact with the content. The first time I moved them online I used Present.me, which I’m not sure even exists anymore. About three years ago I re-recorded them, uploaded them to YouTube and added subtitles: a far more user-friendly experience overall.

There aren’t many – six in a four-month course – partly because they’re pretty time-consuming to make for someone who doesn’t do this on a regular basis and partly because I don’t think a writing skills course actually requires many. The longest screencast is just under ten minutes, if you don’t count the one in the revision unit, where I chat about what students can expect at the exam (a little under 15 minutes). That one is unscripted and those are likely to be longer anyway.

Eventually I settled on the longest screencast – the ten-minute one – to experiment with. As with the drag & drop, I first added the H5P interactive content activity to the course and selected the content type: this time around it was interactive video. You can either upload a video directly (in which case I think there’s a size restriction) or add a link to YT, which was the route I took. One aspect I was immediately unhappy about was the disappearance of subtitles; apart from the fact that they’re important to ensure accessibility, I think they can be helpful even for pretty advanced students. I got around this (sort of) by embedding the YT video directly below the interactive one and recommending the students first try the interactive version, then watch the one with the subtitles if they felt they needed them.

The screencasts are based on short sets of slides that are often meant to be presented in the following way: I do a bit of talking, the students work in pairs to answer a question or discuss it as a group, and then we check their ideas on the next slide. Because of this, in the recordings I would often ask the students to pause the video and try to answer a question I’d asked – one that they would address in pairs or groups in class. I’d suggest they make a note of their responses somehow, so they could compare them with what came next in the screencast. These were great natural places to add questions to the video and I took advantage of them.

The interactive video content type lets you add a range of different question/interaction types, (MCQs, T/F, drag the word – which can be used for gapfills – matching, and more). I was able to add a link to external content as well, and at the end I wrote up a brief summary of what the video was about and made it into a gapfill activity. I rather liked the option of having the students choose the best summary out of three possible ones – I gather this is a separate question type – but it seemed like it might take a while to set up and I didn’t have much time.

Another advantage of these H5P activities is that you can view the results of the interactions in the Moodle gradebook and see how well the students did on average, as well as if there’s anyone who seems to need a little extra help. Of course, what you can’t see is whether those who did well maybe did a bit of research before answering the questions or if they simply knew the answers from before, nor can you see if those who did less well rushed a little/took a random stab at the answers or if they didn’t really understand/follow the explanations in the video. I did add a question about this to the list of questions the students might want to address in their learning journals, so we’ll see if any interesting insights emerge.

How do you feel about interactive videos: have you used them with your students? Are there any effective tools you would recommend for this besides H5P? I recently came across an article which recommended Edpuzzle, but I’m sure there are others. Thanks for reading!