Anonymous reviewer

This may actually turn out to be my second post this month, which hasn’t happened since I started blogging. I don’t want to jinx things, so here goes. It’s about the Moodle workshop – their peer review activity. I suspect it works like most online peer reviews do: first you set up the task, then there’s a submission period, an assessment period and when that’s over everyone sees the feedback their classmates have given them. Pretty straightforward. Oh, and I always make it a double-blind process; I don’t think there would be as much useful feedback if the students knew whose work they were reviewing.

I’ve run the peer review a number of times now and have introduced a couple of tweaks along the way, so I thought it was about time I had some kind of written record of how things developed.

Photo taken from ELTpics by @aClilToClimb, used under a CC BY-NC 4.0 license.

In my course this activity is part of a unit on descriptive writing, so the idea is to jazz up a bland piece of writing using a number of possible strategies. The feedback students give each other is on how successfully these strategies have been used, not on language accuracy. At least that is the plan, although people occasionally have given feedback not entirely restricted to strategy use. The bland piece of writing comes from a book – I hesitate to call it a coursebook because it’s not, but the course is built on the material it covers to an extent. The book, however, doesn’t suggest the students do anything other than improve upon this piece, so the peer review is my online adaptation.

The submission stage lasts around 5 days. Moodle allows you to be quite strict about this, which means that it’s up to you to accept late submissions or not. I did the first year, but this proved to be complicated during the next stage – assessment. If you decide on a deadline by which work needs to be submitted, everyone who has submitted something can start assessing at the same time. If you accept late submissions, some students will need to wait before they can start assessing. I decided this wasn’t fair and at the risk of not including everyone in the task, I’ve now had a fixed submission deadline for a couple of years.

When you have all the submissions, they need to be shuffled around and allocated to other students. This can be done by the system or manually. I always do it manually, trying for a balance between weaker and stronger students. The assessment stage takes another 3–4 days, and again it is up to the instructor how (in)flexible they want to be regarding the deadline. I don’t think I’ve ever had everyone observe the deadline; I always need to nag gently remind some people to finish their assessment.

I could simply set a cut-off time after which the system would not allow further assessments, but my feeling is that it would be unfair on those students who have given their peers feedback but wouldn’t receive any themselves. As it is, there’s always at least one student who doesn’t assess and ignores my DM, and I then do the assessment myself so as not to hold the activity up forever.

Tweaks

  1. Something I tried in one of the early iterations of the course was to organize a second round of the activity for those students who had missed the deadline the first time around. I think I only did that once, because it quickly dawned on me that this an invitation to be taken advantage of.
  2. Also early on, the majority of students wrote their learning journal entry as soon as they completed the submission phase, which meant that few people reflected on giving feedback. This has since changed (I schedule the deadlines differently) and now most say how they feel about the peer review and what they have learned from it, if anything. There have been some interesting comments re the perceived inadequacy of someone who is not a teacher giving feedback.
  3. When the assistant mods started helping me, I asked them to assess the students’ work as well, so each student would end up with two assessments. This, arguably, is not strictly peer review in the sense that the assistant mods had done the activity when they were doing the course themselves, but perhaps it could be argued that they are still students, so in a sense it is peer review?
  4. A change introduced last year is that each student now has to give feedback to two other students. In MOOCs I think it’s common to review the work of several people, but this makes sense because of the low completion rate – you want to be sure everyone will receive at least some feedback.
  5. Last semester, when the course was run in blended format, the students did the activity online but when we next met in class we had a follow-up discussion. I picked out some of the comments from their learning journals – these are shared with the group so I knew that these were thoughts the students were more or less comfortable sharing – and in pairs they decided if they agreed with the statements. Then, when I knew each pair would be ready to say something, we discussed them as a class.

A reason I like peer review so much is that I think it is transferable to life outside the classroom. Not the submission–assessment process perhaps, but realizing the importance of giving useful feedback. Focusing on specific issues, not the person. Being helpful and identifying what could be done differently and possibly more effectively. Realizing that a piece of writing can be improved upon even if we don’t focus on simply correcting grammar errors.

How do you feel about peer review? If you run an online course do you do this type of activity with your students? How about in the classroom?

Advertisements

Behind the scenes online

Yamanaka Tamaki: seat at theater  (CC BY-NC-ND 4.0)

Since the semester began my days have been a blur of office work, proofreading and translations, and my online course. Note these are three separate categories: proofreading and translating has always been a side gig (only it used to be in addition to teaching) and now the online course is a side gig too. Although… if you were to go by the financial compensation, I’m not sure if it qualifies as an actual job or a hobby. The main reason I’ve hung on to it for now is that my office job isn’t permanent and maybe more importantly, I couldn’t imagine not doing any teaching. Period.

I finally have a bit of time to blog without feeling like there is something else I really should be doing (or at least feeling too much like there is something else… you get the idea) so I thought I would tell you what it’s like to run an online course that’s in its tenth iteration. More specifically, what this involves on part of the instructor.

First things first – the course needs a thorough overhaul. Its glory days (when it won an award) are long gone and it is scary how quickly material dates. By this I mean, for instance, the visuals I did back in 2013 (and which it must be said had a distinctly amateurish air even then) really need to go. We’ll see if and when this is going to happen: my feeling is that if I continue to teach next year and if it’s online, the course will need to be revamped.

Regardless, you might think that if all the material is up there, there isn’t much for the instructor to do. A quick reminder for those who may be new to the blog – this is a 4-month course run entirely in Moodle apart from the introductory two weeks and the final exam, which take place in a classroom environment. The course comprises 8 units, all of which take roughly 10 days, plus a revision unit at the end. If you’re thinking, no, she cannot do math, don’t forget the 2 offline weeks plus another 10-day orientation period online.

Moodle

At the start of the semester, I copy the course materials from last year into a new course. I always forget exactly how to do this; it’s not terribly complicated but it’s been a year since I last did it and it could be a tad more intuitive. What actually gets copied (in my case) is everything but the student interactions. Specifically, this means that a forum, for example, will be copied but it won’t have any contributions, including the opening post(s), which are generally mine.

Each unit is made visible to the students when the last one is over (which, I can’t resist saying, means it’s not a self-paste course – those who saw that tweet yesterday will know what I mean 😛 ). Fast finishers need to wait. It’s easier on the instructor insofar as I don’t need to go through all the units at once in September. Before I unhide each one, I read all the materials again and adapt anything that needs adapting. I realize this is vague. For instance, last year there was quite a bit of adaptation because the course was delivered in blended format, so several chapters had to be hidden and as a result the text in the remaining ones had to be rendered coherent overall and not as if something was obviously missing in between. This year a lot of the stuff that was left out has gone back in.

There’s usually something new in most units, like two years ago I redid all the screencasts which had previously been on Present.me – which I have no idea if anyone uses anymore – using Screencastomatic and uploaded them to YouTube (which I should’ve done in the first place) and added subtitles/captions – what’s it called if it’s in the same language but doesn’t include stuff like *door closes* or *phone rings*? Incidentally, that *will* happen as you’re recording, but I digress. This year I made some new visuals in Canva – okay, one so far and am planning to make more for the next unit – and I created punctuation quizzes using the cloze test question type. I was very excited about this because I learned how to do it on a course in June and … well, I was mainly excited by the fact that I was able to still figure it out in November.

I then check if the external links all still work and make sure that any activity that requires student interaction is set up properly, like add opening posts to forum discussions, links to new Padlet walls, and instructions to new wikis. Then when I’m sure we’re good to go, I add the deadline to the course calendar. I usually set one day aside when I have a couple of hours and do this for each unit.

In the meantime, while the students are working on the unit before the one I’m finishing up (the last one they can see), I keep an eye on what is going on – the pace they’re progressing at, if there’s someone who hasn’t logged on for a while and needs to be contacted, and deal with any questions. Ideally, I will also be grading and giving feedback, and have written a post about how this can be done more quickly and effectively.

Mahara

When the course is run online as opposed to blended delivery, there’s a learning journal component as well. I have written about this previously and continue to get a lot of help from my assistant moderators (usually graduate students). What I do here in terms of setting things up is limited to the beginning of the course when we need a new group that everyone can join and share their journals in a safe environment. We could argue whether the journals should be private or not; right now the course is designed so that they can be read and commented on by anyone in the group. I’m free to read student reflections and respond if I want to, or if I think a post needs responding to – not always the same thing.

Facebook

This isn’t mandatory if you run a Moodle course, obviously, but I have been using it for communication with the assistant mods. I like the option of having a private group and getting feedback on whether the mods have seen a post – normally this isn’t a problem as they are very prompt to respond – but the puppet master in me likes to think she’s got it all under control. Like the Mahara component, this requires less time as the course progresses, and mostly involves letting the mods know if an activity is coming up I’m hoping they’ll contribute to. This is defined beforehand, so the mods know at the start of the semester how much work they’ll be required to do. Of course, other questions crop up, usually to do with particular students and issues they might have experienced, and how the mods should respond.

That’s more or less it. I’m happy to be able to say that even in the tenth semester I still very much enjoy running the course and there’s always a rush of excitement when people start joining, getting to know the environment and finding their way around. As opposed to almost 5 years ago, a lot more instructors use Moodle at the institution now and so the students will generally be familiar with it, but mostly as a content repository and/or a place to submit assignments. I *am* looking forward to designing something quite new though.

Do you find that there’s a point at which an online course needs a makeover and what does that depend on?

Copy and paste for teachers

Photo taken from ELTpics by @mk_elt, (CC BY-NC 2.0)

Last week I finally got started on grading and feedback in the latest iteration of my writing skills online course, and thought I’d do a brief post on my comment bank. Note to self: see if you can come up with a catchier title than “My comment bank”.

Last summer I was listening to an episode of the Professional Adjunct podcast, in which the hosts, Beth and Jim, discuss an article called “Ten Tips for More Efficient and Effective Grading”, originally published on Faculty Focus. I’m not a regular subscriber to this podcast, but the couple of episodes I’ve listened to have addressed various aspects of teaching online – the asynchronous kind, which I do.

When they got to point 2 – comment banks – I remember it only then dawned on me that I had in fact been creating a comment bank for a few semesters without having expressly set out to do so, or even realizing that was what I’d been doing. I mean, I teach English. I’ve used coursebooks with text banks in the back of the TB, so the concept is familiar. And yet the idea of a comment bank came as a surprise – not the I-didn’t-think-anyone-else-had-come-up-with-this kind of surprise, but more of an oh-I-have-something-like-that-who-knew kind.

I find this type of resource especially useful for asynchronous online contexts or whenever you need to give feedback on work submitted electronically. Once you’ve designed a course in an LMS that covers a whole semester, it’s likely to maintain a more or less similar structure for at least a couple of semesters. For instance, there will be a discussion forum in unit 1 – it may differ in terms of the reading the students need to do to take part, or in terms of the opening post – but they’ll still be taking part in a discussion and you’ll probably want to give them a grade and feedback on that.

I originally used to add comments under headings like “Unit 1, discussion 1” to a Word document, but after a while I switched to Google Docs as I can access the bank across devices. This morning, for example, I had an hour to kill on campus, so I used the office computer to add feedback – pretty convenient.

I have a rubric for each activity that I give feedback on, so if it’s a discussion, I’m looking for a minimum number of posts, minimum number of words per post, participation by deadline, relevance to topic – this is not in order of importance – and so I will usually first comment on how successfully the student has followed the rubric. This can be taken straight from the comment bank and requires minimal adaptation. Then, if there is anything specific to a particular student that I would like to address, I will add a personalized comment. I tend to end with suggestions on what to watch out for in the next discussion, and these are often from the comment bank as well, since they come up more often than you might expect. A case in point would be encouraging them to run a spell check on their posts and pointing them to a resource we have in the course where they can find more information on how to do that, should they need to.

A more recent addition to the comment bank has been marking sections of the text in different colors for convenience. There are at least 5 or 6 comments that I will be using and/or building on for an activity, so it’s far easier to find my way around if each is a different color. I wish I could say I was color coding them – it sounds more organized – but it’s nothing as sophisticated as that; I just mark them in different color so each comment stands out from the ones above and below.

So that’s it, really. It’s a significant timesaver, relatively simple to do – okay, it does take up a bit of time the first time around, but you’ll be tweaking it every semester anyway, so it doesn’t need to be perfect – and is very convenient if you’re doing your grading via an LMS. Plus I think it makes me look at student work more objectively and fairly, although that may just be wishful thinking.

Do you use a comment bank? Any tips you’d like to share? I’m particularly interested if anyone uses anything similar in a classroom environment.