This is the first blog in the CogSciSci symposium on retrieval practice in the classroom. You should read the introduction to this symposium here before reading this article. 

Retrieval Practice in the Classroom: Lethal Mutation?

In the introduction to this series, it was noted that retrieval practice (RP) can go wrong and is in danger of lethal mutation. In this entry I’m going to look at how RP can be implemented in the classroom to try and anticipate and mitigate some of the concerns raised. 

In the introduction, Coe’s non-exhaustive list of potential pitfalls with RP was noted:

 

  1. Teachers might generate retrieval questions that focus solely on factual recall (these questions are easier to generate) rather than requiring any higher-order thinking.
  2. Questions might be too easy and boost confidence without providing real challenge, which is likely to be a key ingredient for generating the kind of learning hoped for.
  3. Teachers might allocate too much time to the quizzes, effectively losing the time they need to cover new material. 

 

 

My perspective as a teacher is a little different to Coe’s, and I want to add a couple more to his list:

4. Ideas become tangled together and students associate concepts which should not be associated, leading to error further down the line

5. On a motivational level, students who day after day, lesson after lesson, get questions wrong in Do Now activities are going to get worse, not better

6. Feedback: if students don’t get good feedback, they will embed the wrong answer in their long term memory

7. The curse of genericism: my experience is that once something becomes policy in a general blanket way, bad things happen

I think Coe’s framework of skill, understanding and commitment is incredibly powerful and has helped push my thinking on this. My slightly reformulated version is:

  • Skill: do teachers have the classroom tools and craft knowledge to be able to implement RP effectively?
  • Understanding: do teachers understand the conditions under which RP is effective?
  • Commitment: how does a programme of retrieval practice affect teachers’ workload? Is it easily implementable? What is its opportunity cost?

I think it is important to add that if it turns out RP is too difficult to get right, I will be the first to argue that we should stop doing it: if an intervention is hard to pull off and carries many likely negative outcomes then it shouldn’t be done– find something easier instead. 

What follows is a description of how I try to use RP in the classroom that attempts to take into account points 1-7 above from within Coe’s skill/understanding/commitment framework. I don’t make any claims to have got it perfect yet, but I think it’s a strong start. I’ve included a summary table at the end because the different strands get a little tied together throughout what follows. 

Do Now

In my department’s policy, every lesson starts with the retrieval roulette on the board, other than if we are doing a practical or an assessment. For those who don’t know, the roulette is a simple Excel program. You put questions and answers in and it spits them out onto your board in a random order. 

It’s very plug and play: I built it to make that commitment bit a little easier for people. But the work for effective roulette use starts before you enter the classroom. Writing the questions and answers is damn difficult as you have to balance:

  1. The need for brevity generally (longer = harder to memorise and assess)
  2. The need for precise language (which often leads to brevity)
  3. The need to sequence them correctly 

Essentially, the questions and answers become a curricular tool. They help you, as classroom teacher, come to grips with what it is exactly that you want students to know and remember in the long term. They force you to think about the order in which you are going to teach different ideas and how concepts lead one to the next. If you take it seriously, writing the questions is an empowering process, and this is all completely independent of the actual RP in the classroom (for more see here).

Back to the chase. The lesson starts and the Do Now is on the board. Students have their heads down working from memory on the questions. Now is not the time to sit back and enjoy a few minutes of emails and lesson admin. Now is the time to get round the class as quickly as you can, looking at as many student responses as you can. Things you have to do:

  • Make sure all your students have actually started the damn thing. Narrate compliance (“well done for getting started Daniel…”) and use Least Invasive Intervention with students who need to get a move on
  • Some students may get stuck on a question and just stare off into space. Remind them to just move on to the next one
  • Some students may start using their notes. Tell them to stop doing this until they have tried all the questions from memory. I occasionally allow students to then use their notes and update answers in a different colour pen
  • Have a general scan at some of the responses, particularly ones which you think will be common errors. For example, I recently saw a student answer the question “which state or states of matter can be compressed?” with “gases and liquids” – I’ll come back to this later
  • Don’t bother spending ages with one student – gather data now, come back to it later
  • Keep an eye on timing. You don’t want it to drag out, but equally you want students to have as much chance to answer as many questions as possible. This is why sometimes the “go back into your notes and check” thing can work well with students who have blitzed through it 

Random or prerequisite knowledge?

The roulette is set up to do random questions from the entire course. You can also set it up to assess prerequisite knowledge for today’s learning. All teachers should start a period of new learning with a period of retrieval practice on prior knowledge. This can either be done by manipulating the roulette to give you questions relevant to today, hand picking a few yourself or just going over it verbally. None of that stops it being crucial to regularly revisit old material that isn’t relevant to today’s learning, and the random element of the roulette should ensure good coverage over time. 

Review

Once you’re happy enough students have had a good crack at the questions, it’s time to review and give feedback. Target a student to answer your question, and do the usual thing. Make sure it’s right, ask other students if they agree, that kind of thing. Let’s say you ask a student “which state or states of matter can be compressed?” and they say “gas,” make sure you then go to that student whose work you checked before and say “Dave, what did you write? Why is that wrong?” Without this, there’s good odds Dave will just tick his answer as correct, not knowing that the answer “gas” really means “gas, and only gas.”

One thing you shouldn’t do is to just stick the answers on the board. You want students to reflect on their work deeply, and if you put the answers on the board they won’t do that. They’ll just speed through at whatever pace and just try and see if there is something on the board that vaguely matches what’s on their page (even if they are well trained). Much better to go slowly one by one with you leading the discussion. 

Where’s the challenge?

This is also a good opportunity to build in further challenge and encourage higher order thinking. “Ok that’s great, we know that gases can be compressed. Can somebody put their hand up and tell me why gases can be compressed…[wait time]…Amy?” 

Coe argues that we need to think about the challenge of the questions, and as I’ve written before I don’t think that means just how many things are in the question (e.g. in what three ways can you increase gas pressure) or the abstractness (e.g. what causes gas pressure), but also the spacing introduces a desirable difficulty. An “easy” question like “what is the word for the process of a gas turning to a liquid” can provide adequate challenge if the elapsed time between initial teaching and retrieval is lengthy (for a more extended discussion of this please refer to Damian’s upcoming blog in this series.) 

It’s important to be able to look at the set of questions on the board and know which ones are the most challenging because that affects what you do with the questions afterwards and during. Sometimes I’ll see a question like “in what direction are the oscillations in a transverse wave” and think “actually they aren’t ready yet to give me a good definition” and I’ll ask them to draw a diagram instead. Equally, if a particular question for this class at this time is not challenging enough, I’ll need to have a follow-up prepared. 

As a department, an activity we do in meetings is to put a mini-quiz on the board and order the questions by challenge for our classes as they currently stand. We can then really delve behind the questions and discuss – in a subject specific, disciplinary way – what it is that makes them more or less challenging and what we will do about that in the classroom. 

I would agree with Coe that the format doesn’t lend itself to really “higher order” questioning in the sense of “application” or “transfer” questions, but that isn’t really their goal. The goal of the roulette is to prepare students so that when they are faced with questions like that (as they will be in class drill work or in assessments) they have the intellectual building blocks to formulate strong answers. 

Draw the links

During the review session, take the opportunity to show the links between some topics and to discriminate between others. Sometimes I might spend five minutes between question 1 and 2 asking further questions to the class until I get to a point at which they blend one to the next. For example, if I have question 1 asking for the word equation for anaerobic respiration and question 2 asking about the root hair cell, I might ask a ton of questions about where the various substances involved in respiration come from and go to, the purpose of respiration, where it happens, what kinds of cells need to do a lot of it, active transport and then the function of the root hair cell in transporting mineral ions into the plant. 

Obviously a lot of the time that isn’t feasible. When it’s not, make sure to distinguish between topics by being super explicit about where certain concepts come from and how they are separate to others. Don’t be afraid of saying things like “ok this is a question about electrolysis which has very little to do with bioaccumulation.” To be sure, given an hour you might be able to get from one to the other but this probably isn’t a good use of your time.

Get data

It’s vital to use the retrieval practice as an opportunity for you to gather data about your class’s burgeoning knowledge. The quickest and easiest way to do this is by doing a simple “hands up if you got question 1 correct…2…3…” and so on. 

Take a note

Let’s say there’s something a lot of your students got wrong. At this point you have two options: re-teach is option 1, and option 2 is to take a note, move on and pencil it in to tackle another time. Here’s an example of the kinds of thing each one might be appropriate for:

Reteach Take a note and move on
Something that won’t take you long Something that will take you a very long time
Something that is prerequisite for today’s learning (i.e. without it students cannot access today’s learning) Something that isn’t prerequisite for today’s learning
Something you feel confident just going straight off the bat on  Something you really want to check your subject knowledge and instructional design on first
Something you can think of practice work on the hoof for (or you have stored somewhere easily accessible)  Something you want to print or design some resources for

How long does all this take?

This is a good question, and there is no easy answer. Sometimes, I will only take five minutes over the Do Now. Other times, it could take the bulk of the lesson. This is where teacher agency is important and you have to take a departmental line. Personally, I don’t believe in content coverage for its own sake. If students are forgetting the stuff you taught them a few months ago, unless you change something you’re going to find yourself in exactly the same place in another few months’ time. So if you need the time, take the time. As above, you may prefer to put a particular topic to the side and come back to it later; that’s also fine. 

Part of this is a change in philosophy that thinks less about “how am I going to secure progress across this lesson” and more about “how am I going to secure progress over time,” where progress over time is anchored in an understanding of students’ long term memories increasing in size and complexity. 

Motivation

If your students keep getting lots of things wrong in RP, you need to stop, and you need to stop today. It’s just going to make things worse. Instead, you need to take an approach which allows them to slowly but surely start to get things right, described here and here. In short, give students a way to start feeling some success in the Do Now and more success will follow. 

We send a soft copy of the retrieval roulette home for students to access. We give them a yellow exercise book and expect them to complete and mark a set number of mini-quizzes each week (5 to 7 normally). Once a week we take in the books and have a quick scan. I take a note of students who are getting them all correct (fishy), not doing their marking properly or just generally struggling. I’ll use whole class techniques to address these issues first by literally saying “Andrew’s work was great because he was really careful to make sure that all his answers were perfect. David’s wasn’t as good because there were times where he put a tick but there was definitely detail missing from his answer. For example…”

Once the majority of your students are in good routines it becomes that much easier to have targeted conversations with those who are struggling for whatever reason. 

Hold to account

As time goes on and students get used to the process, it’s easier to hold them to account in class for their responses. It’s a lot easier to say “I’m disappointed that you don’t know that” etc if you have:

  • Given them a resource to help them “know that”
  • Got them in a homework routine to help them “know that”
  • Offered a ton of support already to help them “know that”
  • Already done this question in class a few times to help them “know that”

If you aren’t doing these things then sometimes it can feel pretty unfair. If a student in your class hasn’t been repeatedly exposed to material from six months ago, then is it really surprising that they don’t remember it now? Is it really their fault?

From a motivational perspective it’s also worth explaining why you do these mini-quizzes. Make it clear to students that it’s the best way for them to learn, make it clear that you think it’s part of your job to make sure they don’t forget things and – perhaps most importantly – make it clear to them that with a bit of hard work they can get really good at this business. 

Monitoring and tracking

Lots of options here, but the key thing is to make sure your monitoring of students is supportive and keeps the stakes low. You don’t want to stress the students out as that will ruin the results. You want them to know that it’s ok for them to get stuff wrong, that this is about strengthening their memories and making them into brilliant students. At the same time you may want to give a slightly more formal one every couple of weeks on a piece of paper that you can have a look at just to give you a less biased position on where your students currently stand. 

Policy: does it have to be at the start of the lesson?

No. Lessons are arbitrary and silly units of time. But, if you want to get into the habit of doing it, the best time is at the start of the lesson, every lesson. You also save time – students come straight in, they know exactly what to do with minimal fuss and delay. Of course you should trust your team to decide if on a particular day they want to do things differently, but having department-wide consistency and agreement on this kind of thing is very powerful. 

Policy: getting buy-in

If teachers don’t know why they are doing this, then it won’t go well and they won’t be doing most of the things I was talking about above. You need to explain it, and repeatedly. Send people studies. Have them do (FREE) online courses on retrieval practice. Invite members of the team to present for five minutes about something interesting they discovered about their class’s knowledge after a Do Now. There are lots of different things you can do to bring people in, so try to do so. 

Policy: 

To allay the concerns regarding… Make sure…

 

Factual recall and higher order thinking …that the Do Now is only the start of your lesson; use it as a springboard for further questioning
Challenge …that you know which questions are most challenging, which ones are the least, why, and what you are going to do about it.
Time  … to squeeze the most out of your time, even if it feels like you are going very slowly
Tangling ideas …to explicitly point to links between ideas and to discriminate between them when necessary
Motivation …to give your students a route to retrieval success and celebrate their achievements 
Genericism …to trust your team to create the questions, formulate how they will be used and then train to have the skill to implement in the classroom

Anyway I hope that helps and look forward to the discussion!