Don’t make the mistake of ignoring student feedback in curriculum reviews

We hand year 11 and 12 students the syllabus. “This is the most important document for this subject,” we say. If we expect students to eat, breathe and live the syllabus, shouldn’t they also have a say when it comes time to review it?

The NSW Education Standards Authority (NESA) is doing a wholesale review of its Stage 6 (year 11 and 12) courses. Teachers and industry will be carefully crafting submissions.

Students, the party with the most skin on this game, won’t be doing the same. But they should have their voice heard.

Ask any year 12 graduate about their syllabi and you’ll have to put a time limit on the conversation. They’ve got a litany of complaints and suggestions; many of which would improve the experience of students studying a subject. Which, if implemented, could motivate them to recommend the course to a peer.

There’s no point asking year 12 graduates to write formal submissions. Instead, what about focus groups or online surveys, with participants facilitated by schools?

Senior syllabi are packed. As an NSW Economics teacher, I would consistently struggle to complete the course to an in-depth level on time. If I felt this pressure, how might students feel as time marched on and dot points remained to do?

When it comes to the syllabus, Let’s hear from the students. And let’s hear from as many as possible — public/private, rural/metro, single sex and coed. The more voices, the better.

You should (consider) changing your multiple choice answers

I vividly remember teacher after teacher telling me DON’T change my multi choice answers.

“Your first answer is most likely the most correct,” I heard their collective voices say.

You may have heard some similar advice. But what if this piece of conventional advice is wrong? What if we should think again about changing our minds — with multis and life in general?

This is the premise of a book by Adam Grant, an organisational psychologist, a bit of a rock star academic, who has best selling books, popular TED talks and a large following.

Grant has written a book titled Think Again. The tag line: “Discover how rethinking can lead to excellence at work and wisdom in life.”

Another few choice quotes: 

• “Embrace the joy of being wrong”

• “You don’t have to believe everything you think”.

The reason the book stood out to me is because Grant has a section in the prologue focused on multiple choice questions. He discusses the conventional belief about not changing your first answers and then offers two contrary views.

First, he discusses that when a group of psychologists reviewed thirty‑three studies, “they found that in every one, the majority of answer revisions were from wrong to right”. 

Second, he talks about how psychologists counted eraser marks on the exams of more than 1,500 students in Illinois. Only a quarter of the changes were from right to wrong, while half were from wrong to right. 

So, what should you take from this? 

Let’s start with something you should avoid: don’t second guess yourself as you go through the multis in an exam. Annotate the questions, eliminate the least likely and go for the most correct response (based on the key words of the question).

Here’s something you can apply from Grant’s work. When you are reviewing your multi choice answers, consider changing your response. Consider the fact that you may have another look, ‘think again’, and discover a more correct response. This is an acceptable strategy and, as the research shows, it can be an effective one.

Also, I suggest reading the whole book. It’s a really interesting read with lots of great real-world examples. Adam Grant’s book always make me feel wicked smart. Also, check out my video on the topic below.

My current annoyance with marks

In terms of, well, everything, marks are extremely frustrating to me. You hold an assessable task. Students crave the marks. Teachers obsess over the marks. Then...the marks disappear and the process starts again.

In an ideal world, the marks would disappear and the feedback would stick. In fact, in an ideal world, students would compare feedback with each other, rather than marks. They would take the time to carefully consider the feedback that teachers have carefully considered. 

I have lots of grudges against marks. But today’s issue is that marks are a snapshot of where students have been, rather than where they could be.

Students are fixated on a number that represents a point in time in the past. That’s the mark. Feedback, however, represents where students could be in the future. I think that’s more valuable. 

There’s this great documentary on Netflix called The Playbook that looks at different coaches and their approaches. The first ep is with NBA coach Doc Rivers. He makes this great point about how coaches aren’t focused on where a player is today but rather where they could be tomorrow. And this is the power of feedback. 

Good quality feedback is like the road map to future success. Marks are like looking in the rearview window. Students have the choice about where their focus lies. 

Also: I’d really love to see students compile a list of all teacher feedback they’ve received and look to apply it to all their future tasks. It’s something I’ve suggested, but no-one’s bit so far.

Help students add more detail

I recently marked Year 12 half-yearly exams and the two most comment bits of feedback I gave:

• Be specific

• Fully explain your points.

Generally, I find student responses meander. They don’t give concrete details and they only offer superficial explanations. This is frustrating — for students and teachers. For students, because they can’t access higher marks. For teachers, because students could do so much better if they only did some elaboration.

I try and help my students provide more detail and explanation through a particular activity. I’ve named this ‘What works, what doesn’t’.

I make it simple for myself. I take a past exam multiple choice question, such as the one I’ve included below. This comes from the 2017 NSW Economics HSC (Source: NESA).

Then, I paste this onto a doc and create a very simple table that sits below the multi question.

Students then complete the task. They must provide the explanation — the specifics, the details — to clearly articulate why the response is correct or why it is incorrect.

I make it clear to students: if you say, “because it’s the right answer” or “because it’s wrong answer”, that’s insufficient. You’ll be asked to try it again. 

As I circulate in the class, there’s a couple of prompts I’ll use to get students to add to their explanation:

• What makes you say that? (Thanks Project Zero!)

• Why is this the case? 

• What’s the error in logic that’s being made here?

An extra step is having students go through the process verbally. To have them provide their explanations in a conversation with you, rather than writing them down. This is time consuming and it’s not possible to interact with every student in this way. But, when you can do it, I find it very valuable in checking on student understanding and their ability to explain. In detail.

A methodical approach to multis

I think multiple choice questions are an amazing teaching opportunity in economics. 

Think about how tough it is to create a multiple choice question. You need to choose some content, link it to other content or a hypothetical example, make the students think and then include answer options that are similar but still have a clear(ish) winner.

Think also about the value in having students decode and deconstruct multis in a very methodical way to test their content knowledge. I see this as a valuable endeavour.

I’ve created a video (see below) going through all the multis in the 2019 NSW Economics HSC. Check out the exam and marking guide. The video takes you through my process, which I in turn share with my students in class. I am methodical because I want them to be methodical. I want them to take the time now, while they’re practicing, to really interrogate the questions and thoughtfully exclude options, not just identify the correct answer.

Let’s be clear: I don’t expect students to undertake all of this detailed process in their exams. But I do expect students to carry through some of these strategies, particularly some form of annotation. I continually encourage my students to do this all the time, multis, shorts, essays, whatever.

Oh. If you’ve got a better way of answering question 14, let me know in the comments. I didn’t love that one. 

Multi thoughts on improving multis

Economics exams love multiple choice questions. These multis are supposedly designed to test student knowledge and often confuse the hell out of people. How can you help your students do better on these questions?

One strategy I use with my students is to do away with the answer categories. You can physically take them off or just cover them with a piece of paper.

Here, you’re asking student to stop jumping straight to the answer categories. Instead: ask your students to think about likely answers, prime their thinking, jot a few thoughts down, then look at the answer categories.

Let’s have a look at how this could work. Here’s a multiple choice question in its entirety.

Don’t give this to your students. Instead, just give them the question:

Ask them to list their thoughts. Maybe they start talking about people out of the labour force, about people who have given up on seeking work. Maybe they talk about people who are NOT classified as unemployed. All of this process stuff is good. Really good.

Then, put the answer categories back and ask them to answer the question.

The idea is to help students put a routine in place. I want students to stop jumping into questions and start by thinking carefully about the question.