Student Marking Exemplars – flipping a revision session for answering a SAQ

How to answer a Short Answer Question? This is the most frequent request I get from students when asked what they would like to cover in a study revision session. Their exams consist of a combination of Multiple Choice Questions (MCQs) and Short Answer Questions (SAQs). Typically for each student their performance on the SAQs is lower than for the MCQs. Not a surprise, it’s much harder to synthesise an answer on a blank sheet of paper than to recognise or work out the correct answer from a list of plausible options.

My 1st year chemistry students experience answering a formative SAQ under exam conditions ahead of a summative mid-term test to help familiarise them with what to expect. Feedback during this session helps to clarify what a good SAQ answer consists of using the formative SAQ as a specific example.

The key message I emphasise is to show both knowledge and understanding in their answers. If the question uses list or identify, then this requires knowledge of the correct answer. It is not asking to explain your answer or justify your reasoning. If the question asks for an explanation or to discuss the answer then understanding is required. The weighting on question parts within a SAQ can also indicate this with knowledge only questions weighted less than knowledge + understanding questions.

knowledge

The students were also provided with written guidance on answering SAQs which included the main types of questions for 1st year chemistry.                               Top Tips for answering a SAQ                                               

top tips SAQ

A short answer questions means that the answer is relatively short compared to an essay. The answer may involve working out a calculation or may consist of several sentences and could include chemical structures/diagrams.

An answer that’s longer than necessary won’t cause you to lose marks, as long as everything you write is correct. But if incorrect, you will lose marks. In addition, writing more than necessary wastes time that could be spent on other questions. So you should only write more if you think it’s correct and crucial to answering the question.

Despite this guidance and further feedback from all staff within the module during smaller workshops, there was still a lack of confidence from the students with SAQs.

So I decided to flip a revision activity and provide the students with two exemplars answering a SAQ on the sample exam paper. I created the two answers, both with deliberate errors and the activity required the students to mark the questions. I explained that the students should mark both answers according to a 40:60 ratio for knowledge: understanding demonstrated in each answer. I wanted to revisit the initial guidance from the formative workshop on knowledge and understanding. Answer A was almost a top answer but included an error. Answer B was incorrect but aspects of the answer demonstrated correct understanding of the key concepts asked in the question. The aim here was to illustrate how we look to reward correct knowledge and understanding, that chemistry SAQs are not necessarily text only as they may include chemical structures or figures. I marked Answer A as 90% and Answer B as 40%.

It was clear early on that this activity challenged the students to think, they had to reflect on their own level of knowledge and understanding of the topic whilst marking the answers.

What was most interesting was the marks the students gave to both Answer A (my mark 90%) and Answer B (my mark 40%). I used mentimeter to capture their marks and it caused amusement when student marks were polled as the range of marks was indeed quite large!

The average student mark for Answer A was 56% and the average for Answer B was 54%. The highest mark for Answer A was 80% and the lowest was 20%. A similar pattern was seen for Answer B, highest mark 82%, lowest 30%. The students were challenged with the process of evaluation and from Bloom’s Taxonomy, unless good knowledge and understanding exists as a strong foundation, the intellectual skills of evaluating and analysing will be difficult to achieve. In terms of Answer B, 1/3 of the students gave a grade above 60%. Student feedback on the session was very positive, it got them thinking about SAQs from a new perspective and it was good feedback to them on what they needed to revisit in their revision. I used this activity as a pilot and will definitely want to repeat again including other topics and the critical aspect is to include some common errors or misconceptions in the exemplar answers.

Blooms

Bloom’s Taxonomy (I don’t represent anymore using a pyramid)

Feedback and the Ryanair effect

What has Ryanair got to do with feedback in education? If you have ever had the pleasure to fly with Ryanair then you will have met the Ryanair effect! Knock airport in the west of Ireland is my destination frequently,  only 1 hour 10 mins from Luton. When the plane touches down on the runaway which was built over boggy terrain,  a familiar sound rings out – the Ryanair trumpet fanfair! The recording begins and a voice sounds

“Thank you for flying with Ryanair” followed by

“Last year, over 90 per cent of our flights arrived on time. We hope you enjoyed yours and we look forward to welcoming you on board again soon.”

Yet another on time flight! I have heard that jingle so many times, I think the association has become hard-wired into my brain! Now if Ryanair asked me in a survey,  “do our flights arrive on time?” I would find it difficult not to agree and answer positively, why? because they have reminded me so many times that their flights arrive on time.

So let’s transfer to feedback. Colleagues from my Department were feeling exasperated with student dissatisfaction on feedback in the post module questionnaire. The teaching team were giving so much quality feedback, feedback prior to activities, feedback immediately after tasks, feedback 1 week after some assessments but always delivering feedback in a timely manner according to the university regulations. There was no way to increase the feedback even if they tried!

And so they decided to have some fun with a serious intent as I like to frame it. They ordered bright pink caps with the word FEEDBACK clearly visible in white to use the following academic year. feedback-caps-2

The pink caps were worn anytime feedback was provided to the students. The students found it amusing, the staff had a giggle too, well you really couldn’t get overly serious wearing a bright pink cap! But the serious intent came to light in the student ratings at the end of the module. There was a 36% increase in the rating on feedback being timely and 26% increase in the rating on feedback being helpful.

Remember nothing changed from the previous year in relation to feedback other than highlighting when it was happening by wearing the bright pink cap. A very happy teaching team feeling justly recognised for their efforts. For me this is exactly like the Ryanair effect with their annoying yet memorable jingle.

So when it comes to feedback, do our students actually recognise feedback? Probably not unless we help them to identify it. Most students would say feedback is the comments they receive on their work. But feedback is more than that.  How can we help students to identify feedback? Small quick wins, this is definitely one of those.