“I am not happy with my mark” – Tough!

A lecturer releases coursework grades for a recent assessment and then emails begin to flow into their inbox from students who are not at all happy with their results. One student for example did ok on the Multiple Choice Questions (MCQs) but the Short Answer Question (SAQ) was below a pass grade. The email which was brief and very much to the point stated “I am not happy with my mark and would like to arrange a meeting to discuss this with you”. The lecturer begins to feel overwhelmed with the emails and thinks “so you don’t agree with the mark, tough!”

This was a conversation I had recently with a colleague, the emotional responses both from the students to their feedback and the immediate feelings of the lecturer, wanting to do the best they can and feeling pressure from students, feeling like they have to do more and more. This creates and contributes to a sense of resentment, the belief that students are not taking responsibility for their learning and simply complain about their grades. A barrier is created for effective feedback dialogue.

Dr Naomi Winstone’s research on learners’ proactive recipience within the feedback process has been a catalyst for me professionally, prompting our investigations into approaches that support feedback dialogue in our Programmes. A funded project on “Demystifying Academic Expectations” has enabled some further insights into approaches that will impact positively on our students, empowering them to learn more effectively.

So what happened with the student who emailed?

After the initial TOUGH response, acknowledging those reactive emotions that surfaced, the lecturer responded by email and arranged a meeting with the student.

The lecturer made sure that they were in a calm state ahead of the meeting so that they could have an adult to adult conversation with the student. They had in the back of their mind, that initial response that this student might just want to argue the mark but also that they could be wrong, the student might be concerned and looking to understand how to improve. The important point, let’s not jump to conclusions ahead of the meeting.

After exchanging pleasantries, the lecturer asked them, “how well do you think you did on this assignment?”

The student explained their concerns, they were performing better on other aspects of their studies and this grade worried them because it was the SAQ that resulted in the poor mark overall. They will be writing further SAQs on other modules and in exams so understanding what is not working for them is very important.

The lecturer explained the criteria and standard against which their work is being judged and then examined the student’ work against this to highlight the feedback and have a discussion on that.

From the conversation with my colleague, the feedback dialogue process that took place between the lecturer and student had a very positive outcome. Creating the right environment to have that conversation was key.

The lecturer then addressed answering SAQs in a subsequent workshop with all the students that supports demystifying academic expectations.

  • An activity to help students focus on the verbs used in questions and what the difference between verbs used was the first part of the workshop. Students wrote in their own words what they understood was meant by “define”, “discuss”, “explain” etc.

activity

  • Then the lecturer introduced Bloom’s Taxonomy in a brief format and the development of intellectual thinking skills. So for a first year student remembering and understanding are important skills with perhaps some applying. In second year the skills of applying and analysing develop to distinguish the expectations required between those levels of study. The students were then asked to consider the verbs on the activity sheet and consider which levels of Bloom’s Taxonomy are addressed.
  • Following discussions, a summary of frequently used verbs was presented where students could compare to their own interpretations.

Verbs in questions_title (2)

  • The students then examined some questions from different topics so that they could apply this approach within a specific context.
  • The final part of the workshop involved exemplars that students could evaluate. The exemplars produced by the lecturer were created to illustrate an outstanding example, a good example and a poor example. This is something that Prof Phil Race mentioned to me when I had an insightful conversation about exemplars. Thank you Phil. The lecturer deliberately included some errors in the exemplars that students included in previous assessments so this was feeding back to the students their errors. The students evaluated each exemplar and then using a think-pair-share approach they discussed their evaluations with each other.
  • The lecturer then gave the students a behind the scenes look into how they would mark the exemplars, this is the demystifying aspect, helping students to understand what goes on from the academic perspective. Students could ask question to clarify the differences between the exemplars and how the lecturer awarded and deducted marks.

Student feedback on this has been extremely positive, when you hear students use “amazing” it must have had a positive impact on their learning experience which ultimately is the reason we devise and pilot new approaches in our teaching. I think unpacking the ‘black box’ and demystifying academic expecatations can only help and support our students.

 

Winstone, NE, Nash, RA, Parker, M & Rowntree, J 2017, ‘Supporting learners’ agentic engagement with feedback: a systematic review and a taxonomy of recipience processes‘ Educational Psychologist, vol. 52, no. 1, pp. 17-37. https://doi.org/10.1080/00461520.2016.1207538

Student Marking Exemplars – flipping a revision session for answering a SAQ

How to answer a Short Answer Question? This is the most frequent request I get from students when asked what they would like to cover in a study revision session. Their exams consist of a combination of Multiple Choice Questions (MCQs) and Short Answer Questions (SAQs). Typically for each student their performance on the SAQs is lower than for the MCQs. Not a surprise, it’s much harder to synthesise an answer on a blank sheet of paper than to recognise or work out the correct answer from a list of plausible options.

My 1st year chemistry students experience answering a formative SAQ under exam conditions ahead of a summative mid-term test to help familiarise them with what to expect. Feedback during this session helps to clarify what a good SAQ answer consists of using the formative SAQ as a specific example.

The key message I emphasise is to show both knowledge and understanding in their answers. If the question uses list or identify, then this requires knowledge of the correct answer. It is not asking to explain your answer or justify your reasoning. If the question asks for an explanation or to discuss the answer then understanding is required. The weighting on question parts within a SAQ can also indicate this with knowledge only questions weighted less than knowledge + understanding questions.

knowledge

The students were also provided with written guidance on answering SAQs which included the main types of questions for 1st year chemistry.                               Top Tips for answering a SAQ                                               

top tips SAQ

A short answer questions means that the answer is relatively short compared to an essay. The answer may involve working out a calculation or may consist of several sentences and could include chemical structures/diagrams.

An answer that’s longer than necessary won’t cause you to lose marks, as long as everything you write is correct. But if incorrect, you will lose marks. In addition, writing more than necessary wastes time that could be spent on other questions. So you should only write more if you think it’s correct and crucial to answering the question.

Despite this guidance and further feedback from all staff within the module during smaller workshops, there was still a lack of confidence from the students with SAQs.

So I decided to flip a revision activity and provide the students with two exemplars answering a SAQ on the sample exam paper. I created the two answers, both with deliberate errors and the activity required the students to mark the questions. I explained that the students should mark both answers according to a 40:60 ratio for knowledge: understanding demonstrated in each answer. I wanted to revisit the initial guidance from the formative workshop on knowledge and understanding. Answer A was almost a top answer but included an error. Answer B was incorrect but aspects of the answer demonstrated correct understanding of the key concepts asked in the question. The aim here was to illustrate how we look to reward correct knowledge and understanding, that chemistry SAQs are not necessarily text only as they may include chemical structures or figures. I marked Answer A as 90% and Answer B as 40%.

It was clear early on that this activity challenged the students to think, they had to reflect on their own level of knowledge and understanding of the topic whilst marking the answers.

What was most interesting was the marks the students gave to both Answer A (my mark 90%) and Answer B (my mark 40%). I used mentimeter to capture their marks and it caused amusement when student marks were polled as the range of marks was indeed quite large!

The average student mark for Answer A was 56% and the average for Answer B was 54%. The highest mark for Answer A was 80% and the lowest was 20%. A similar pattern was seen for Answer B, highest mark 82%, lowest 30%. The students were challenged with the process of evaluation and from Bloom’s Taxonomy, unless good knowledge and understanding exists as a strong foundation, the intellectual skills of evaluating and analysing will be difficult to achieve. In terms of Answer B, 1/3 of the students gave a grade above 60%. Student feedback on the session was very positive, it got them thinking about SAQs from a new perspective and it was good feedback to them on what they needed to revisit in their revision. I used this activity as a pilot and will definitely want to repeat again including other topics and the critical aspect is to include some common errors or misconceptions in the exemplar answers.

Blooms

Bloom’s Taxonomy (I don’t represent anymore using a pyramid)

Feedback and the Ryanair effect

What has Ryanair got to do with feedback in education? If you have ever had the pleasure to fly with Ryanair then you will have met the Ryanair effect! Knock airport in the west of Ireland is my destination frequently,  only 1 hour 10 mins from Luton. When the plane touches down on the runaway which was built over boggy terrain,  a familiar sound rings out – the Ryanair trumpet fanfair! The recording begins and a voice sounds

“Thank you for flying with Ryanair” followed by

“Last year, over 90 per cent of our flights arrived on time. We hope you enjoyed yours and we look forward to welcoming you on board again soon.”

Yet another on time flight! I have heard that jingle so many times, I think the association has become hard-wired into my brain! Now if Ryanair asked me in a survey,  “do our flights arrive on time?” I would find it difficult not to agree and answer positively, why? because they have reminded me so many times that their flights arrive on time.

So let’s transfer to feedback. Colleagues from my Department were feeling exasperated with student dissatisfaction on feedback in the post module questionnaire. The teaching team were giving so much quality feedback, feedback prior to activities, feedback immediately after tasks, feedback 1 week after some assessments but always delivering feedback in a timely manner according to the university regulations. There was no way to increase the feedback even if they tried!

And so they decided to have some fun with a serious intent as I like to frame it. They ordered bright pink caps with the word FEEDBACK clearly visible in white to use the following academic year. feedback-caps-2

The pink caps were worn anytime feedback was provided to the students. The students found it amusing, the staff had a giggle too, well you really couldn’t get overly serious wearing a bright pink cap! But the serious intent came to light in the student ratings at the end of the module. There was a 36% increase in the rating on feedback being timely and 26% increase in the rating on feedback being helpful.

Remember nothing changed from the previous year in relation to feedback other than highlighting when it was happening by wearing the bright pink cap. A very happy teaching team feeling justly recognised for their efforts. For me this is exactly like the Ryanair effect with their annoying yet memorable jingle.

So when it comes to feedback, do our students actually recognise feedback? Probably not unless we help them to identify it. Most students would say feedback is the comments they receive on their work. But feedback is more than that.  How can we help students to identify feedback? Small quick wins, this is definitely one of those.