EssAI Marking? Completed it.

Happy Tuesday, human,

Two weeks ago Maidstone Grammar for Girls hosted myself, Ben White, Dan Boorman and Professor Becky Allen to discuss AI with colleagues. Becky set out her stall as to the future of education and LLMs. Myself and Dan provided many examples of how to get the best out of AI for the busy teacher. Finally, Ben took us through the dangers of smart phones. It was a great day so thank you to all those turned up. Hopefully, we can run another event in the next academic year.

 🤖 Industry updates

• Anatomically wrong! → More researchers are turning to AI image generators with disastrous consequences for published research. This paper contains this anatomically incorrect model and gibberish language. If reviewers do not clamp down on this behaviour, there could well be a crisis in confidence.

  

  An AI image used in the Wu, et. al paper

• InstructionAI Coach → Iris Connect have launched this AI-assisted lesson observation tool. The premise seems simple enough: record your lesson, get AI to feedback based on a pre-determined developmental focus based on Rosenshine’s Principles of Instruction before sharing it with a coach or colleague. I have to say, I would be interested in trying it out.

 ✨ Fresh prompts

A new study looking at the use of AI in essay marking reports:

‘We find that the current iteration of ChatGPT scoring is not statistically significantly different from human scoring; substantial agreement with humans is achievable and may be sufficient for low-stakes, formative assessment purposes.’

Here is a prompt that the authors of the studied used. Why not try it with your own essay writing? All you would need to do is upload your own rubric and change the scoring to suite your school.

Pretend you are a secondary school teacher scoring class essays based on this holistic rubric from 1 (minimum) to 6 (maximum), with the distance between each number (e.g., 1–2, 2–3) considered equal. A score of 6 means that the essay presents a clear, compelling, and accurate argument that addresses all requirements of the prompt; supports claim with relevant and sufficient evidence and compelling reasoning that connects evidence to claims; integrates sufficient, appropriate evidence from multiple sources and attributes evidence to sources, citing the title, author, and/or genre;
evaluates reliability of sources and discusses how they confirm each other; effectively addresses and refutes a counterclaim with strong evidence and reasoning; writing is well organised with a strong introduction, body, and conclusion and transitions to create coherence; demonstrates effective and varied sentence fluency with little to no errors in writing conventions; uses sophisticated language and academic tone.
Each essay is indicated by “” and the text immediately prior to each essay is the id of the essay.
For each following essay, provide only the overall score of 1–6 based on the above rubric. Do not provide feedback other than the score. Set
temperature to 0.1.

I have long suspected that something like this could be built to support Year 6 teachers when it comes to writing moderation. It certainly seems like it is something worth investigating further.

As ever, thanks for reading and keep on prompting! Mr A 🦾