Another dot in the blogosphere?

AI-yai-yai

Posted on: November 27, 2019

Two weeks ago, I shared this announcement about Singapore’s ten-year plan for AI and focused on how it might affect schooling.

I left my reflection on AI for grading on slow burn for a while. I am enjoying a break, but I also enjoy wrestling with dubious change.

Yes, dubious. But first, two pretexts.

First, the vendors that the Ministry of Education, Singapore, works with are not going to be transparent with their technologies, so I cannot be absolutely certain of the AI development runway, timeline, and capabilities.

Next, the field of AI is not new and it is diverse. Parts of it evolve more quickly or slowly than we might expect. For example, handwriting recognition has been around since before Microsoft released its slate PCs. It was good enough to recognise some doctor scrawls even back then!

However, the Hollywood vision that AI will replace or even kill us off has not materialised. An expert might point out that AI is not good at making social predictions and ethical decisions. I simply point out that artificial intelligence is still no match for natural stupidity.

Back to the issue — we need to consume claims made by policymakers and edtech vendors critically. And more critically if they are reported by the mainstream media that thrives on sensationalism.

Do not take my word for it. Take this expert’s view that some claims are “snake oil”. In his slides, he put these claims into three categories: Genuine and rapid progress; imperfect but improving; and fundamentally dubious.

An expert’s view that some AI claims are “snake oil”. In his slides, he put the claims into three categories: Genuine and rapid progress; imperfect but improving; and fundamentally dubious. Slide #10 at https://www.cs.princeton.edu/~arvindn/talks/MIT-STS-AI-snakeoil.pdf
Source

I highlighted “automated essay grading” in the screenshot above because that coincides with our 2022 plan to “launch automated marking system for English in primary and secondary schools”.

The fundamental issue is AI’s ability to automate judgement. Some judgements are simple and objective, others are complex and subjective. Written language falls in the latter category particularly when the writers get older and are expected to write in more complex and subjective ways.

Anyone who has had to grade essays will know what rubrics and “standardisation” sessions are. Rubrics provide standards, guidelines, and point allocation. Standardisation meetings are when a group of assessors get a small and common set of essays, grade those essays, and compare the marks. Those same meetings set the standard for the definitions of subjectivity, disagreement, and frustration.

Might AI in three years be able to find the holy grail of objective and perfect grading of subjective and imperfect writing? Perhaps. If it does so, it might be less a result of rapid technological evolution and more one of social manipulation.

To facilitate AI processing of essays, students might be required to use proprietary tools and platforms. For example, they might have to use word processed forms instead of handwriting. They could be told to write in machine readable ways, e.g., only five paragraphs, structured paragraphs, model phrases, etc. In other words, force-fitting writers and writing.

This is already how some tuition and enrichment centres operate. They reduce essay writing to formulae and teach these strategies to kids. Students are not encouraged to make mistakes, learn from them, or develop creative and critical thought. They are taught to game the algorithms.

The algorithms are the teachers’ expectations and rubrics now. They could be the AI algorithms in future. But the same reductionist strategy applies because we foolishly prefer shortcuts.

The AI expert I highlighted earlier focused on how ill-equipped AI is to predict social outcomes. He concluded his talk with this slide.

Concluding slide (#21) from https://www.cs.princeton.edu/~arvindn/talks/MIT-STS-AI-snakeoil.pdf
Source

We might also apply his last two points to automated essay grading: Resist commercial-only interest aimed to hide what AI cannot do, and focus on what is accurate and transparent.

This is not my way of stifling innovation as enable by educational technology. I wear my badge of edtech evangelist proudly. But I keep that badge polished with critical thought and informed practice.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

This site uses Akismet to reduce spam. Learn how your comment data is processed.

http://edublogawards.com/files/2012/11/finalistlifetime-1lds82x.png
http://edublogawards.com/2010awards/best-elearning-corporate-education-edublog-2010/

Click to see all the nominees!

QR code


Get a mobile QR code app to figure out what this means!

My tweets

Archives

Usage policy

%d bloggers like this: