Another dot in the blogosphere?

Posts Tagged ‘grading

 
I am in the middle of an intense grading exercise. This happens every semester because I get at least one Friday class which has deadlines on Friday evenings. This means that I burn weekends to give timely feedback to my students.

This practice is not unusual because I was a teacher and professor. I am married to a teacher and my parents were teachers. Marking scripts on weekends is a sad family norm.

What I find irritating is that I need to have at least five (sometimes six) web browser tabs open to provide feedback and grade. These is a tab each for:

  1. The institutional LMS (to access the online scripts)
  2. The feedback tool (for commenting on student work)
  3. A timer (because the LMS page times out every 30 minutes)
  4. A Google Sheet (for recording my marks)
  5. A Google Doc (for backing up my comments)

When I am in a public place like a library, I have a sixth tab to play soothing music through my headphones. I do this to drown out inconsiderate users who talk in the study space.

Technically I should need only three tabs open: The LMS (to get the scripts), the feedback tool (to read and comment on them), and the Google Sheet (to record the marks).

However, the auto-timeout of the LMS page requires me to refresh it, so I need the timer to remind me. If I do not pay attention, the LMS page times out in the background and my last set of comments does not get saved. The feedback tool also just freezes.

I need the Google Doc tab to back up my comments. I have been burnt by a systems administrator who removed my LMS account and its data (all the assignment comments), so I need the Google Doc for continuity between semesters.

Then there are the people who talk in the library… sigh. Ambassador-at-large, Tommy Koh, recently described us as a first world country with third world citizens. It is the small things that add up. Small things like being considerate in a shared space.

Needing five (or six) tabs open to grade and give feedback is a first-world problem. But the problems that cause them have been around for as long as Man has soiled the Earth.

This rant has been brought to you by Misanthrope Plus.

The issue of “grading on the curve” raised its ugly head in the news. This time the headline was a bold declaration:

But there was more to the headline. The article highlighted a variety of curve equivalent and curve adjacent schemes. Then there was a university don’s claim:

The claim that was not substantiated seemed to be that grading on a curve was part of assessment and that this was useful feedback. Specifically:

  • How is grading on a curve part of assessment when the other entities in the same article also claim they have done away with such moderating?
  • How exactly does sorting students on a curve provide feedback on meeting course objectives?

I do not know if he did not elaborate, or if the journalist or her editor left this out. Either way we have a claim without explanation or backing. None of us should take unsubstantiated claims seriously. Thankfully none of us will be graded on a curve to be critical thinkers.

Far wiser and more articulate people have shared their thoughts on assessment, grading and feedback. So I reshare what they shared.

From these and the work of others, I distill some wisdoms into these image quotes.

Formative feedback

Quantitative grading ends learning. Quality feedback sustains learning.

By the time you read this, I might be done with four intense sessions of performative evaluations of adult learners over the last two days.

Long story short: My learners need to provide evidence that they can facilitate student-centric lessons in a university context. This is as challenging to do as it is to evaluate.

To complicate evaluative matters, I have two batches of learners. I will need to grade the final written assignments of the second cohort as I evaluate the performative skills of the first.
 

 

This got me thinking about how much grading can be like the Rotten Tomatoes (RT) rating system for movies. If you are not familiar with RT, the video below provides a concise description and criticism of its flaws.
 

Video source

People might refer to a movie’s RT score as representation of its performance and then decide whether to watch it or not. While a single number is quick and convenient, it may not be valid or reliable.

The same could be said about essay grading and performative evaluations. As much as we use guidelines, standards, or rubrics, subjectivity is still an important factor.

I have reflected before on why and how I embrace subjectivity. This is particularly important when we need to find a balance between maintaining standards and treating each learner as an individual.

I am heavily influenced by Todd Rose’s work, The End of Average. We need to learn to recognise the contexts where using an average is meaningless. If we insist on grading strictly on a curve or using an unrealistic standard, we do more harm than good.

This harm affects our learners because we do not treat them as individuals and start where they are. The harm taints the teaching profession because we practice blindly. The harm persists when we act without question.

Does it take a rotten tomato hitting us in the face before we change?

Yesterday I shared some thoughts on providing formative feedback on written assignments. Today I share some strategies I use to do so efficiently and effectively.
 

Get out of comfort zones
I try not to work at home. There are too many distractions there largely thanks to my fast Internet connection. I might get sidetracked by YouTube videos or social media alerts.

I make my way to a library or a coffee place. I do not have the comforts of home and I switch to my all-business frame of mind.

One benefit of being at a coffee place is that I cannot stay long. I cannot hog a table or I might have time-limited Internet access. The time limit is an incentive to work efficiently.
 

Block distractions
Being in a public place means it gets noisy much of the time. I counter this by donning a pair of noise-cancelling headphones.

I have different playlists to help drown out ambient noise or bad store music. Some music is calming if the work I am reviewing agitates me; some of the tunes are energetic if my human battery runs low.

I also remove my glasses as I do not need them for close up work. Doing this has the benefit of not being distracted by what appears in my peripheral vision.
 

Set a quota and stick to it
I use the number of scripts and the feedback due date to calculate how many assignments I need to process a day. Then I divide each day’s quota into a before-lunch and an after-lunch set.

The quota needs to be achievable. Some initial trial and error has built my experience up on this matter. Calculating the quota is the easy part.

The difficult part is keeping to the quota. By this I do not mean completing the AM or PM session’s amount. I do this consistently. I mean stopping when I seem to have gained momentum.

It is important to stop because I do not want to mentally exhaust myself. If I am tired, it will show in my feedback and this is not fair to my learners. I also need time to rest, play, or finish the now cold cup of coffee on the table.
 

Revisit earlier scripts
It is tempting to grade, provide feedback, and never want to see a piece of writing again. However, I find it useful to revisit earlier scripts to
to check for the consistency of my feedback.

I think that we operate like the tides when we process assignments. We seem to wash up consistently like waves crashing on a beach. But no two waves are identical and there is the overall ebb or flow of tides too.

This could be due to my energy, the time of day, the overall nature of the class, the behaviour and expectations of learners who submit at different times, and so much more.
 

Embrace subjectivity
I do not try to be a robot or strive to be absolutely objective. This is impossible.

I embrace subjectivity but do so professionally. By this I mean that I take into account how a group responded to one or more lessons, what strategies I used with them, what content we uncovered together, etc.

We might have a menu to follow, but each dish is a bit different from another. I acknowledge and respect that by seeing how it was prepared (focusing on the process), not what it looks like (focusing on the product).
 

Strive for timeliness
One thing that makes feedback effective is timeliness. Wait too long to provide it and all effort it lost because learners have forgotten what they wrote.

The assignments I grade and provide feedback on are submitted online. After the submission deadline, I process assignments in the order they were submitted. This means that the first person to submit gets feedback first.

I could resort to an administratively convenient alphabetical order, but this is not fair to students who might have Z surnames but A response times.

End note: I prepared much of this reflection after processing a day’s quota of assignments. My mind was still buzzing, so I put my energy into sharing some ideas.

Grading Papers All Day (9/365) by Ms. Tina, on Flickr
Creative Commons Attribution-Noncommercial-No Derivative Works 2.0 Generic License  by  Ms. Tina 

Walking about in a mall, I saw a promotion to Pay Less, Taste More. Our Ministry of Education advocates Teach Less, Learn More.

After reading these three articles and reflections, I think we could and should Grade Less, Learn More.

The first piece by Bud the Teacher started the ball rolling . In his own words:

For starters, I think teachers, in general, grade too many things. So one way to streamline would be to “grade” less. And that doesn’t mean that teachers shouldn’t ask students to write, and write often. But we don’t need to grade everything that comes to us. In fact, we should grade very little of it.

In the context of writing, Bud went on to say that teachers have built up the wrong expectations that they are their students’ only audience and that they alone evaluate what that writing is worth.

He argued that technology should not be the means of streamlining grading when the problem lay with the assumptions, design, and expectations of grading.

Put another way, the questions he might have asked could include: Why are students writing for just one person? Why not leverage on authentic and public writing spheres? Why not approach the task as a fellow reader and learner in order to critique instead of just grade?

Cathy Davidson wrote the second article and she was as thought-provoking as always.

She started by describing seven reasons why standardized testing, while great for grading, are not that great for evaluating learning. So she suggested a key alternative: Get learners to create by learning how to code.

I do not think that all learners need to code, but I do think that all learners need to be given the freedom and opportunity to express themselves and to create. Only then are they putting theory into practice. Only then are they making mistakes. Only then are they really trying.

As a result of reading these and other articles, I had to ask myself if I had reflected on these issues before. I had in the third piece.

Two years ago, I had concluded that modernizing a problem (e.g., creating grading software) was not the solution. We might create more intelligent systems, but the problem of testing irrelevant skills in irrelevant ways would still remain.

No, the solution lies in questioning the established but problematic model. It starts with accepting that grading more sometimes leads to learning less.

Grading takes time and students do not get timely feedback. Students do not pick up as much content or as many skills as teachers wish. Teachers do all they can to keep up with the grading load and do not have the bandwidth to learn new strategies.

A solution might lie in figuring out how to grade less so that everyone learns more. By grading less I do not mean asking for lazier teachers. We need teachers who question, learn, and act on what they realize is better for their learners.

Teachers could grade less by redesigning tests and focusing on key things that need grading. They could leverage on their learners’ need to communicate, create, and critique. Teachers could model these life skills and teach their students to peer assess in authentic environments.

As they do this, I bet they would learn more about what their learners are capable of and what new value they as teachers bring to the table.

“News” broke recently of software that helped to “grade” student essays in some schools in Singapore. Here is one source that you don’t have to pay for to read about the use of WriteToLearn and Criterion.

I call it “news” because the software has been at least 10 years in the making and because the headline (Hey, the computer gave me an A for my essay) is sensationalist.

It might be news to some, but technology that is ten-years-old cannot be described, as it was in the article, in its infancy.

The headline is designed to give the impression that computers are taking over. Why not, right? After all, Watson, IBM’s latest AI has beaten human champions over the last three days in the gameshow, Jeopardy.

While some might be wondering when we are going to bow to our technology overlords, at the moment technology like WriteToLearn and Criterion are still just tools. We use them, not the other way around.

Those writing tools are great at doing what people cannot do: Using tireless brute force to analyze an essay with rules and provide recommendations more quickly than a teacher can for the learner. They are not designed to replace the teacher. Not yet anyway.

Two unoriginal thoughts came to my mind:

  • Any teacher that can be replaced by a computer deserves to be. Depending on your search, that quote might be attributed to David Thornburg or Arthur C Clarke.
  • The essence of being human involves asking questions, not answering them. I think that quote came from John Seely Brown (at the end of this NYT article).

We may have a finite capacity to hold knowledge. But we have an infinite capacity to wonder and create.


http://edublogawards.com/files/2012/11/finalistlifetime-1lds82x.png
http://edublogawards.com/2010awards/best-elearning-corporate-education-edublog-2010/

Click to see all the nominees!

QR code


Get a mobile QR code app to figure out what this means!

Archives

Usage policy

%d bloggers like this: