Another dot in the blogosphere?

Alternatives to SFTs?

Posted on: December 6, 2021

I have been consistent about my stance against end-of-course student feedback on teaching (SFTs). Today my reflection was prompted by this tweet.

I am confident that, like me, this professor and others like him, do not get bad reviews. We are against a data collection method that is flawed. 

I caution administrators against using SFTs as the only measure of faculty teaching because SFTs are:

  • not valid in if they do not measure if effective learning took place
  • used for purposes other than to improve instruction
  • summative in that they do not allow teaching faculty to make changes that semester
  • reliant on student self-reports as a single data source

The tweet highlighted how invalid SFTs can be. No matter the questions asked, students might bias their answers because of non-teaching or superficial traits of their instructor/facilitator. The questions in an SFT are also likely to focus on teaching-related aspects of a course (e.g., the LMS) instead of how much or how well they learnt.

SFTs designed to measure traditional and face-to-face teaching methods also might not align to online methods or facilitative approaches. For example, SFTs rarely (if ever) focus on the design of effective asynchronous learning resources or personalised online coaching.

Administrators use SFTs to rank faculty during promotion and retention exercises. This is clear to any full-time university faculty with a significant teaching load. I know of ex-colleagues who would game the system by currying favour with their students so that they would get good SFTs. 

These folk needed the most help improving their instruction, but since they got good enough SFTs, they did not reflect and improve on their practice. They just got better at gaming the system.

If SFTs are primarily for improving the quality of courses and instruction, they cannot be implemented at the end of a course. Good teachers collect feedback constantly so they can make adjustments on the run.

Insisting that data from the end of one course should inform the design and implementation for the next one misses the point — teaching is dynamic and complex. You can take the same instructor, design, and content, but different batches of students will react differently.

SFTs also rely on self-reports by students. These are equivalent to the Kirkpatrick Level 1 “smiley sheets” that seek opinion rather than fact. If students like you, they will rate you higher than you serve. The opposite is also true.

So what else can we do in addition to or as alternatives to SFTs? In my reflection earlier this year, I suggested “multiple methods, e.g., observations, artefact analysis, informal polling of students, critical reflection”. 

Today I would add that faculty portfolios capture these methods. Remarks from casual observations by fellow faculty, marked up video recordings, key takeaways from brief but regular student polls, and faculty reflections can be collated on online platforms like a blog or Google Site.

Portfolios have another plus: They put the ownership of the design, implementation, and evaluation of courses in the hands of teaching faculty. If these instructors carefully maintain their portfolios outside university, they can take them wherever they go. 

That said, portfolios do not resolve the biggest problem with SFTs. They might still be about teaching. What matters is whether students learnt, what they learnt, how much and how well they learnt it, etc. 

That problem is not an easy one to solve. Students might view courses merely as stepping stones to paper qualifications. There is the long tail of learning, i.e., their ah-ha moments might occur outside the course and these are not captured. Their in-course learning might not be intentional but still desirable, e.g., they learnt how to manage their time, but these too are not measured.

The biggest problem is that both administrators and faculty might be content with measuring the low-hanging fruit. After all, it is easy to hide behind the rock called It Has Always Been Done This Way.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

This site uses Akismet to reduce spam. Learn how your comment data is processed.

Archives

Usage policy

%d bloggers like this: