Another dot in the blogosphere?

Posts Tagged ‘student

The simple message in the tweet below hides a profound principle of evaluation.

Why is the message a timely one? In many university campuses all over the world, an academic semester is nearing an end or already ended. It is time for the end-of-course evaluations.

Instructors who do not have teaching backgrounds, those who resent teaching, or those who cannot teach well are dreading these evaluations. If only they would collectively point out that such exercises are based on a flawed approach .

Many end-of-course evaluations (otherwise known as student feedback on teaching, SFTs) read like customer satisfaction surveys because they are often designed by administrators, not assessment and evaluation experts.

Even a well-prepared instructional designer should be able to point out that SFTs often only operates at Level 1 of the Kirkpatrick Evaluation Model. It is a simple measure of each student’s snapshot reaction to weeks or months of coursework.

SFTs should be about the effectiveness of teaching and the quality of learning. But if you unpack the questions in most evaluation forms, you will rate such “evaluations” as satisfaction surveys instead. A researcher with rudimentary knowledge of data collection will tell you that such information is not valid — it does not measure what it is supposed to measure.

I have reflected before on how I do not place much stock in SFTs if they are not well designed and implemented. I ignore the results even though I do well in them. How can I when I know that they are not valid measures? Why should I be satisfied with unsatisfactory practices?

Video source

The details in the article, Application installed on students’ devices does not track personal information, reminded me about some unanswered questions on student data management and learner self-regulation.

First, some background on the device management application (DMA) that will be installed in all student-owned devices. It reportedly does not keep track of “location, identification numbers or passwords”. It should not.

But the DMA will “capture data on students’ online activities such as web search history… and device information such as the operating system”. If forensics can use those to identify a person, is that not “personal information”?

Consider how your typing rhythm can already be used to identify you or how the different sounds of a keyboard can be used to figure out what you are typing. Raw data generated by a person can be identifiable and personal data.

According to the tweeted news article, a petition against the DMA from around 6,000 individuals did not dissuade the powers-that-be. The authorities argued that data collection and remote monitoring is necessary to protect children from undesirable sites and behaviours. Cue scary sounds and imagery of pornography, gambling, predators, and screen time.

For argument‘s sake, let’s assume that the data is absolutely secure from hackers. It is, however, available to “appointed DMA vendors”. What might the vendors do with such data? They could use it to develop more applications that profit them (example: plagiarism detectors use student-papers for free but charge a fee for its service).

If the vendors slide on integrity or if data is hacked, the online preferences and habits of our students becomes a trove of ad targeting, market development, data bundling and reselling, etc. We need only examine our own experiences with entities like Facebook — we are not the customer, we are the product — to see how this might happen.

Declaring that student data will be securely stored, stringently controlled, and lawfully protected does not guarantee that the policies on all three will not loosen over time. Consider a recent lesson on how TraceTogether data was supposed to only be for COVID-19 contact tracing, but now can also be used to investigate seven forms of serious crimes.

The declaration also does not indicate an expiration and/or expunging of user data. Bluetooth data from TraceTogether is deleted every 25 days.

Another question to ask about data use is: What will MOE/vendors do if the monitoring results in red flags? The alerts could be due to truly nefarious activities (example: the youth who recently self-radicalised and wanted to attack Muslims) or legitimate research on terrorism. What systems are in place in terms of algorithms and human monitors? What constitutes are reasonable response?

Perhaps my questions have already been answered but have not been made public. Perhaps my questions might provoke some reflection.

But I certainly want to provoke some thought and action in the area of student self-management. Might using tools like the DMA create a reliance on them? Such tools trigger extrinsic motivations, e.g., fear of detection for visiting unauthorised sites or waiting for pings from the system to meet deadlines.

We need such tools to be scaffolds. Scaffolds are removed from buildings as they are constructed because they stand on their own. What else will be put in place to ensure that our students learn to stand independently and think responsibly on their own?

I know of a few schools that rely on educational and social scaffolds instead of DMA-like tools. Students use their phones and computers like we might at home. These devices are unencumbered as we wish or as locked down as we make them. We decide.

The message that tools like the DMA as all-powerful and monitoring might provide some comfort to the public. This is disingenuous because more nuanced questions have not been addressed about their use. Equally, not enough emphasis has been placed on actually nurturing independent and responsible learners.

When I first saw this tweet…

… I wondered how soon this next tweet would emerge. It did not take long.

Why? The authorities here — all authorities here — do not enjoy unsanctioned protests. Given the pattern of authoritative responses, I expected no different.

School principals and teachers likely asked themselves: Was this my student? The protesters looked older, so they are probably working, in post-secondary institutes, or waiting to enter one. But schools are still concerned about their reputations by association.

The protesters were responding to this news: MOE denies stopping transgender student from getting hormone therapy. The protesters’ first response was about highlighting the treatment of LGBTQ students.

So what is our next collective response?

 
I have had the privilege and misfortune of experiencing how student feedback on teaching (SFT) is done in different universities.

When I was a full-time professor, the institute I worked at specialised in teacher education and had experts in survey metrics. So no surprises — the SFTs were better designed and constantly improved upon.

One of the best improvements was the recognition that different instructors had different approaches. Each instructor had a set of fixed questions, but could also choose and suggest another set of questions.

As an adjunct instructor now and roving workshop facilitator, I have been subject to feedback processes that would not have passed the face validity test at my previous workplace.

One practice is administration using only positive feedback to market their courses. Feedback, if validly measured, should be used to improve the next semester’s offering, not be a shiny star in a pamphlet.

Another bad practice is sampling a fraction of a class. If there is a sampling strategy, it must be clear and representative. Feedback is not valid if only some participants provide it.

Yet another SFT foible is not sharing the feedback with the facilitator or instructor. One institute that operated this way had multiple sections of a course taught by different instructors. However, the feedback did not collect the name of their primary instructor because classes were shared.

All the examples I described attempted to conduct SFT. None do it perfectly. But some are better informed than others. Might they not share their practices with one another? If they do, will institutional pride or the status quo stand in the way?

This tweet provoked me to think and here is my response: You can own processes like learning.

I am an autodidact. Every day I consume content from my RSS feeds, Twitter stream, podcast episodes, and YouTube subscription. Then I reflect on them so that something rises to the surface.

That is my learning process. I take it, make it, guard it, and practice it daily. I take ownership of that process while respecting that others may have similar or different ways of learning.

I also take ownership of my workshop facilitation processes, so much so that I took one opportunity years ago to create a quick video of one example.


Video source

For me, ownership is about taking responsibility, not about taking possession. I take responsibility for supporting my family; there is nothing tangible to possess about that.

Likewise I take responsibility for educating myself (I own the processes I described above), but I do not possess that learning. The overall process is not transactional; it is transformative.

Ownership is evidenced by shows of agency and empowerment, be it for students or teachers. The article linked in the tweet below provides some how-tos of promoting agency and empowerment.

Ownership of learning by learners is not a frivolous fad. It is a foundation upon which to nurture resilient and independent learners.

Every now and then thank serendipity. Today is one such day.

Will Richardson offered this tweeted thought on student agency.

But it is one thing to know what “student agency” is. It is another to implement it practically.

So how might teachers nurture student agency? This MindShift article offers eight values to embody. I paraphrase them:

  1. Facilitate authentic decision-making
  2. Help learners to know their strengths and challenges while learning
  3. Leverage on natural curiosity and passions
  4. Show learners how questions drive learning
  5. Make room for authentic forms of assessment
  6. Model different learning strategies
  7. Showcase and celebrate learning
  8. Focus students on learning-to-be, not just learning-about

I would only add this thought to those of the two contributors: It is important to start this with learners when they are young. They are the most natural learners and have not had the energy and curiosity schooled out of them yet. If they start young, they grow to expect such an approach as they get older and they operate more independently as learners.

Agency is giving learners the opportunities to make decisions. Empowerment is enabling them to take meaningful and self-driven action.

I have avoided reading and reviewing this opinion piece Analytics can help universities better support students’ learning. When I scanned the content earlier this month, my edtech Spidey sense got triggered. Why?
 

 
Take the oft cited reason for leveraging on the data: They “provide information for faculty members to formulate intervention strategies to support individual students in their learning.”

Nowhere in the op piece was there mention of students giving permission for their data to be used that way. Students are paying for an education and a diploma; they are not paying to be data-mined.

I am not against enhancing better study or enabling individualisation of learning. I am against the unethical or unsanctioned use of student data.

Consider the unfair use of student-generated data. Modern universities rely on learning management systems (LMS) for blended and online learning. These LMS are likely to integrate plagiarism checking add-ons like Turnitin. When students submit their work, Turnitin gets an ever-increasing and improving database. It also charges its partner universities hefty subscription fees for the service.

Now take a step back: Students pay university fees while helping a university partner and the university partner makes money off student-generated data. What do students get back in return?

Students do not necessarily learn how to be more responsible academic writers. They might actually learn to game the system. Is that worth their data?

Back to the article. It highlighted two risks:

First, an overly aggressive use of such techniques can be overbearing for students. Second, there is a danger of adverse predictions/expectations leading to self-fulfilling prophecies.

These are real risks, but they sidestep the more fundamental issues of data permissions and fair use. What is done to protect students when they are not even aware of how and when their data is used?

This is not about having a more stringent version of our PDPA* — perhaps an act that disallows any agency from sharing our data with third parties without our express consent.

It is about not telling students that their data is used for behavioural pattern recognition and to benefit a third party. While not on the scale of what Cambridge Analytica did to manipulate political elections, the principle is the same — unsanctioned and potentially unethical use of a population’s data.

*I wonder why polytechnics are included in the list of agencies (last updated 18 March 2013) responsible for personal data protection but universities are not.

I try not to roll my eyes whenever I read this repeated claim:

Another hot topic raised at the forum was whether MOE would consider reducing class sizes from 40 to 25 students to allow teachers to give more attention to their charges.

However, Mr Ong pointed out that the 1:40 teacher-student ratio in a classroom is actually 1:15 for primary schools in terms of the overall numbers of teachers and students in a school, with the ratio reduced for secondary school (1:12 or 1:13) and junior colleges (1:11).

This latest iteration was reported in this news article.

One public response to this claim was Class size in schools: For teachers, the real work is not just a ratio.

Take a straw poll among mainstream school teachers with classes and very few will report they have classes of those sizes. (I am excluding special needs teachers, counsellors, teaching adjuncts, etc.)

Do the ratios account for every education officer, even the ones on extended leave or pursuing professional development? Do they include every possible teaching intervention, i.e., from individual coaching to mass lectures? If so, it is possible to get such ratios on a spreadsheet.

Personally, I have seen class size rise instead of fall. I teach at university level and this semester my classes swelled to between 32 and 36 per session when they were hovering around 20 previously.

University policies are not directly under the direct purview of the MOE. Each school or department within the university decides on class sizes using the resources it has. I know of one group that has tutorial classes that average about 50 students while others run studios with students you can count on one hand.

But that is my point — the adaptive variability is not obvious in a spreadsheet. When you take averages of such variability, you get simplified ratios that remove all nuance.

This opinion piece, Not a good idea to start school later, is not about the good of the students. Instead, it is about their parents, the employers of the parents, the transport companies.

Now these other stakeholders also have a say. The problem is that their say is dominant and overwhelms what is important. That is why there is no change. The question of why we do not start school later is perennial and so are the standard answers.

The problem is not just that we keep revisiting this issue and not change anything. It is that we normalise the cycle, and in doing so, lose sight of what is important (the learner) and instead dwell on what is urgent (everything else).

What is important is seldom urgent. And what is urgent is seldom important. -- Dwight D. Eisenhower.

I had an uncomfortable gut feeling when I read this CNA article about biometric payments being available to schools here in 2018.

I had to dig deep for why I was uncomfortable. After all, I am all for technology making lives better. And therein lay the problem: In doing good, there was also the potential for harm.

The good is the sheer convenience of going cashless while being able to track spending. This might be the start of basic financial literacy.

According to the news article, the system has safety measures:

Fingerprint information will not be stored on the device. Instead, the prints will be encrypted and stored securely in a cloud database.

Anti-spoofing technology will also be put in place to ensure that the fingerprints are real and that the person making the payment is present.

This is the trifecta of data accuracy (reading), data security (keeping), and data integrity (reliably identifying). If just one to fails, the system’s users are harmed. Take the recent Instagram hack, for example.

For the sake of argument, let us assume that the three data concepts are sound in practice. What is the harm then?

To answer this question, we need to ask at least one other question: What else can vendors do with the data that is accurate, “stored securely”, and reliable?

The short answer is lots. One needs only look at what Facebook and Google did (and continue to do) with our data. They offer their services for “free” to us because our data serves up advertisements which make these companies money. Lots of it.

One needs only to casually search for data breaches and infringements involving these two companies. For example:

The last item was not so much about the privacy of data as about the use and manipulation of data. That is my point: Assuring stakeholders that data is accurate, authentic, and safe is not enough; it is the lack of transparency and foresight about what can be done with that data.

Students are particularly vulnerable because adults make decisions about their data and the kids have no say in the biometric scheme. By this I am referring to the scheme being employed as a Smart Nation initiative, not the choice of whether to join the scheme.

The issue is so serious that the Electronic Frontier Foundation (EFF) has tips for teachers about student privacy. These include:

  • Making digital literacy part of the curriculum
  • Advocating for better training for teachers
  • Getting parental consent
  • Selecting technology tools carefully
  • Building community of like-minded privacy advocates

A Smart Nation needs people to make smart choices. To do that, people need good information. Where is the information about how the data might be used both intentionally and peripherally? What promises and standards of practice can service vendors and providers be held to? Where is the public debate on the data privacy of the especially vulnerable?


Archives

Usage policy

%d bloggers like this: