Another dot in the blogosphere?

Posts Tagged ‘survey

This is my response to newspaper articles [Today] [STonline] on a study by Singapore’s Institute of Policy Studies (IPS). I also respond in longer form to tweets about the articles or study.

First some background, disclosures, and caveats.

According to one article, the study was “a quantitative look at the views of 1,500 citizen or permanent resident (PR) parents with children in local primary schools on their perceptions about Singapore’s education system at that level”.

I am not linked to the IPS nor do I have a stake in what it does. As an educator, I have a stake in how people process reports of such studies because it reflects our collective capacity to think critically.

My intent is to provide some insights based on my experiences as a teacher educator and researcher. In the latter capacity, I have had to design and conduct research, supervise it, and be consulted on designs, strategies, and methodologies.

However, without full and immediate access to the actual IPS report and data, I have to take the newspaper articles at face value. I also have to assume that the research group implemented the survey-based study rigorously and ethically.

The headline by the Today paper was click(bait)-worthy. It was not the only finding, but the paper thought it would grab eyeballs.

At least two people tweeted and wanted to know if other stakeholders like parents and the students themselves were asked about the impact of the PSLE.


I understand their concerns, but this was probably not on the research agenda. I say this not to dismiss the importance of their questions.

Good research is focused in order to be practical, to manage limited time and resources, and to shed a spotlight on a fuzzy issue. The questions about teachers and students could be addressed in another study.

It might help to view the study as a snapshot of early stage policy implementation. MOE has passed policy of “every school, a good school” and shared upcoming changes to the PSLE. The big question is: What is the buy in?

MOE can more easily manage the buy in among teachers and students. Parents are a different matter, so the study rightfully focused on that group.

The study was not about making any comparison. It was about taking a snapshot of public opinion.

This is also not a question that the IPS could seek answers to in mainstream schools here. Except for international, private, and most special needs schools, all mainstream Primary schools subscribe to the PSLE and do not have alternatives like e-portfolios. Some home-schooled children even take the PSLE.

This is actually a critical question that needs to be asked.

Our Prime Minister hinted at it in his National Day Rally speech in 2013 and MOE responded with some changes — IMO superficial changes — in late 2016.

If enough stakeholders question the timing or value of PSLE, then the followup questions revolve around the WHEN and HOW of change.

According to the ST article, “the sample of parents… had a proportionate number of children in almost all the 180 or so primary schools here.”

Now this could mean that there was less than ten parents representing each school on average. We cannot be sure if some schools were over or under-represented, nor can we be absolutely certain that the respondents were representative of parents in general. This is why national surveys rely on large returns.

That said, surveys, whether voluntary or solicited, tend to be taken by those who want to have their say. You can never be absolutely certain if you have are missing a silent majority or have a data from a vocal minority. However, a large return tends to balance things out.

The survey study seemed to rely on descriptive statistics. At least, that is what the papers focused on. If that is the case, a statistical analysis was not in the design. If it was, there would be specific research questions based on hypotheses.

Not every study needs a statistical analysis. If this was a snapshot or preliminary study, the descriptive statistics paint a picture that highlight more questions or help policymakers suggest future strategies.

Overall, I do not fault a study for attempting to paint a broad picture that no one else seemed to have a clear view of. It sets the stage for more query and critical analysis.

But I do have one more potshot to take and it is directed at the newspapers.

The contrast of what was highlighted by each paper of the same study could not be more stark.

To be fair, both papers had a few articles on the same study to highlight different topics. But what the newspapers choose to tweet is an indication of what they value. This is no different from what any of us chooses to tweet.

I chose to call out the subjectivity of any press that thinks of itself as objective or impartial. Any study and press article has bias, some have more and some less.

As content creators, we should make our bias transparently obvious. As critical thinkers and doers, we should try to figure out what the biases are first.

Last week I received email from GeBIZ to complete a survey (PDF file) and then either email the file or fax it.

Gebiz email requesting for survey returns.

The message and instructions begged these questions:

Perhaps someone conspired to rile GeBIZ users up so much that they would provide feedback to demand for more efficient and effective practices.

An online version of the form is both more efficient and effective.

  • Its submission is immediate as is a confirmation of receipt.
  • There is no need for people to compile data from two different sources into one.
  • The data can be automatically collated and analysed without first being inputted manually from the emailed PDFs or faxes, thereby reducing human error.

If this is what happens to a survey, I dare not imagine how other processes might be compromised.

As an educator, I cannot help but wonder what messages actions like these send to the larger system. Are these indicators of push-backs on progress?

I do not think that my concern is unwarranted. While mainstream school teachers are not quite affected Internet restrictions, there are already restrictions on services like Dropbox and mobile services.

If plans are only as good as their implementation, why does “smart talk, dumb walk” persist?

Policies crafted by leaders shape the work environment and culture. If higher-ups associate the Internet, social media, or anything “e” as dangerous or wasting time, they will enact policies that reinforce such hang-ups and nurture a culture based on fear.

Consider this scenario. Imagine I propose that school personnel decide on whether they spend money only on a textbook collection or Chromebooks. The books do not raise an eyebrow, but the response to Chromebooks is “Yes, but…”.

As different as schools are now compared to a generation ago, values and practices today are arguably still entrenched in the past. Ask teachers if they integrate technology and it is still common to hear phrases like “technology to enhance”, “the basics are more important”, “we don’t want the kids to be distracted”, or “the exams are handwritten”.

Technology should not just enhance, it should enable learning. The basics have changed and are more complex and kids need to be empowered. Very little outside of conventional exams and schools is handwritten. Even GeBIZ asked for email replies.

Despite the smart talk and inspiring rhetoric, what actually makes a difference is the walk. It easy to say you want innovation in schools. It is more difficult to create conditions for change.

Last Friday I took my son out for a treat at a fast food joint. We opted to try a special menu option that the restaurant offered. As we were among the few trying that option, a “survey uncle” asked me to participate in a poll.

The survey uncle was apologetic for interrupting our meal, but thankful that he had found me. He explained that he had trouble finding my “demographic” (read: old and with purchasing power) so I humoured him.
 

Survey on Obama’s Cairo Speech by Swamibu, on Flickr
Creative Commons Creative Commons Attribution-Noncommercial 2.0 Generic License   by  Swamibu 

 
The survey was in only English and on an iPad. Participants also had to complete the survey on their own so that the poll was uninfluenced by the survey takers.

I remarked to the survey uncle that he was at the restaurant on the wrong day and time. It was Friday afternoon and kids in school uniforms probably outnumbered the adults 50 to 1. He informed me that it was the polling company’s decision to choose the survey period.

Here is lesson number 1: Listen to your stakeholders and learn about their habits. The company might think it knows better, but it does not.

The survey uncle also recognized that quite a few aunties and uncles in my age group and older 1) were not comfortable taking a survey on an iPad, and 2) did not understand the language of the poll. He had suggested to his bosses that an alternative survey be provided in Mandarin.

Lesson number 2: Listen to your troops on the ground because they are more aware of the issues. For example, the survey uncle realized that the kids around us only bought the cheaper meal items while those that went for the special menu were few and far between. He struggled to meet the quota so that the findings were at least statistically useful.

Lesson number 3: Reach out to your stakeholders in a manner they would be responsive to. As one size does not fit all, you are likely to need different approaches, e.g., in the survey context, this could mean using iPads, paper, and interviews.

The lessons apply to schooling and education. Policymakers and administrators might think they know better or see more from their vantage point. But as long as they are not on the ground, they cannot relate to issues that prevent new policies and change from taking root.

To be more effective, they need to listen to their stakeholders and teachers first. When they do reach out, it should be with a sympathetic and open ear first, not with a closed or iron fist.

survey_on_flipped_video_pedagogy_workshop

Here are some results of the survey I posted online. They have helped me shape the proposal to YouTube/Google.

Almost three quarters of the surveyees opted for the workshop to be held on Fridays, 4-7pm. I have incorporated the time into the proposal.

The same proportion preferred a buffet approach to a fixed menu of pedagogical-technical content. The buffet approach is going to be challenging and I will have to manage expectations of all stakeholders.

The top two pedagogy-related topics were meaningful content delivery and the pedagogy of questions. I expected the former because teachers rarely venture beyond what they already know. But I was very pleased to see that some were willing to break out of the norm.

The top three video skill-related topics were screencasting, and a tie between simple animation and vlogging. All were expected since screencasting and animation are what most teachers have seen other teachers do on video. I wonder how many will be comfortable in vlogging/demonstrating mode.

About two-thirds said they would attend a Singapore-based conference with a focus on flipping in May 2015. Even more (just over 80%) said they would attend an unconference before the conference. You cannot imagine how happy that makes me because this is an indication of what teachers value at conferences: Not sitting and listening, but connecting and talking.

survey_on_flipped_video_pedagogy_workshop

I am planning on offering free workshops for Singapore educators on Fridays in February 2015. If all goes to according to plan, the workshops will be sponsored by and hosted at Google, Singapore.

Depending on demand and funding, there might be anywhere between one to three sessions. I am planning for 15-20 educators to attend each session. The sessions are likely to be similar, but might be tweaked for improvements. Each session will last about three hours.

The workshops will focus on the designing of flipped learning with videos. The two main learning gains are pedagogical and technical in nature.

That is all I can share for now. I need to get information from potential participants. Please complete this 12-question survey by 11.59pm, Sunday, 21 Dec 2014.

The first seven questions are about the workshop, the next two questions are about a conference and an unconference in Singapore (both with foci on flipping), and the last three collect demographic information.

In question 3, I ask if participants would like a “fixed menu” or a “buffet” of learning options. Both will have flipped components, but the fixed menu will have fewer options which will be determined by other questions in the survey. The buffet will provide a larger variety of resources, but this will require much more independent work.

I reached out to #edsg after learning that a contact of mine was thinking of organizing a flipcon (conference on flipped classrooms/learning).

I prepared a quick survey because I believe in making data-informed decisions and not just gut feel or good intent. Thankfully 43 generous folks participated.

Here are some answers summarized in visual form. Click on the image below for a larger version.

flipcon_survey

Almost three-quarters of the respondents said they would attend a flipcon, but most were not willing to pay much for it.

My guess is that most teachers do not realize how much it costs to fly in and host speakers, rent a venue, and cater for the event. Most teachers here also do not have to pay for their professional development if their school principal gives approval to attend the event and make claims. This might sound harsh, but this is a reality that many teachers are not aware of and a privilege that they have that many teachers elsewhere do not.

I was pleasantly surprised that almost two-thirds of the respondents said they would rather attend a hybrid conference. This would be an excellent way to model flipping and to design for better interaction during the conference.

Sadly, two thirds would rather attend to learn instead of sharing. This might be because they have not flipped, are not confident about sharing, or fear the impact of sharing.

If fear is what is stopping them from sharing, we have a bigger problem than to flip or not to flip. Part of the value system of flipping is the sharing of ideas and resources openly to keep this ground-up movement going forward.

I would also like to reply to some comments and questions raised in response to the open ended question.

While a lot of us are certainly keen to share, to present at an event typically makes us put in a lot of time and effort. So perhaps to reduce the kind of stress, it can be an unconference where the theme is to share about struggles, so it needn’t be too formal and stressful. Just a thought… =D

I agree. I even mooted the idea that if the flipcon does not happen that we band together and organize our own unconference!

I would like to know about the failures. Often conferences only focus on the spectacle of the successes without highlighting the initial failures.

Again, I agree. That is why I phrased one of the questions: Would you be keen on sharing your flipping journey/struggles at the event?

How much time the pre-conference would take. [sic]

I am not sure what a pre-conference is.

Perhaps the respondent was thinking about the watching of presenter’s videos in a hybrid conference and then attending the face-to-face component. If this is the case, then my answer would be: As long as you wish because you watch what you want and interact with whom you wish.

After all, self-directed and independent learning is one of the desired outcomes of flipping.

Follow-up classroom activities after flipped content.

I must correct a misconception here. Far too many teachers think that flipping is only about what happens outside the classroom or focus their energies on that.

Flipping is about improving the quality of what happens inside the classroom. What happens in the classroom are not follow-ups; they are core to the process.

Question 4 should include an additional option “Perhaps in near future”

Question 4 was “Would you be keen on sharing your flipping journey/struggles at the event?” In order to gauge how many people would speak and share, this was a Yes/No question.

Perhaps in the near future is 1) non-committal, 2) does not actually help answer the question (would you share at the event), and 3) gives no clear indication of when.

CAN THE COST BE LOWERED? IM NOT WILLING TO PAY AT ALL 😦 is it applicable to all subjects?
Sigh. I do not want to get snarky so I am not going to say anything other than pointing out what I wrote about paying for the event.

I will reply to the question about academic subjects. Flipping is content neutral and can be applied even to meetings and conferences.

Oh, and watch the all CAPS. That means you are SHOUTING.

Have a nice day.

It would be awesome. Flipped learning is something that’s essential in education today and teachers need a platform to learn from one another.

Thank you. This is the kind of attitude and energy we need.

I asked all five of my classes (130 trainee teachers) to participate in a short survey about Google Docs. To date, 114 of them have responded.

I used Google Forms to create the survey. Part of the automatically generated summary is shown below (click image to see it in original size).

2009-10-29-google_apps_surveyHalf of them have never used any part of the Google suite while half have used Google Docs. The half that have some experience can help the half that haven’t!

As expected, nine out of ten have not heard that schools will adopt Google Apps by the end of the year. The announcement was made only in late September.

I am hoping that my attempt at integrating Google Apps will not only ease the transition but also enable them to learn the concepts of EdPsych2 more actively, collaboratively and meaningfully.


http://edublogawards.com/files/2012/11/finalistlifetime-1lds82x.png
http://edublogawards.com/2010awards/best-elearning-corporate-education-edublog-2010/

Click to see all the nominees!

QR code


Get a mobile QR code app to figure out what this means!

My tweets

Archives

Usage policy

%d bloggers like this: