Online and so-called “kalye” surveys, particularly those broadcasted on social media platforms, may be biased and should prompt the public to be critical of the results, according to the University of the Philippines School of Statistics (UPSTAT).
UPSTAT faculty members called out individuals such as vloggers and organizations, including media companies, who conduct surveys with unclear methodologies.
The faculty expressed their dismay with the “cavalier disregard for the principles of data gathering” by those undertaking kalye surveys and “suspicious online surveys.” These include PR companies, private individuals, and media organizations.
“Data collection processes may also generate bias, such as when an interviewer purposely affects the answers of the respondents. Biased methodologies will only give biased results,” the faculty members said in a news statement. “Thus, we call on the public to be critical of surveys in this respect. Do not immediately accept survey results as they are.”
UPSTAT faculty members said surveys are conducted and reveal facts, beliefs, sentiments and opinions based on the representation of a population.
However, in order to generate results that are above board, the methodology should not be biased. Part of ensuring the absence of bias is in sample selection, which could only favor a segment of the population.
UPSTAT pointed out there is more to survey credibility than the margin of error. The public needs to consider how the sample population was chosen and which sectors of the population have been selected to participate in the survey.
Other indicators of proper survey implementation control are the events surrounding the period of data gathering; neutrality or professionalism in the tone of the interview done by enumerators; and other control mechanisms that ensure the accuracy of protocols in data collection.
“We, the faculty members of the University of the Philippines School of Statistics, speak and lend our collective voice to call out the pervasive abuse of survey methodologies by some entities doing their own brand of research, and promoting them in social media,” UPSTAT faculty members said.
“We are dismayed by their cavalier disregard for the principles of data gathering. We also witness PR companies, private individuals, and even some media organizations publishing results of surveys with unclear methodologies,” they added.
However, UPSTAT faculty members expressed their appreciation to reputable organizations that maintain and uphold the highest standards in conducting survey research.
They called on these organizations to help in promoting critical understanding of survey results in the Philippines. The faculty members also said they hope that surveys will be used for “illumination and sound decision-making.”
Last year, in his presentation at the 20th Jaime V. Ongpin Annual Memorial Lecture on Public Service in Business and Government, Philippine Institute for Development Studies Senior Research Fellow Jose Ramon G. Albert said online and kalye surveys are not always statistically reliable.
Albert added methodology, which involves stopping people on the street and getting their opinions, will not lead to a representative result, or a reflection of the entire exercise.
Relying even on Google Trends, Albert said, may also yield unreliable results. He said results on Google Trends would significantly be affected by the words used in these posts, which may skew results on who is leading Google Trends.
He noted that trolls might also inflate data on these big data sources “who are employed by all sides of the political spectrum.” The employment of the trolls, he said however, would depend on the size of the election “war chest” of a candidate.
Albert said it is important for firms and other institutions to be transparent with their methods, including the identity of the financier of the survey.
It also requires the public to be critical—not skeptical—of the surveys and for politicians and their supporters to be open to survey results.
UPSTAT faculty members called out individuals such as vloggers and organizations, including media companies, who conduct surveys with unclear methodologies.
The faculty expressed their dismay with the “cavalier disregard for the principles of data gathering” by those undertaking kalye surveys and “suspicious online surveys.” These include PR companies, private individuals, and media organizations.
“Data collection processes may also generate bias, such as when an interviewer purposely affects the answers of the respondents. Biased methodologies will only give biased results,” the faculty members said in a news statement. “Thus, we call on the public to be critical of surveys in this respect. Do not immediately accept survey results as they are.”
UPSTAT faculty members said surveys are conducted and reveal facts, beliefs, sentiments and opinions based on the representation of a population.
However, in order to generate results that are above board, the methodology should not be biased. Part of ensuring the absence of bias is in sample selection, which could only favor a segment of the population.
UPSTAT pointed out there is more to survey credibility than the margin of error. The public needs to consider how the sample population was chosen and which sectors of the population have been selected to participate in the survey.
Other indicators of proper survey implementation control are the events surrounding the period of data gathering; neutrality or professionalism in the tone of the interview done by enumerators; and other control mechanisms that ensure the accuracy of protocols in data collection.
“We, the faculty members of the University of the Philippines School of Statistics, speak and lend our collective voice to call out the pervasive abuse of survey methodologies by some entities doing their own brand of research, and promoting them in social media,” UPSTAT faculty members said.
“We are dismayed by their cavalier disregard for the principles of data gathering. We also witness PR companies, private individuals, and even some media organizations publishing results of surveys with unclear methodologies,” they added.
However, UPSTAT faculty members expressed their appreciation to reputable organizations that maintain and uphold the highest standards in conducting survey research.
They called on these organizations to help in promoting critical understanding of survey results in the Philippines. The faculty members also said they hope that surveys will be used for “illumination and sound decision-making.”
Last year, in his presentation at the 20th Jaime V. Ongpin Annual Memorial Lecture on Public Service in Business and Government, Philippine Institute for Development Studies Senior Research Fellow Jose Ramon G. Albert said online and kalye surveys are not always statistically reliable.
Albert added methodology, which involves stopping people on the street and getting their opinions, will not lead to a representative result, or a reflection of the entire exercise.
Relying even on Google Trends, Albert said, may also yield unreliable results. He said results on Google Trends would significantly be affected by the words used in these posts, which may skew results on who is leading Google Trends.
He noted that trolls might also inflate data on these big data sources “who are employed by all sides of the political spectrum.” The employment of the trolls, he said however, would depend on the size of the election “war chest” of a candidate.
Albert said it is important for firms and other institutions to be transparent with their methods, including the identity of the financier of the survey.
It also requires the public to be critical—not skeptical—of the surveys and for politicians and their supporters to be open to survey results.
