Temple University is taking advantage of our ever-growing digital society in order to get a better idea of what Philadelphians are like in the physical world. With the proliferation of mobile devices and the Internet, many experts are beginning to realize that their uses when it comes to statistically analyzing the city could be beneficial in advancing survey collection methods.
This new movement is called BeHeardPhilly and it is running on the platform that it is “a social movement in Philadelphia that gives everyone a chance to have their voices heard as many organizations, like the city government and local non-profits, have questions and want to know their opinions and thoughts on important social issues,” according to its official website. Its ultimate goal is to use these advancements to further understand how we can analyze what Philadelphians are thinking and feeling.
According to an article by Philly.com, the Institute of Survey Research (ISR) here at Temple makes use of “bus stop ads, social media and neighborhood outreach to recruit a panel of several thousand Philadelphians who will be indefinitely used for survey requests.”
The article goes on to state there is a five dollar incentive to complete surveys and this method “Will get its first application in the coming days, as the Institute for Survey Research gathers information about Philadelphian’s traffic related knowledge and behaviors for the city’s street department.”
Furthermore, this new method is utilizing a new “opt-in” approach, which could eliminate some of the larger problems statistical surveying faces: low participation. Through the mobility of cell phones and the Internet, participants who opt into this program will be able to provide answers to survey questions quickly and efficiently.
Temple University’s campus is the first to do such a thing. An article by Philly.com quotes staff from the ISR stating, “BeHeardPhilly is the first city wide panel of its type in the country.”
Calling a house or cell phone, as tradition allows, leads to dismal participation rates. This, paired with an increase in security and privacy concerns, leave little to work with when it comes to properly surveying the population.
“People who don’t have land lines and don’t answer cell phone calls kill one of the most dependable ways to build a probability sample [and] fears of hacking and information theft has left people wary about answering questions, said Robert Santos, vice president of the American Statistical Association and chief methodologist at the Urban Institute in Washington D.C.” according to the article from Philly.com.
Not only are those factors contributing to low participation rates, but Heidi Grunwald, director of the ISR here at Temple, also states these methods can be extremely expensive, and can cost “up to $20,000.” Unfortunately, according to Roger Tourangeau, who was quoted in the article by Philly.com discussing the repercussions of the mere expense of these samples, “the other option isn’t to do a probability sample [in these cases,] the other option is to do nothing.”
Essentially this means that if there is no money to perform the survey, then the necessary population information we need is simply not gathered. The opt-in method and proliferation of the Internet has allowed for a way around that barrier. By using this opt-in method, “Response rates rise to numbers around 30 and 40 percent,” according to the article by Philly.com.
This could solve some problems, but could also raise others. Legitimacy of information is one of the issues this new method could be facing. A paper done by the Oxford Internet Institute about the validity of surveys states, “The most obvious threat to external validity for web-surveys was, that large sections of the population did not have access to the Internet.”
This presents a valid point. If the surveys are only available to those who can access the Internet, this leaves a portion of the population unable to provide their input. However, statistics by the Pew Research Center show that upwards of 84 percent of American adults use the Internet. With these numbers, it becomes clear that this does not present the same problems it did when the Internet was first getting started. In 2000, for example, the statistics show that only about 52 percent of American adults were using the Internet.
Other issues this new method presents include its ability to include “older, less educated and poorer” individuals, according to the article by Philly.com. This could lead to some form of statistical inaccuracy, which is not always needed, but “raises the possibility of getting results that can’t be tests for accuracy,” Philly.com quotes Santos stating in its article.
Moreover, these participants are indefinitely involved in the surveys once they sign up. According to the article by Philly.com, this could also present some issues when using and analyzing the results, “a panel that’s used repeatedly has a tendency to suffer from people dropping off the panel or ignoring surveys, and there is also the chance of panelists becoming conditioned in ways that skew results.”
Regardless, many experts agree that this is on the cutting edge of research tactics and believe its uses are valuable, “Stats experts say the approach has worth. It’s a good way to measure issues’ importance to a community, or to determine attitudes,” Santos is quoted saying in the article by Philly.com, “Using a random panel is more precise, but some questions don’t need to be accurate to the decimal point to provide insight into a population.”
Temple University, however, is taking measures to flat line some of these problems by “actively working to find volunteers in communities underrepresented in opt-in studies” and “steadily [recruiting] to ensure the panel continues to be representative of the city’s demographics” according to the article by Philly.com.