The internet has become a popular medium for conducting research, surveys especially. For an internet survey to be successful, it must follow certain guidelines. Science Buddies wrote an article filled with good suggestions on how to create an effective internet survey, summarized here:
- Keep the questions short, while still ensuring that you capture all of the information that you need.
- Develop a set of objectives for your research and list out that information that you are trying to capture.
- It is important to know when to use multiple choice questions, and open questions. Structured questions make data collection and analysis easier and quicker. Open ended questions are good when you’re trying to capture new ideas.
- Make sure the answers to multiple choice questions don’t overlap.
- Include an ‘other’ option in multiple choice questions to ensure accurate data.
- All of the responses in multiple choice questions should be similar so that no single response stands out. Consistency ensures that you aren’t leading respondents to a certain answer.
- In general, if you are trying to get an opinion about something, it is better to use a rating question rather than a ranking question. Rankings basically just tell you how much the respondent likes something relative to something else.
- Minimize the number of open ended questions in your survey.
- State your intentions with the research, and people will be more willing to help.
- Make sure you include instructions.
- Don’t ask for personal information unless you need it, and put it at the end of the questionnaire.
- Ask only one question at a time- no double barreled question. For example, How have teachers and students at your school responded to the new lunch period? This question is ambiguous because it is asking about both teachers and students, and should be split into two separate questions.
- Make sure that your questions are unbiased- no leading questions.
- Don’t ask questions that respondents won’t be able to answer, or can’t remember.
It’s easy to analyze the data from multiple choice questions, but analyzing data from open ended questions takes a bit more work. The Planning Council for Health and Human Services came out with an article with some helpful tips on how to analyze data from open-ended questions, which is summarized here:
- Read through all of the responses
- Develop categories that include the themes that emerged in your initial review
- Assign each response to a category- this is known as ‘coding’
- Check your categories. You might find most of your responses fall into one category that can be broken into more specific subcategories
- Review for major theme
- Identify patterns and trends
If you follow the tips from these two articles, you are well on your way to designing and analyzing an effective survey. As a part of this class, I actually designed and analyzed my own survey based on the above suggestions. My survey sought to answer whether participants thought internet-based surveys provide the same quality of data as non-internet-based surveys and why they think that. After I finished designing it, I posted a link to it on my Facebook and got 31 responses over 2 days.
The first question I asked was multiple choice being ‘Which of the following survey formats would you prefer to take given the choice?’ with the options ‘internet based,’ ‘in-person,’ ‘via mail,’ and ‘over the phone.’ I made sure not to make any of my questions biased in any way. Respondents overwhelmingly preferred internet based surveys. One participant voted for in-person, and another for via mail, but everyone else chose internet based.
The next question was ‘What do you think about the quality of online surveys?’ with a follow-up question of ‘Why?’ I made sure to make most of my questions multiple choice, the only open-ended one I asked was ‘Why?’ The results on this question were pretty split, with most people thinking that online surveys are just as good as surveys taken in another format. Those who thought internet surveys were higher quality argued that they are more convenient, can reach more people, and people are more likely to answer honestly. Those who argued that the quality is the same said that people will answer the same in any format, and format is just a personal preference. Those who argued that quality of online surveys is worse said that it’s easy to skew results, people won’t take them seriously or rush them, or that the sample of people who take an online survey isn’t representative of the population.
Next I asked about experience and frequency of use of the internet to see if people who used the internet more frequently were more likely to prefer online surveys. Everyone who took the survey was experienced with the internet, and reported using it either multiple times a day or almost constantly, so it’s hard to distinguish if there’s a trend there.
Next I asked for some demographic information so I could see if there were any trends among certain groups of people. I was careful not to overlap answer choices for age so that there was no confusion. I also asked about education level. The major of people who took the survey were in between the ages of 18-22, however there were a few respondents that were 36+. The majority also has completed some college, or has a Bachelor’s degree. A couple of respondents reported having Master’s degrees. I looked for trends in survey preference among the older adults/those with higher educations, however I didn’t see any; their answers were representative of the rest of the sample.
According to my survey, people prefer online surveys. However, opinion on the quality of online surveys is still divided among respondents. The majority agreed that online surveys are better, or at least equivalent in quality to surveys given in other formats. Demographic factors did not seem to have an effect on the results, however my sample was small and results may be different in a survey with a large sample size. After getting this experience designing surveys online, I can honestly say that it definitely is an easy way to conduct research and to get fast results.
Surveys aren’t the only way to obtain data about people online though. Tweets can be used as a way to obtain data for heart disease and contagious illnesses, website hits can be used as data for health epidemiology, and Yelp reviews can be used as data for food-borne illness.
Harrison et al. did a study called “Using Online Reviews by Restaurant Patrons to Identify Unreported Cases of Food Borne Illness” which looked more in depth at how Yelp reviews can be used as data for food borne illness. The researchers in the study analyzed 294,000 Yelp reviews using a software program developed for the study. Of those reviews, 893 were found to have key words in the review such as “sick” and “vomit.” From there, a food borne disease epidemiologist looked at the reviews with the keywords, where 499 were found to have described an event consistent with food borne illness. 129 of these required further investigation again, resulting in 27 phone interviews, and 3 restaurants being identified as having multiple food handling violations. The main take away from this study is that online reviews may help to identify unreported outbreaks of food borne illness and restaurants with deficiencies in food handling.