Choosing the right administration method for your research
by Pamela Narins, Manager, Market Research
Conducting a survey research project can be a big and expensive undertaking. Getting usable results on time and within budget requires extensive planning and organization. In this and subsequent editions of this article, I will outline the stages of putting together a successful survey research project.
The survey process contains many steps, each of which has a direct impact on those that follow. In designing research, you must go through these steps which help you respond to specific needs. Figure 1 is a chart of questions you should ask yourself before starting a survey, and the specific action that should follow. The first two editions of "Survey Samplings" concentrated on step number 2. Now we are ready to select an administration method.
Choosing a method
Choosing one of these methods depends upon several factors: the availability of time and funds, the sample population and the amount and kind of information you want to collect. The following discussion of administration methods points out their strengths and weaknesses and some of the impact each has on the fourth step - question design.
Mailed survey: Strengths
This is also a good method to get more than just a gut-level reaction since respondents have plenty of time to think about their answers. The questions can be more complicated than if the survey were administered over the phone. Interviewer bias, which will be discussed below, is not a problem because an interviewer is not present. Respondent anonymity is another big advantage of this method.
Mailed survey: Weaknesses
Another possible drawback of mailed surveys is the length of time required for the field period. Respondents should be allowed between one month and six weeks to complete and return their surveys.
As far as planning goes, finding or buying an appropriate mailing list can also present a challenge, depending on the population you wish to survey and your budget limitations.
Telephone survey: Strengths
Unlike mailed surveys, researchers have some control over who responds. Telephone interviews usually begin with a series of screening questions, ensuring that the possible respondent fits the parameters of the survey. For example, you may only want to interview people who eat a certain kind of breakfast cereal or have a specific job.
Telephone survey: Weaknesses
While interviewer bias is possible, the researcher can reduce its likelihood by monitoring phone personnel.
You should try and eliminate the possibility that your telephone survey be confused with a telemarketing effort. Many times sales calls can be disguised as research. Growing discussions within the research community address this issue as it relates to ethics, data quality and respondent fatigue (the phenomenon that results from an over-surveyed population).
Face-to-Face interview: Strengths
Additionally, face-to-face interviews give you with an enormous amount of control over questionnaire length, format and complexity. Effective probing and reading of body language by the interviewer can add data which is unattainable with other methods.
Face-to-Face interview: Weaknesses
This last point, interviewer bias, should not be overlooked. Even the most professional interviewers can give verbal or non-verbal cues to their respondents. As a result, respondents may alter their answers in order to impress or please the interviewer.
Another way to mix methods is to leave an additional questionnaire after doing a face-to-face interview. This way, you could collect information from other household members or allow respondents additional time to answer questions.
A telephone survey could be useful as a screening tool by obtaining information about a respondent, then asking them to participate in a follow-up, face-to-face interview.
Also useful, is focus group before a quantitative study to help define the study's scope, or even after a survey to help add texture to results.
Summary of strengths and weaknesses
Impact of strengths and weaknesses on question design
Face-to-face interviews allow for the greatest flexibility of question types: questions can be long, sensitive, have many parts, refer to visual aids and more.
While there is no "right" way to solve survey research problems, a better understanding of strengths and weaknesses inherent in the tools will help you come up with the most effective solution.
Editor's note: the next edition of "Survey Samplings" will
discuss components of good question writing.