SPSS Inc. SPSS Navigation
SPSS Navigation

Choosing the right administration method for your research

by Pamela Narins, Manager, Market Research

Conducting a survey research project can be a big and expensive undertaking. Getting usable results on time and within budget requires extensive planning and organization. In this and subsequent editions of this article, I will outline the stages of putting together a successful survey research project.

The survey process contains many steps, each of which has a direct impact on those that follow. In designing research, you must go through these steps which help you respond to specific needs. Figure 1 is a chart of questions you should ask yourself before starting a survey, and the specific action that should follow. The first two editions of "Survey Samplings" concentrated on step number 2. Now we are ready to select an administration method.

Figure 1
Figure 1

Choosing a method
Survey administration means the way in which a questionnaire is presented to a respondent. There are three traditional methods of survey administration: mailed, telephone and face-to-face surveys. Mailed surveys are usually self-administered; that is, the respondents fill in the questionnaires themselves. Telephone and face-to-face surveys are administered by another person.

Choosing one of these methods depends upon several factors: the availability of time and funds, the sample population and the amount and kind of information you want to collect. The following discussion of administration methods points out their strengths and weaknesses and some of the impact each has on the fourth step - question design.

Mailed survey: Strengths
This can be a fairly inexpensive method. Typically, relatively inexpensive printing and distribution costs are incurred, but you need not send survey administrators off-site or pay for respondents to come to you.

This is also a good method to get more than just a gut-level reaction since respondents have plenty of time to think about their answers. The questions can be more complicated than if the survey were administered over the phone. Interviewer bias, which will be discussed below, is not a problem because an interviewer is not present. Respondent anonymity is another big advantage of this method.

Mailed survey: Weaknesses
Low response rates are the biggest problem associated with mailed surveys. Additionally, there is the possibility that those who do respond are not representative of the population you are trying to survey. This response bias can be very serious if you are doing mailed surveys and requires diligence to counteract.

Another possible drawback of mailed surveys is the length of time required for the field period. Respondents should be allowed between one month and six weeks to complete and return their surveys.

As far as planning goes, finding or buying an appropriate mailing list can also present a challenge, depending on the population you wish to survey and your budget limitations.

Telephone survey: Strengths
These surveys have increased in popularity as the technology to handle them has advanced. Compared to face-to-face surveys, they are relatively inexpensive to conduct. They have a very short field period and a high response rate. Additionally, the possibility of interviewer-introduced bias is reasonably low (see face-to-face interviews below).

Unlike mailed surveys, researchers have some control over who responds. Telephone interviews usually begin with a series of screening questions, ensuring that the possible respondent fits the parameters of the survey. For example, you may only want to interview people who eat a certain kind of breakfast cereal or have a specific job.

Telephone survey: Weaknesses
The questionnaires should not be very long (try to stick to 10 minutes maximum). Questions should be easy to read and easy to understand; pretesting is a good idea to verify that questions which seem reasonable on paper actually translate well when spoken. Some open-ended questions are possible, but should be kept at a minimum. The interviewer's ability to accurately capture the answer over the phone is more limited than during a face-to-face interview.

While interviewer bias is possible, the researcher can reduce its likelihood by monitoring phone personnel.

You should try and eliminate the possibility that your telephone survey be confused with a telemarketing effort. Many times sales calls can be disguised as research. Growing discussions within the research community address this issue as it relates to ethics, data quality and respondent fatigue (the phenomenon that results from an over-surveyed population).

Face-to-Face interview: Strengths
The face-to-face interview has many advantages, one being a very high response rate. Also, access to difficult-to-reach populations, such as people without telephones or those with unusual work schedules, is possible.

Additionally, face-to-face interviews give you with an enormous amount of control over questionnaire length, format and complexity. Effective probing and reading of body language by the interviewer can add data which is unattainable with other methods.

Face-to-Face interview: Weaknesses
This method can require the most resources. Face-to-face interviews can be very expensive because field interviewers must be highly trained. Training and travel costs alone can be prohibitive. Other drawbacks include the length of time required for completion of the field period, the lack of perceived respondent anonymity and the chance that interviewer bias will be introduced.

This last point, interviewer bias, should not be overlooked. Even the most professional interviewers can give verbal or non-verbal cues to their respondents. As a result, respondents may alter their answers in order to impress or please the interviewer.

Combining methods
Sometimes, combining administration methods will give you the best results. For example, if you want to do a telephone survey which requires a visual aid, you need not abandon the telephone survey for face-to-face interviews. You could mail or fax part of your survey to the respondent before continuing with the call.

Another way to mix methods is to leave an additional questionnaire after doing a face-to-face interview. This way, you could collect information from other household members or allow respondents additional time to answer questions.

A telephone survey could be useful as a screening tool by obtaining information about a respondent, then asking them to participate in a follow-up, face-to-face interview.

Also useful, is focus group before a quantitative study to help define the study's scope, or even after a survey to help add texture to results.

Figure 2
Figure 2

Summary of strengths and weaknesses
The table in figure 2 summarizes the relative strengths and weaknesses of the three survey methods. When evaluating these methods, ask yourself how important each one of the characteristics is to your project. Is respondent anonymity critical? If so, you may want to avoid face-to-face interviews or modify that method. If you need a high response rate and feel that a mailed survey is the right approach, you will have to be diligent in your follow-up contacts. Adding an incentive to return a mailed survey is often helpful, as are follow-up phone calls, cards or other reminders.

Impact of strengths and weaknesses on question design
As I mentioned above, the choice of an administration method has a direct impact on the format of your questions. Questions asked over the phone should be short and easy to read. Questions written for a mailed survey can be slightly longer and more complex. They can also refer to other parts of the questionnaire or to additional materials included with it.

Face-to-face interviews allow for the greatest flexibility of question types: questions can be long, sensitive, have many parts, refer to visual aids and more.

While there is no "right" way to solve survey research problems, a better understanding of strengths and weaknesses inherent in the tools will help you come up with the most effective solution.

Editor's note: the next edition of "Survey Samplings" will discuss components of good question writing.




Other Links
SPSS P&S | SPSS MR | SPSS Science | SPSS Quality
Data Mining | Customer Services | Technical Support | SPSS Worldwide | Press Center
Investor Center | Cool Stuff | Consulting | SPSS Training | How to Order
Software | Newsletters | Books & References | FAQs

Webmaster | Talk to Us | Search | SPSS Home
Copyright 1999, SPSS Inc. All rights reserved.
Terms and Conditions