Computer-assisted web interviewing

With the increasing use of the Internet, online questionnaires have become a popular way of collecting information. The design of an online questionnaire often has an affect how the quality of data gathered. There are many factors in designing an online questionnaire guidelines, available question formats, administration, quality and ethic issues should be reviewed.

Using Online Questionnaires

There are several reasons why someone would utilize online questionnaires as their preferred testing method. A few of the advantages and disadvantages of this method have been summarized below [1]:

Advantages

• The administrator has greater flexibility in displaying questions. Questions can be displayed with [1]:
  • Check boxes
  • Pull down menus
  • Pop up menus
  • Help screens
  • Graphics
• An online forum allows responses to be received more quickly from subjects [3].
• This method is also cheaper to administer, as there are no costs associated with purchasing paper or other materials for printing. Postage costs are also mitigated [9].
• Since data is collected into a central database, the time for analysis is subsequently reduced [10].
• It is easier to correct errors on an online questionnaire, since the administrator does not have to reprint all the questionnaires for distribution [1].

Disadvantages

• Not everyone has access to the Internet, so the response rate is limited [1].
• Many people are not receptive to completing questionnaires online [5].
• Studies indicate that the demographic that responds to online questionnaire invitations are generally biased to younger people [6].
  

Sections:
1. Using Online Questionnaires 2. Online Questionnaire Paradigm 3. Response Format 4. Administration of Online Questionnares 5. Quality of Online Questionnaires 6. Ethics in Online Questionnaires

Online Questionnaire Paradigm

There are several important paradigms that should be considered when creating an online questionnaire [1].

Collection and Prioritization of Data

• The objectives of the initial inquiry need to be reviewed to determine what information needs to be gathered [8].
  
• The required information should be ranked in order of significance in an unbiased manner. Topics should not lead a person into making false conclusions [7].
  

Online Questionnaire Format

• The questionnaire should begin with a short introduction that informs the subject why the questionnaire is being conducted [7].
  
• Questions for questionnaire should be created in the most appropriate type of format that facilitates understanding [1].
  
• In creating the layout of the online questionnaire, “smart branching” should be utilized to lessen complexity. For example, if a subject selects “yes” to a question, the questionnaire would automatically jump to the next relevant question and vice versa [9].
  
• A brief “thank you” note should be included at the end of the questionnaire[9].

Questionnaire Length

• As a rule of thumb, the questionnaire should not exceed 5 minutes [7].
  
  • Generally, 4 multiple choice questions take 1 minute [7].
    
  • One short answer question is equivalent to 3 multiple choice questions[7].
    

Prototyping

• A sample of the questionnaire should be distributed to at least 5 people, prior to publication on the web. Upon their completion of the questionnaire, feedback from the participants should be obtained [1].
  
  • Information relating to whether they understood the main point of the questionnaire should be gathered [3].
    
  • It is important to distinguish if participants had any difficulties with any of the questions [5].
    
  • The feedback of the subjects should be utilized to make any necessary changes to the questionnaire [3].
    

Sections:
1. Using Online Questionnaires 2. Online Questionnaire Paradigm 3. Response Format 4. Administration of Online Questionnares 5. Quality of Online Questionnaires 6. Ethics in Online Questionnaires

Response Format

In designing a questionnaire, the evaluation method should be kept in mind when choosing the response format. In this section, there are various response formats that can be used on online questionnaires [1].

Radio Button

The respondent is required to click on the circle, which corresponds to the desired answer. A dot in the middle will appear once an answer is chosen. Only one answer can be chosen [1].

• Recommended when the choice of answers are mutually exclusive.
  
• No default answer should be provided. If a default answer is provided, it may be mistaken as an answer if the respondent chooses to skip the question.
  
• Require precision in clicking [1].

Check Boxes

The respondent is required to click on the boxes next to the answer that corresponds to the desired choice. A checkmark will appear in the box once an answer is chosen. More than one answer can be selected [1].

• If there are many options, a simple matrix is recommended [5].
  
• When using check boxes, if more than one answer can be checked, it should be specified in the instructions [5].
  
• If “none of the above” is required, provide it with a radio button to prevent an erroneous check on this choice incase another answer has been chosen [1].

Drop Down Menus

The respondent is required to click on the arrow on the far right side of the box [1]. Once clicked, a display with a list of answers will appear [3]. A scroll bar may appear on the right hand side if a large number of answers are displayed [3]. The respondent can click on the highlighted section of the list to select an answer [1]. This answer will then appear in the box. Only one answer can be selected for this type of question [1].

• Good option for long lists such as state/country of residence [3].
  
• Should be avoided for items where typing is faster. For example, year of birth [3].
  
• In designing drop-down boxes, do not make the first option visible to the respondent. This may be misleading where no answer may be chosen [5].

Text Boxes

The respondent is required to click inside of the text box to get the cursor inside the box. Once the cursor is blinking inside of the box, the answer of the question can be typed in [5].

• Make the size of the text box according to the desired and required amount of information from the respondent [1].
  
• Provide concise and clear input instructions. For example, if the questionnaire is asking for a date, the format should be specified (MM/DD/YYYY) [1].
  

Sections:
1. Using Online Questionnaires 2. Online Questionnaire Paradigm 3. Response Format 4. Administration of Online Questionnares 5. Quality of Online Questionnaires 6. Ethics in Online Questionnaires

Administration of Online Questionnaires

Once the questionnaire is designed, it must be administered to the appropriate sample population for data collection [6]. Attracting the appropriate target audience often requires advertisement. There are various methods used to attract participants

• bulletin boards
• mass emails
• advertisements in commercial areas
• mail
• monetary incentives
• discounts on company products

This usually helps in attracting willing participant which ultimately provide better quality data as opposed to relunctant participants.

Location of administration for the online questionnaire may be a factor in the administration if a specific environment is required [1]. A quiet environment may be needed for questions, which require a certain amount of concentration [4]. The questionnaire may need to be administered in a secluded environment to protect sensitive information provided by the participant [8]. Security measures in the software may also need to be added in these cases [9]. In contrast, online questionnaires may also be very informal, relaxed and can be conducted in the comfort of someone’s home [1].

Sections:
1. Using Online Questionnaires 2. Online Questionnaire Paradigm 3. Response Format 4. Administration of Online Questionnares 5. Quality of Online Questionnaires 6. Ethics in Online Questionnaires

Quality of Online Questionnaires

Questionnaire quality can be measured through the value of the data obtained and participant satisfaction [1]. To maintain a high quality questionnaire length, conciseness and question sequence should be considered [4]. First, questionnaires should only be as long as they need to be [3]. Conciseness can be achieved through removing redundant and irrelevant questions, which can add frustration to the participant, but not value to the research [6]. Finally, placing questions in a logical sequence also gives the participant a better mental map as he/she is filling out the questionnaire [3]. Moving randomly between subjects and having answers in a non-intuitive sequence can confuse the participant [1].

Sections:
1. Using Online Questionnaires 2. Online Questionnaire Paradigm 3. Response Format 4. Administration of Online Questionnares 5. Quality of Online Questionnaires 6. Ethics in Online Questionnaires

Ethics in Online Questionnaires

Ethical issues should be considered when gathering data from a target audience. Below are common things one should keep in mind when considering the rights and interests of the participant [1].

• Participant should not be obliged to answer any of the questions [8].
  
• Questionnaires should have the option to be anonymous [9].
  
• Confidentiality must be imposed on certain questionnaires. Identification may be required on questionnaires that need follow up. Although, in this case the administrator may choose to use identifying numbers rather than names. In this case, the participant of the questionnaire should completely understand what the number is used for and why it is there [4].
  
• Questions should have the option of “I don’t know” so the participant feels he/she has the opportunity to plead ignorance so that inaccurate data is not provided [1].
  
• Questions should not trick the participant. They should be worded clearly; the participant should feel comfortable and know exactly what he/she is responding to [5].
  
• Participant in most cases should know why the questionnaire is taking place and what the information will be used for [3].
  
• In some cases, the questionnaire should be reviewed by some an ethics committee or outside party [1].
  

Sections:
1. Using Online Questionnaires 2. Online Questionnaire Paradigm 3. Response Format 4. Administration of Online Questionnares 5. Quality of Online Questionnaires 6. Ethics in Online Questionnaires

References

[1] Sharp, H., Rogers, Y., Preece, J., Interaction Design: Beyond Human-Computer Interaction. John Wiley & Sons, Inc. 2002

[2] Norman, Donald A. The Design of Everyday Things. Basic Books. 2002

[3] Bradburn, Norman M., Sudman, Seymour, Wansink, Brian, Asking Questions: The Definitive Guide to Questionnaire Design – For Market Research, Political Polls, and Social and Health Questionnaires. Jossey-Bass. 2004

[4] Couper, Mick P., Baker, Reginald P., Clark, Cynthia Z. F., Martin, Jean, Nicholls, William L., O'Reilly, James M., Computer Assisted Questionnaire Information Collection. John Wiley & Sons, Inc. 1998

[5] Presser, Stanley, Rothgeb, Jennifer M., Couper, Mick P., Lessler, Judith T., Martin, Elizabeth, Martin, Jean, Singer, Eleanor, Methods for Testing and Evaluating Questionnaire Questionnaires. John Wiley & Sons, Inc. 2004

[6] Groves, Robert M., Fowler, Floyd J., Couper, Mick P., Lepkowski, James M., Singer, Eleanor, Tourangeau, Roger, Questionnaire Methodology. John Wiley & Sons, Inc. 2004

[7] QuestionPro, "Preparing an Online Questionnaire - How to Conduct an Online Questionnaire" [cited Mar 10, 2007]. Available HTTP: http://www.questionpro.com/akira/showArticle.do?articleID=build01

[8] National Research Council of Canada, "Online Questionnaire Design" [cited Mar 10, 2007]. Available HTTP: http://iit-iti.nrc-cnrc.gc.ca/projects-projets/questionnaire_e.html

[9] Online Questionnaire Design Guide, "Web Based Questionnaires" [cited Mar 10, 2007]. Available HTTP: http://lap.umd.edu/questionnaire_design/questionnaires.html

[10] StatPac, "Questionnaire Design - General Considerations" [cited Feb 24, 2007]. Available HTTP: http://www.statpac.com/questionnaires/questionnaire-design.htm

[11] Quirk's Market Research Review, "Comparing Seven Forms of On-Line Questionnaireing" [cited Feb 24, 2007]. Available HTTP: http://www.quirks.com/articles/article.asp?arg_ArticleId=510

See Also

Questionnaires

Questionnaire construction

Paid survey

Data Mining

External Links

Preparing an Online Questionnaire - How to Conduct an Online Questionnaire

National Research Council of Canada

Online Questionnaire Design Guide

Questionnaire Design: General Considerations

Comparing seven forms of Online Surveying

Sections:
1. Using Online Questionnaires 2. Online Questionnaire Paradigm 3. Response Format 4. Administration of Online Questionnares 5. Quality of Online Questionnaires 6. Ethics in Online Questionnaires