Writing a Questionnaire for a Conjoint Study

A choice-based conjoint questionnaire will typically contain:

• Screening questions, questions for quota management, and weighting
• The choice questions
• Questions for validation
• Questions for other objectives that will be addressed in the same study.

Screening questions, questions for quota management, and weighting

As with most questionnaires, choice-based conjoint questionnaires typically require:

• Screening questions at the beginning of the study to check that people qualify (e.g., if doing a choice-based conjoint study on chocolate, you likely do not want to include non-chocolate buyers in the study).
• Questions for quota management (e.g., age, gender, and region, to increase the likelihood that the study’s sample is consistent with the population at large).
• Questions for use in weighting the sample to be representative of the population at large (e.g., age, gender, region, education, product usage)

The choice questions

When presenting the choice questions to respondents, it is a good idea to

• Have an introductory screen explaining why the research is important and making it clear that there will be multiple similar questions. Choice questions are a bit painful for respondents. The first one is a bit hard to figure out. The second and ensuing questions are boring. The questions can be quite difficult to answer. Consequently, it is inevitable that some level of error, be it through confusion or random clicking, will infect the data collected in a choice model. We can increase the quality of the data by making the tasks as simple as possible and asking as few questions as possible. But, we can also improve the quality of the data by explaining to the respondents why it is in their interest to provide high-quality data.
• Introduce the second question with something like “Here is another question. It has the same structure as the previous one, but the details changed. Please read it carefully.
• Make it easy for the respondent to see how many choice questions they have completed, and how many are yet to come. From a respondent's perspective, the questions are repetitive, and the user does not want to feel they go on forever.

Questions for validation

When conducting a choice-based conjoint study, it is often useful to ask additional questions that allow you to demonstrate that the data collected by the choice-based conjoint is valid. There are a number of ways of doing this, including:

• Have a question after the first choice question asking “Why did you make that choice?”, and permitting the respondent to provide an open-ended answer. Reviewing the resulting data can pinpoint problems. If doing this approach, show the respondent the previous choice or give them the ability to use a back button to return to it. 
• Creating holdout tasks showing specific scenarios of interest. These are questions with either a similar structure to the choice questions or designed to have superior ecological validity. For example, one question may show alternatives that represent the current state of the market and the second may show a key scenario that the stakeholders are interested in (e.g., with one of the products at a different price, or, a new product introduced).
• Any other questions that may give additional insight into interpreting the results of the choice-based conjoint, such as:
  • usage
  • perceptions
  • relevant attitudes
  • demographics. 

See also

Checking the External Validity of a Choice Model.