How to Check Attributes and Attribute Levels

The over-riding concerns when identifying relevant attributes and attribute levels for a conjoint study are that the attributes and their levels:

1. Can be used to create all scenarios of interest.
2. Are interpreted consistently by respondents and analysts.

The attributes and levels can create all the scenarios of interest

The following checklist helps to ensure that the attributes and levels satisfy the theoretical criteria:

• Attributes and their levels describe all interesting future scenarios. A conjoint study is usually designed with the goal of giving insight into some future scenario (e.g., what will happen when a price is reduced or when a new product is launched). Such a scenario can only be evaluated if the attributes and levels accurately describe the future scenario. For example, Miles Per Gallon of Gas is a poor attribute in the car market, as electric vehicles cannot be described using this attribute. A better attribute is the energy cost per mile. For attributes that are continuous, such as price, you do not have to test all the levels. Instead, you can test the end-points and key price points in-between, relying on interpolation for the rest.
• Attributes and their levels describe all interesting current scenarios. More often than not it is appropriate to test a conjoint model by verifying that it can accurately predict the current state of the market. This can only be done if the attributes and levels accurately describe the current market. For example, if studying price elasticity of brands of beer, you can't predict the current market if you ignore any major brands or formats (i.e., can and bottle formats).
• Attributes and their levels describe interesting historic scenarios. Sometimes markets have recently experienced some interesting competitive action, such as a change in pricing, product features, or the addition or removal of a product from the market. By ensuring that the attributes and levels accurately describe these historic scenarios it is possible to gain additional information to assess in checking and improving a conjoint model.
• Attributes must be mutually exclusive, within and across attributes. For example, if studying preferences for airlines it makes no sense to have one attribute dealing with time from arrival at the airport to boarding of the plane and another dealing with check-in time, as one is a consequence of the other (unless a price change is anticipated).
• Attributes must be exhaustive if a market share prediction is required.  If the objective is to simply find out the role of, say, price versus quality, you can design a methodology that just contains price and quality attributes. If, however, the marketer wants to predict market share, the model must include every attribute that is believed to relate to consumer choice. In practice we only ever attempt to model attributes that are the key determinants of choice, thereby ensuring that we can never give completely accurate predictions from choice models. In general, the greater the number of attributes included in a model, the more accurate the model can be (although questionnaire complexity and length may reduce its accuracy). The simplest method of doing this is to create a super-attribute; for example, you could let consumers trade-off price versus brand, with the brand description communicating all of the other relevant information, thereby making the attributes exhaustive.

Descriptions are interpreted consistently by respondents and stakeholders

Ultimately, a conjoint study is used by its stakeholders to make conclusions about how the world works. If the descriptions are written in a way that the stakeholders infer conclusions that are different from those made by respondents when completing a questionnaire, the research will be misleading.

Two things increase the odds that attribute and attribute level descriptions are correctly and consistently interpreted by both respondents and the users of the conjoint study:

• Descriptions must be simple. When consumers are evaluating many alternatives or attributes it is important to ensure that the task is made as easy for them as possible. Attribute level descriptions that contain large amounts of text can either increase the time taken to complete the task or, cause the respondent to simply not read the description. In such circumstances prescaling (Louviere, Jordan J. (1988), Analyzing Decision Making: Metric Conjoint Analysis, SAGE Publications Inc, p. 51.) may be advisable – that is, use attribute levels with names like “standard”, but either explain in detail what these attributes mean beforehand or, collect data on what consumers perceive these attribute levels to mean.
• Descriptions must be unambiguous. It is tempting to use somewhat ambiguous terms to describe attributes and attribute levels. Referring to “price discounts”, for example, as an attribute, is particularly dangerous as consumers will differ from each other in their perceptions of what it means, and they will also differ from management, making it almost impossible to interpret the results of the research using this attribute.
• There should be as few attributes and levels as possible.
  • The time allotted for the interview. The more attributes and attribute levels, the longer the interview. While we can sometimes convince respondents to participate in long interviews, it is doubtful that the quality of the responses is high towards the end.
  • The level of respondent involvement. In some categories, such as insurance and banking, there may be numerous relevant attributes. Respondent interest, and, consequently involvement, may be so low as to make research conducted using the complete set of attributes to be of dubious quality.
  • Task complexity. The more attributes, the more difficult the task.

It can be useful to get additional feedback on the attributes and attribute levels by conducting think-aloud interviews once the questionnaire has been written.