Common Mistakes when Creating Weights

It is not unusual to get weird error messages from weighting software (e.g., “no convergence after 2500 iterations”). And, sometimes, the results just don’t make sense (e.g., too low effective sample sizes). Common causes of errors are:

No sample for a target group

A sub-group in the population can only be weighted to be greater than 0% if there are some sub-group members in the sample. For example, if the survey did not interview in Hawaii, then either:

• The target for Hawaii needs to be set to 0 (or removed from the adjustment variable and the targets).
• The Hawaii category needs to be merged with some other category.
• The state variable needs to be removed or replaced as an adjustment variable (e.g., by four regions).

Targets for categorical adjustment variables that don’t add up

For example:

• Targets are expressed in percentages that do not add up to 100%.
• Population targets (i.e., which include expansion factors) that are different for different adjustment variables.

Impossible numeric targets

The target for a numeric adjustment variable needs to be no greater than the highest observed value and no smaller than the lowest observed value for the variable. Otherwise, an error will occur.

Inconsistent categorical targets   

Consider the two sets of targets provided below. Provided that the underlying data in the survey is consistent, these targets are impossible, as the one on the left will attempt to create a weight where people with children account for 50% of the weighted sample, whereas the one on the right implies that 40% have children. No weight can satisfy these two sets of targets.

With large numbers of variables, this type of mistake is challenging to spot.

Inconsistent numeric targets        

Numeric targets also need to be consistent. Trivial cases are easy to spot. For example, if you have two share variables that sum up to more than 1.0, you will get an error (assuming that they only sum up to 1 in the data itself).

However, numeric variables can have difficult-to-spot problems. Suppose two numeric variables are highly correlated. In that case, you need to consider this correlation when specifying their targets (e.g., it may not be possible to increase the average body weight of a sample while reducing its average height).

Using weight factors instead of targets

Let’s say that the sample contains 50% men, and the target is 70%. A weight factor of, say, .5/.7 can be computed for men. A common mistake is to compute the weight factor and enter it as the target (the correct number to enter is 50%).