Checking a Model for Convergence

Many calculations have exact formulas, where, if you apply the formula, the result is not in dispute. For example, if 10 people liked a product and 20 people were asked whether they liked it or not, the percentage to like the product is exactly 10/20 = 50%.

Most more-advanced calculations (such as clustering and Hierarchical Bayes) have no exact formulas. Instead, very clever trial-and-error approaches are used to calculate the results. However, it is not always obvious when the result is good enough.

When a result from a trial-and-error approach is good enough, it is said to have converged. This article discusses:

• How to work out if convergence has not occurred
• How to achieve convergence
• The impact of non-convergence

How to work out if convergence has not occurred

Most software will warn users if the model has not converged. For example, if SPSS's k-means cluster analysis has not converged, it produces the following warning:

Similarly, Displayr automatically puts a warning icon next to the page and model in the Pages tree if it has not converged:  . When the model is clicked on, the detail of the warning is then shown above the object inspector. For example:

The one general exception to this is the Hierarchical Bayes model. It is only in recent years that reasonably reliable ways of assessing its convergence have been developed. Consequently, while Q and Displayr will provide a warning if convergence is not achieved, you should not assume other software has such a warning. For an overview of convergence of Hierarchical Bayes, see Checking Convergence When Using Hierarchical Bayes for MaxDiff. Though written from the perspective of MaxDiff, it is also applicable for other applications, such as conjoint. 

How to achieve convergence

The general solution when a model has not converged is to increase the number of iterations. The trade-off is that increasing the number of iterations can increase the amount of time taken to perform the analysis - significantly. For example, running a simple straight-forward model with 100 iterations may take a few minutes to process vs almost ten minutes for 1000 iterations. It's not out of the question for a model, that is sufficiently complex, to run hours.

The impact of non-convergence

Convergence is a hygiene test. As with much hygiene, such as washing hands and wearing masks when there are air-born viruses, this hygiene test falls into the category of being prudent rather than always necessary.

Broadly speaking, non-convergence can mean any of the following:

• There is a fundamentally incorrect assumption built into the model.
• There is an error in the algorithm being used to fit the model.
• The way that convergence is being checked is too stringent.
• There is an error in how convergence is being checked.
• The person that wrote the software assumed too few iterations (this is particularly common when the software is very old).

Models that fail this test are commonly still good. However, as with the rest of hygiene, it doesn't follow that it is unnecessary - convergence is something that reduces the chance of problems rather than eliminates them.

The outputs below are from Displayr. The output on the left is from a Hierarchical Bayes choice-based conjoint model estimated with Displayr's default number of iterations (100). This model had yet to converge. The model on the right had converged at 2,000 iterations. Note that the results are close to identical. However, as mentioned above, this is not guaranteed to be the case.