Overview of How to Finding Insights in Data

Once the data has been collected, we need to find the meaning in the data and find the most effective way of communicating that meaning. This process typically includes four stages:

1. Creating an analysis plan
2. Data reduction
3. Exploratory analysis to iteratively explore ideas
4. Creating the reporting

These are performed iteratively rather than as a set list of steps to follow in order.

Creating an analysis plan

An analysis plan is a written document that describes how data is to be analyzed. The specifics of the analysis plan always depend on the type of data and the study's goals.

As an example, for survey data, an analysis plan typically consists of instructions for:

• Checking the data.
• Cleaning the data.
• Weighting the data (if needed).
• Data preparation.
• Advanced analysis (if needed).
• List of required tables - commonly referred to as table specs.

• Planned visualizations.

Data reduction

When an analysis plan is created, it should create a relatively large set of analyses outputs, most of which are tables. The core stage of finding meaning in data is systematically taking this large set of analyses and compressing them into nuggets of information. That is, reduce the amount of data.

To perform data reduction, the following techniques can be performed iteratively and repeatedly:

• Delete insufficiently-rigorous analyses.
• Delete uninteresting analyses.
• Remove clutter.
• Merge similar things.
• Replace data with summary statistics.
• Reorder.
• Change the scale.
• Decompose.
• Use "common sense".

For more detail, see Data Reduction.

Exploratory analysis to iteratively explore ideas

A former professional boxer, Mike Tyson, once famously said, "Everybody has a plan until they get punched in the mouth." It is the norm, rather than the exception, that the analysis plan does not envisage all the required analyses. During the exploratory analysis, we can uncover things that require further analysis and exploration. Other times the planned analysis fails to reveal anything interesting at all. 

The main mechanisms for ensuring appropriate exploratory analysis are to:

• Revisit the analysis plan and expand it.
• Ensure that the person performing the data crunching is the same person interpreting the results. The key concept to be aware of here is the OODA Loop, a concept in air combat that stands for observe, orient, decide, and act. The basic idea is that the faster a pilot can observe a problem, work through the options (orient), decide, and implement their plan (act), the more likely they are to win. Whereas, if the person responsible for creating the analysis (act) is different from the person responsible for observing, orienting, and deciding, the slower the OODA loop, and the more time is required to generate insight. 
• Workshops with the stakeholders. Often people doing the analysis don't have sufficient context of the project to find interesting results in the data. They just don't know what is interesting. The fix for this is for stakeholders and the analysts to work together to provide the context during the analysis process (e.g., using ideation workshops).

Creating the reporting

The finding of meaning continues during the process of working out how to present and communicate the data. This again relates to the OODA loop. As you work out what the data means and find the best way to communicate the data, you inevitably gain additional insights and this feeds back into the analysis process. In particular, the processes of creating recommendation pyramids and choosing visualizations can often lead to additional insights.