Survey Statistics - it's all a numbers game

The double edged sword of surveys : if done right, they provide a wealth of information to take a data informed decision but come with the downside of going down a rabbit hole if not executed correctly.

If you are looking for quantitative research, relying on a collective inputs of mass to arrive at informed decisions, look no further. The benefits of surveys far outweigh the drawbacks (which can be easily averted following a systematic and unbiased approach).

There are a lot of aspects of surveys which need to be considered at design stage - from inherent biases to overcome to type of questions to consider. Survey tools have evolved drastically and provide a plethora of question formats to select. So what's the best type of question to select ? 

• Objective : What hypothesis is the question trying to validate ? Every question has to have a motive on why it's being asked. If there isn't any direct linkage to objectives, it's probably not worth asking the question (a compact survey has more chances of completion)
• Analysis : The question format depends on how the answers are going to be analysed. Are the answers going to be used in correlation analysis ? Or are they aiding in structured text analysis ?
• Spectrum : For polarity related questions it is always suggested to provide bipolar ordinal scale with equally spaced response options.

Let's consider this question : 'To what extent do you agree with moderation process?'  

Options on a ranking scale start from 1 (Agree) to 5 (Strongly Agree)

This is a biased unipolar nominal question - there is an underlying false assumption that all users agrees with the process and the gauge it to what extent.

Instead, the question should be : 'To what extent do you agree/disagree with our moderation process ?'

Options on a ranking scale start from 1(Strongly Disagree) 3(Neutral) to 5(Strongly Agree)

Now its an unbiased bipolar nominal question - a much better alternative.

Majority of the survey tools do provide basic chart/graphs related to the responses. However to get the most of the responses, a little bit of statistical analysis provides loads of valuable insights.

• Contrary to what the survey tools display, median is a better gauge for ordinal data than mean
• Of course mode is the only way to measure nominal data
• Correlation analysis between questions (scatter graphs for visuals) yields valuable insights on interdependences in the data set
• Last but not the least, almost types of research data aids in identifying clusters which then translates to segmentation

Clustering is a very powerful method to segmenting target audience or even identifying different personas. It takes correlation to the next level by grouping data sets which exhibit common functionality within the cluster but distinct from the other clusters.

In a nutshell, an innocuous looking survey question holds the potential to provide a wealth of information - if only we can see the magic in the numbers.