CX Survey Design Series: Which Scale To Use?

 

By George Polyard

Download
person taking a survey

Introduction

 

Customer and employee experience programs exist to drive action, improve experiences, and optimize performance. When objective customer data is obtained, it can be useful to drive these results. Most customer experience (CX) and employee experience (EX) programs start with a survey. Surveys remain an important part of any successful CX program.

When designing a survey there are many decisions to make, the first of which is what questions to ask. Next, one must decide on the type of question to use. Questions may be asked in various ways, open text, ranked order, or a scale-based question, for example. This article will focus on the latter, as they are most prevalent and result in quantitative data. Even within the broad category of scale-based questions, there are various options, from thumbs-up/down graphic scale to an 11-point numeric scale, and they may yield different results.

What is a scale & why use one? 

 

Survey questions typically fall into two broad categories: open and closed questions. Open-ended questions ask respondents to share their feedback in an unstructured manner, typically in the form of text, video, or audio. Often these questions may sound like, “What was the primary reason for your score?” In contrast, closed questions place constraints on what answers the respondent can choose. For example, “Did you like your hotel room? Yes or No”. Closed questions require a set of choices, or scales, to be presented to the respondent. Scales are typically intuitive and have a uniform range (e.g. 0-10, unhappy to happy.)

Scales are used to find objective and quantitative data. Scales are useful because, as an alternative to open-ended questions, they provide a defined range that the respondent can choose. How a single respondent answers a scale-based question is somewhat useful. However, where scales have more value is when there is a large response volume.

Types of scales 

 

Dichotomous Scales

The simplest of scales comes with the most threatening name: dichotomous scales. This scale provides just two options, typically opposite. For example, “true/false” or “yes/no”. The advantage of a dichotomous scale is that there is no ambiguity in the resulting data. For example, a casino survey may ask the question, “Did you find your favorite slot machine? – YES or NO”. The resulting data will be especially clean and digestible, 75% of respondents say YES, and 25% say no, for example. However, if the same scale were applied to a broader question, such as “Were you satisfied with your recent hotel stay? YES or NO”, the resulting data of 75% YES and 25% NO may not be as valuable, as a hotel guest’s satisfaction is not binary, and a guest may have a neutral experience.

survey example

Generally, dichotomous scales are best used with two opposite options and when a respondent cannot be neutral. Dichotomous scales are precise, but they lack the granularity found in scales with more options. Additionally, dichotomous scales do not provide a neutral or unsure option, which may contribute to some data being inaccurately answered by the respondent.

Rating Scales

Rating scales, sometimes called ordinal scales, are a step up from dichotomous scales, in that they provide three or more options. These options may be presented verbally, graphically, numerically, or a combination. Rating scales require more interpretation than dichotomous scales and the resulting data may have a calculation applied, such as a Net Promoter Score calculation. (The topic of calculations will be covered in a separate article in this series.)

Verbal & Numeric Rating Scales

Verbal scales, including the popular Likert scale, provide the respondent with multiple word-based categories to choose from, such as, “Strongly Agree” and “Agree”. Verbal scales do not contain any numbers or graphics. Verbal scales are often used in market research-style studies and less so in customer experience surveys. Unbeknownst to the respondents, verbal scales typically have a value associated with each response. For example, though the respondents only see the category name, such as “Disagree”, though this would correlate with a numeric value, such as 1.

verbal survey example

Numeric scales are like verbal scales, though they present the respondent with numbers, such as 0-10, instead of verbal categories. Numeric scales, in addition to graphic, both are considered advantageous, as they are applicable across many languages.

numeric survey example

Verbal and numeric scales have many similarities. For the practitioner, they are easy to implement, many types of questions can easily have a verbal or numeric scale applied. Additionally, for respondents, they are typically easy to understand and contribute to surveys being completed quickly.

Verbal and numeric scales are commonplace in surveys because they provide immediate objective and quantitative data. Both scales have a numeric value attached to each response, therefore the resulting data can be calculated easily, including median, mean, and count. By contrast, in graphic scales, this is more difficult.

Graphic Rating Scale

Graphic scales, a form of non-verbal scales, provide the respondent with multiple graphics to choose from, such as smiley faces or stars. Graphic scales are commonplace on websites, such as Amazon.com reviews or the thumbs-up/down rating on hotel booking websites. A significant advantage of graphic scales is that they are applicable to all languages and do not require translation. Graphic scales are also generally easy to understand and construct. However, the resulting data is more difficult to interpret than verbal and numeric scales. Furthermore, graphic scales are typically only applicable on web-based surveys and not useful in SMS-based surveys.

Rating scale ranges 

 

Rating scales may contain various ranges, from as few as three options to as many as 11. For example, you could ask, “How would you rate your satisfaction? High/Med/Low?”. Rating scale ranges are generally presented in odd numbers, which allow for the respondent to select a neutral selection. However, if there is a question where a midpoint answer is not appropriate, using an even number may be appropriate.

The number of options to include impacts the resulting data. If a survey only uses three options, all responses will fall into those buckets, potentially limiting the resulting data. Seven-point or 11-point scales provide the most granularity.

Conclusion

 

CX programs set out to drive action, optimize performance, and improve experiences. Surveys are a foundational part of most CX programs and their design characteristics, including what scale to use, have impacts on both the response rate and the resulting data. Different scales impact both components and should be carefully considered.

LET'S GET TO WORK