You’ve just completed a member survey and you note that 20% of your respondents express dissatisfaction with the timing of your annual conference. Should you be concerned with that level discontent? Is it time to move the conference to another date?

This is a question that I am often asked when speaking on the topic of member surveys. What percentage of your respondents need to express dissatisfaction before the red flags go up?

My response is often an unsatisfactory, “it depends”. What you need is greater insight into who is expressing this dissatisfaction. I recommend that you add two types of questions to your next survey that will allow you to improve your insights and make better decisions.

1. Where Are They in the Member Journey?

When we conduct a member survey we often hear from a small group of committed and engaged members. While support from this group is critical they may influence the overall results and push the association in a direction that does not represent other key groups. How do you control for the possibility that a small group within the total membership may steer your association off course?

One way is to collect information that captures where the respondent is in their member journey. Are they a new member, someone who has been a member for a few years or a long-time member?

At the same time, it is helpful to know (particularly if you’re a professional association) where they are in their career.  Are they new to the profession, mid-career, late career, nearing retirement or semi-retired?  This information allows you to look at the results through the eyes of different member groups. Knowing what your newer members think, as well as those who are the start of their career, is critical because they will be responsible for the long-term viability of your organization. If dissatisfaction with the timing of conference is higher amongst this group you may have reason for concern. It may be time to dig deeper into their needs.

2. What is Their Net Promoter Score?

I would also recommend collecting information that you can use to segment your respondents into several key groups. Are they promoters (that is, are they association evangelists) or are they detractors (that is, unlikely to recommend membership)?

The net promoter score, also known as NPS, is an open-source tool that can serve as a measure of loyalty to your association. We have tweaked the usual NPS question to read as follows: “How likely is it that you would recommend our organization to a friend or colleague”?  Respondents answer by using a 0 to 10 scale. Typically, this question is followed by an open-ended question asking for reasons for a specific response. Based on the response to the first question, a respondent is classified as a promoter (a score of 9 or 10), passive (scores of 7 or 8) or a detractor (6 or below).

In our example, of dissatisfaction related to the timing of the conference, it would help to know if this is more of an issue amongst your promoters or your detractors. Given the importance of promoters to the health of your association, their views carry significant weight.

Member surveys provide an effective report card for your association and they can highlight potential areas of concern. However, as suggested in this article, you need to look at the results from the perspective of various member segments. Only then will you be able to make informed decisions and take effective action.