By Geoff Thacker

You already know that your association wouldn’t exist without its members. In fact, you probably do a lot to consult members as part of your planning activities to make sure your direction and priorities are in-line with their needs.

Like you, most associations strive to do at least some form of member consultation, ranging from casual discussions to in-depth interviews, town-halls and, most often, member surveys. It is this last of these member feedback channels I want to explore.

With low-cost survey tools like SurveyMonkey at their fingertips and ever-present budgetary pressures, it is easy to see why some associations are foregoing market research expertise and going the DIY route where their member needs surveys are concerned.

If you are a market researcher, you heard it, and if you’re not, you’ve likely said it: “We can do it on SurveyMonkey for much less!”  As a market researcher by trade, I 100% agree with this statement… sort of.  What gets overlooked is that SurveyMonkey is just a tool to collect the data.  Someone still has to design and program the survey, administer the communications and analysis and report on the results.  The true cost of the DIY survey is hard to measure because the value of the considerable staff time is hidden in salary costs on the income statement.

Taking this a step further, the analytic toolkit on many low cost DIY systems is fairly basic.  They provide charts, crosstabs and core descriptive statistics such as mean and median but they don’t really allow you to clean or manipulate the data, perform any advanced analysis or really dig beyond the surface. I have had several clients over the past several years come to me for help after having done their membership study internally to make sense of the data and help to turn it into something usable.

Don’t get me wrong, there is a time and a place where DIY research appropriate, and even recommended….but with your mission critical member research such as the annual member needs survey? Well, that’s just plain bananas!

Here are a few reasons why:

Survey Design: Anybody can ask a question. A market researcher with expertise in member needs and satisfaction knows which questions will get at the data you need to make informed decisions in your subsequent planning activities.

Bias: Avoiding leading questions and bias can be extremely difficult when the survey is designed by someone who works or volunteers for your organization – even those with the best intentions have a hard time avoiding at least some level of bias.

Analysis: While there are certainly some exceptions, most associations simply do not invest in advanced analysis programs such as SPSS or SAS. These tools, which most professional market researchers have, allow you to  slice and dice the data to dig beyond the surface so that you can really understand what the research means for your future direction and priorities.

Benchmarking: When you do member research internally, it can be hard to know whether your results are poor, satisfactory, or excellent because, often, you have nothing to compare it against. In cases where the researcher has conducted other similar projects (we have conducted member needs studies for more than 55 associations), your results can be benchmarked against similar organizations so that you can really see how you measure up across key areas.

So is there a role for SurveyMonkey or other similar tools in your association? Of course! They are low-cost and easy to use and can be great for things like periodic polling and for getting feedback on very specific topics.  SurveyMonkey is a great tool…but don’t forget that it is just that:  a tool. When you hire a professional market researcher to do you survey, you are not paying for the tool, you are paying for professionals with deep expertise in research design, statistical analysis and data interpretation to make sure you are collecting the right information.  Information that you can rely on and help you understand what it means in setting the direction and priorities for your organization.

Your association wouldn’t exist without its members, so you had better make sure you are making decisions based on reliable information and analysis when it comes to member needs, satisfaction and engagement – this is one area where it may be best to consult the experts.

Geoff Thacker is Executive Partner – Research and Strategy at The Portage Group.