Is SurveyMonkey really anonymous?

How to create a truly anonymous survey?

If a survey is done truly anonymously, the respondent should be unable to identify himself or herself. There are lots of different ways to anonymise data and there is no fixed way to do it. What works well for one person may not work so well for others.

You can use various steps to obtain anonymity. Only provide answers to a predefined list of categories (for example: sex, age, weight). These predefined categories will probably reveal too much information and you cannot have enough data when only providing answers to these categories. So you would need lots of questions if only you want to get very basic information from people who only see a questionnaire with sex and weight in it. But if you can get more out of the responses, you might want to consider other options.

But before I go into that, I'll go into what anonymous questionnaire is. What does anonymous mean? To be able to compare results and get good results, the respondents have to be able to see how they responded to a survey. For that, you need their name and email address.

So we say anonymous surveys if it is impossible for anyone to know your identity. Add additional questions on top of the ones from step 1. Add a question about your gender, a question about the number of children you have, a question about the highest grade you have obtained, a question about the current age and so on. Use data analysis and find out which answer to which category. There is no way to hide all info, but with the data analysis you can get more information out of the data than using the simple categories of sex and weight. A nice example of this is a study by David Moolina. (). He shows that you can find out a lot from a simple set of questions. Ask permission to use any form of communication, when you would like to contact the respondents for a follow up. You can send an email or a postal mail containing a URL where the respondents can log in to a short survey. Or you could even send a mail with a questionnaire.

Is SurveyMonkey really anonymous?

In a recent column, we took at look at how we analyze survey results and came across some surprises.

For example, when you're trying to figure out the right number of respondents to a poll, you need to make a decision on whether to exclude respondents who don't complete the survey or weight the sample based on their level of completion. But what if you don't know how many people started the survey but then didn't finish it? If the average of this sample was used for the rest of the analysis, were respondents excluded because they didn't complete it or were their results weighted since some respondents left early?

In practice, it's not that simple. We often have to decide if this one survey isn't completely representative or we have to be cautious about extrapolating it. So how do we know for sure? The short answer is that we just don't really know for sure.

In a classic case of "I think, therefore I am," we tend to think that if we're anonymous, we're actually anonymous. However, this is far from the truth. People are not perfectly anonymous, which can result in unintended consequences when the survey doesn't ask about things we should expect certain groups of people to be more comfortable talking about than others.

When asked to keep information from the government, a majority of Americans will do so (as of 2004). But most of them won't necessarily answer these questions with the complete amount of honesty they can muster. When they're aware that they're being monitored by the government, those who choose to tell the government everything about themselves seem to do it in a way that allows them to conceal their true motives. In essence, they're answering a survey in a way that hides their true reasons behind their answers.

It's for this reason that the way a question is answered can tell a story about a person that's not always accurate. For example, many survey developers use weighted and unweighted samples in their analyses. There's no way to tell which group of people answered the survey, making us worry that different groups of people may be answering in different ways based on the way the survey was asked.

If we know who answered the survey, period. But we don't know who answered the survey and then told the government everything that was asked about the respondent or who answered the survey and then concealed their true motives.