Bad marketing is worse than no marketing. The same holds true for market research. Learn from the mistakes made in a recent sex toy survey so your own research will fare much better AND give you worthwhile data.
Alternative blog post title: Your sex toy survey sucks and here’s why.
Last month I came across a sex toy survey concerning vibrator usage and preferences from a well-known Californian sex toy retailer. They’d announced it on Twitter and provided the link to Survey Monkey.
I won’t bore you with the details, but while no doubt well-intentioned, it was badly executed: full of the errors that will mean nothing to anyone untrained, but well known to even the newest of trainee researchers starting out their MR career.
With years spent at the marketing research coalface, seeing this made me…. weep? No, that would be overly dramatic. I just got frustrated. And for the record, after I’d looked at the survey I reached out to the retailer on Twitter asking them if they wanted to know how their data could have been improved with just a few tweaks. They never replied. Fair enough.
WHY IT SUCKED
In a nutshell, there was no introductory message, no thank you for participating, no explanation of how the results were going to be used. In short, no building up of rapport AT ALL. None, zip, nada.
And given how cack-handed they started, it was pretty obvious it wasn’t going to get better. There were the ropey response options to consider. One question asked about frequency of vibrator usage. The options included such precise terms as ‘often’.
Tell me sports fans, what is ‘often’? Once an hour, once a day, a week or perhaps a month? You tell me. And I’ll bet your definition will vary from the next person.
It’s a relative term and open to interpretation.
How many lingerie sales do you think you’d get if you just told your customers: “Buy this bra, it’s for curvy women,” without giving them any size and cup dimensions.
Or, more appropriately, given the nature of the retailer’s wares on sale, how would you feel if you were told “Ah, here’s the dildo you want if you really want filled up.”
First things first, we’re not all identically sized (apologies if you did indeed attend biology class at school). Then, it’s again relative as one person may feel….ahem…filled with a different number of inches (length and circumference) from the next person.
What would a pilot on final approach in fog say if Air Traffic Control told him his required altitude was “just a bit more now” ?
What if you asked your bank manager what your balance was, and they replied “Well, it’s in the region of this much.”
Do I need to go on? Be precise, dammit!
WATCH OUT FOR THE SEX TOY SURVEY JU JU MEN AND WOMEN!
Or in other words, beware of voodoo polling. Yes, the term is well known in the research world. And has been for decades.
When a survey is sent out willy-nilly with no control over who completes it (‘self-selected’ surveys), then it’s exposed to all sorts of issues. Think of those little poll graphics you see in your newspaper over BREXIT; dog fouling on the streets; closing of bingo halls, you name it. Do you really think they’re at all accurate or representative? Think again. They’re completed by those who are obsessed by the subject, or have an axe to grind. Anything but a representative sample of the local population.
The free Survey Monkey plan allows for IP tracking and email address gathering. But are you really telling me that the creator of this rank amateur sex toy survey is really going to give a damn about who is participating? At best it could be people who have no interest or desire in your wares or indeed your company. At worst, it could be a business rival doing their best to stiff the competition any which way. In this case, by having them think they’re getting ‘valuable insight’. It’s not hard to create a bunch of fake email addresses or get some mates to all fill in a few questionnaires, with orders to respond as randomly as possible, so there’s no sense at all to be made of their responses. Result? Once again, dodgy data.
And that’s before I even mention the stats stuff.
THE STATS STUFF
Survey Monkey’s free survey plan allows for 100 responses. Consider how many people use vibrators in the same locale, let alone the world (we have to assume other people from who knows where also filled in the survey, given there are no quotas or filters on the free plan).
So, in a nutshell there’s a badly designed survey, with subsequent data that could be fairly described as ‘potentially worthless’. Add to this that there’s only one hundred responses, and no controls over who completes it.
Who thinks that one hundred people’s opinions – especially given the absence of any screening – is representative of sex toy users in that locale? And then there’s our old faithful stats friend – margin of error – which alone makes a sample size of 100 hardly worth the time and effort writing and launching the survey.
All in all this survey was a bit of a joke. Thank the stars that when I checked last night, it was no longer up.
*** UPDATE 30/5/17: The survey results were revealed and apparently 375 responses were obtained. While the number of completions is healthier, the complete lack of any respondent screening still makes it pretty worthless from a business and statistical perspective.
If you give a damn about your business to the last penny, then why be so godawfully lacking in detail when it comes to your sex toy survey research? Or maybe many lingerie or adult retailing businesses are just more of a side-hobby, rather than a real bona-fide business where people’s livelihoods are on the line if things aren’t going well? You tell me.
DON’T BLAME THE MONKEY.
Now when all said and done, don’t think for one minute that I’m giving Survey Monkey a hard time in all of this.
Far from it.
The plethora of online survey applications now available is great for small and growing adult retailers wishing to cost-effectively conduct a sex toy survey, ideally among their own customer base with already possessed email addresses that have been verified and are good to go. It’s a far cry from the days when you’d have to commission an agency to do the fieldwork for you.
So I’m not – and I don’t recommend you do, either – shooting the survey apps. Let’s get one thing straight: if the questions are atrocious to begin with, it’s a lost cause. Or in cruder parlance: “Put sh*t in, get sh*t out.”
When splurging out a little for the paid Plans from Survey Monkey and the like, there’s a lot to be said for them. You’ll get higher response limits and some decent bells and whistles included at all stages of the survey process. Add to the equation some well thought out – and written – questions (arising from proper research objectives and specified information needs) and it’s a powerful combination indeed that can really help your business. Trust me on this.
Sadly, crap surveys are something I see time and time again. Why do people think that just because the survey app is available AND they can ask some questions, there’s going to be worthwhile usable data?
Using the same logic, how likely is it that if I am given some paint and a brush I’m going to be immediately capable of creating something like the ceiling of the Sistine Chapel? I’ve been given the same tools as Michelangelo, so surely there’s no problem, right?
There’s just one. I can’t paint for toffee. Even if my life depended on it. Entrusting me with a tin of Dulux, a brush and a bare wall – or ceiling – is a pretty risky move.
The same principle applies to those given access to online survey platforms. Those untrained in marketing research can write questions in the same way that I can hold a paint brush. The tools – namely the brush, the paint – and the online survey platform – are all there. There’s just one thing missing – the knowhow.
LASCIVIOUS MARKETING TO THE RESCUE!
I couldn’t just leave you hanging there after dismembering an awful survey, could I ?
Of course not.
So, let’s learn something from all of this, using a real-life example I found many years ago on a similarly depressing website visitor survey on a UK-based adult retailing website.
Do you know if your partner owns a vibrator?
Those who responded ‘Yes’ were then immediately asked further questions concerning vibrators. So what’s wrong with it?
It’s misleading. Think about the subsequent findings:
“X% of those questioned know if their partner owns a vibrator or not.”
It’s hard to believe the purpose of the question was to really ask respondents if they know whether their partner owns a vibrator. The question in this guise is asking for respondents’ knowledge, rather than ownership per se – the latter of which is much more important to retailers.
While the question writer most likely wanted the survey to be friendly and engaging, in doing so they potentially ruined the data.
You MUST ensure questions are without ambiguity, otherwise the survey – and the subsequent data – is now open to response error, and being rendered useless for the purpose it was designed for.
The question should therefore be phrased simply as follows:-
Does your partner own a vibrator?
Simples. All bases covered.
BUT THERE’S MORE…
Consider possible respondent groups and sub-groups when designing your questions. Give serious consideration to how the research can be used to identify different respondent segments.
Let’s consider the question above once more. Rather than simply asking a ‘Yes/No’ question, why not offer several response options allowing respondents to answer the question, AND potentially identify some new user segments? Change the question to:-
How many vibrators does your partner own?
5 or more
By rewording the question and providing these options, the original question is still being answered, but now there is the opportunity to gain much greater insight. At the analysis stage the responses can be netted to provide (for instance) the following ownership segments: None, Light (1-2), Medium (3-4), and Heavy (5 or more).
[These numbers are illustrative, by the way. Feel free to change them to something more representative if your own customer knowledge suggests otherwise!]
Can you see how invaluable this is, especially when analysing the rest of the data? You can now look at responses to all the other questions based on levels of vibrator ownership, and see whether any key findings are found along these lines. How do heavy owners/users differ from light ones? The data could be extremely revealing, depending on what other questions make up the survey.
This is one of the key reasons for thinking carefully about the questionnaire design process and not rushing things. The question needs to be phrased properly to avoid response error and confusion and so on, but consideration must also be given to how the responses can be used to add to the analysis.
Remember to get the most bang for your buck when it comes to your research!
PS: I’ve got some more things to say on adult retailing market research in my recent interview with European adult retail publication EAN. And watch out for more adult retailing research matters coming soon!
And of course, if you need EXPERT help with your own market or customer research requirements, you know who to contact!