Online surveys in market research have grown exponentially over the past few years. This approach has proven useful in helping researchers explore certain subject matter and collect automated data from different participant sources – increasing overall productivity.
Although conducting surveys is a valuable and legitimate method of data-gathering, like all methods, it still has its flaws. The most glaring of those flaws is the potential for survey fraud.
Survey fraud comes in different forms, but its purpose remains the same. It involves nefarious individuals exploiting weaknesses in the survey distribution process to pilfer incentives. Disingenuous responses can lead to skewed results and compromise data validity. This issue is truly worrisome for researchers because of the havoc it could create on their results and, in turn, their client’s business.
Presenting fake market research can produce many problems such as misguided business decisions, wasted survey costs, lost clients, and lost projects. Fortunately, there are ways to detect and prevent these outcomes from happening to your study.
Detect fraud
Fraudulent survey submissions stem from either manual or automated sources. People could manually enter responses and complete surveys themselves or use software that automates these actions for them. Here are a few things that you should search for to detect fraudulent survey respondents.
Inconsistent responses
To determine if submissions are legitimate or fake, go over your surveys for any inconsistencies in responses. Disingenuous participants are usually in a hurry when answering questions, leading them to resort to some common practices that you could look out for when checking entries.
- Speeding: Refers to participants that respond too fast to questions without giving much thought to their answers.
- Straight-lining: When the answers provided do not vary in a battery of questions, creating a predictable pattern (e.g., straight, zigzag, alternating). This practice is commonly observed in rating scales.
- Skipping questions: Only answering required questions while ignoring conditional questions indicates a lack of interest in providing authentic responses.
- Null answers: Giving extremely short responses that offer no valuable input suggests disinterest in the survey's subject matter. It could be invalid strings (e.g., asdf2!jk), a copy-pasted version of the question, or a nonsense answer.
Personal information
Some of the most significant contributors to low data quality in fake market research are bots and serial submitters. If your survey contains any personally identifiable information, do a cross-check and be mindful of these indicators to determine the validity of your sample.
- Similar credentials: A duplicate email address, username, or password between supposedly ‘different’ participants is a red flag that suggests multiple response submissions for more incentives.
- Fake data: illegitimate or inaccurate data such as phone numbers or addresses suggests fraudulent participation or bots.
Computer information
Determine respondent eligibility and authenticity by checking their digital footprint. This is comprised of information that online users actively or passively share when using the internet. Additionally, your survey URL can also be used to avoid unwanted entries.
- Cookies: You can detect multiple access attempts and track survey progress or completion by tracking cookies on your survey.
- IP Addresses: Every machine connected to the internet has a unique IP address. Check whether submissions have the same IP address or if the address was encrypted using VPN software.
- Survey URL: Check if your survey is reaching your target demographic by tracking its URL location on the internet.
Prevent fraud
Survey fraud is not only isolated to one stage of data collection. It can also occur before or after the survey has been distributed. To obtain the most valid responses, it’s best to implement preventative strategies to filter out ineligible and fraudulent participants.
Pre-data collection
- Single-use links
- Single-use or personalized survey URLs can be sent to your respondents so that when they click the link and submit their response, the survey cannot be accessed again from the same email address or link.
- Limited screening criteria
- Not clarifying screening criteria may seem counterproductive, but it is actually effective in warding off participants that pretend to fit into your qualifications. Fraudsters will have a more challenging time getting into the study because they don’t know the recruitment criteria.
- Having identical responses in two different attempts with only one differing answer in the eligibility screening can suggest people are trying to get through.
- Data validation
- Another method to ensure that your data is legitimate is to include data validating fields in your survey. These fields collect information such as zip codes, phone numbers, and email addresses that are then cross-referenced for authenticity.
- Informed consent
- Divide your informed consent form into separate sections, randomize consent options, and provide them as web pages at the start of your survey. This process requires your respondents to be more thoughtful of what they are clicking, helping to reduce bots and discouraging frauds from participating because they may find that it’s not worth their time.
Data collection
- CAPTCHA
- Ensure that your participants are not bots by including CAPTCHA (Completely Automated Public Turing test to tell Computers and Humans Apart) codes in your survey. CAPTCHA codes consist of a distorted combination of letters and numbers that malicious software cannot copy, preventing bots from accessing your study.
- Repeating questions
- Including similar questions throughout your survey can help reveal any inconsistencies in answers. Placing questions such as ‘what year were you born?’ and ‘how old are you?’ at the beginning and end of the questionnaire can help you distinguish unqualified survey takers and responses. If the answers match, that indicates a valid respondent. Mismatched answers, on the other hand, show fraudulent participation.
- Effort checks
- If you want to know if people are paying attention while completing your survey, effort checks could be beneficial. This strategy consists of placing unrealistic questions (e.g. ‘Did you win a million dollars in the past week?’) in your survey to see if participants really took the time to read and provide genuine responses.
- Attention levels could also be indicated by setting survey progress and completion time parameters. For example, you could exclude participants from the study if they only answered one-third of the questionnaire or took 15 minutes to complete a survey that would typically take 30 to 40 minutes to finish.
- Metadata
- With participant consent, you could track cookies and IP addresses to ensure that every submission is unique to each survey taker. A cookie could be assigned to each respondent when they open the survey link to prevent them from accessing it again. IP addresses could also be recorded with survey results to block submissions that share the same address.
Post-data collection
- Pattern analysis
- After survey results have been recorded, a thorough analysis of each submission could be executed to identify patterns in responses, metadata, and respondent information. This process could be useful in verifying if survey results are consistent and unique throughout the sample.
- Exclusion approach
- Fraudulent submissions could be flagged and discarded or redirected for separate analysis, with each invalid response categorized by duplicates, bots, or both. This strategy could effectively identify areas in your survey that you could improve upon for better results.
The Best Solution to Prevent Fake Market Research
Employing a service with expertise in respondent recruitment ensures that your survey is safe from any fraudulent participation. Civicom’s dedicated recruitment platform, CiviSelect™, has the knowledge, expertise, and global reach to provide you with the most suitable survey participants for a successful market research project.