- Make Sure the Proposal Responds to Your Objectives
The proposal process begins before the research firm offers you their take on how they recommend you conduct your survey and for what price. Instead, the process begins with the first discussion you have about the survey. Did the researcher take the opportunity to ask you specific questions about your objectives, the group of people you’d like to survey, and your ultimate goals? Details regarding your situation should pop up throughout the proposal. Surveys whose proposals do not incorporate your individual needs cannot possibly meet them.
For example, one research firm may recommend a very straightforward survey plan with a low price tag. Another may take time to recommend a more involved approach that considers your needs and objectives but at a higher price.
Ask the higher priced firm to account for the differences. They should have no problem explaining the rationale behind their recommendations. On the other hand, have the low-cost provider explain why they believe such a straightforward survey is right for you.
- Sampling Plan
When reviewing the sampling plan, make sure the proposal mentions sample size, response rate, number of responses, and maximum sampling error. These figures help you determine whether you’ll be able to confidently project the survey results to your entire population of interest. If you’re unsure of the impact these figures have on the quality of your results, ask the researcher. They should be able to explain them in terms you can understand.
If you’re interested in learning details about specific segments of your circulation, make sure that the sampling plan accounts for them. Sometimes a simple random selection of names is enough. In other cases small, but important groups should be over-sampled in order to collect enough responses to tell a story about the group.
- Questionnaire
The quantity and types of information sought from respondents will impact cost. Quantity encompasses the number of questionnaire pages and number of variables to process. Type refers to how the questions will be processed, the data entry involved, and whether all or just some data will be cleaned.
No evaluation is complete until you know the approximate number and types of questions planned for the survey. The number of open-ended questions should be included as well because open-ended questions that capture verbatim responses can impact the response rate and the price of your survey. While these details can change during design, knowing the starting point helps establish what additional questions, pages, or transcribed questions will add to your bottom-line.
In addition, make sure the proposal clearly indicates who will develop the questionnaire content. Also, determine if it includes enough collaboration time to be sufficiently customized to meet your particular needs.
- Data Collection Approach
For online surveys that invite respondents via email to respond to a web-based survey, paying attention to the data collection series can mean the difference between conducting a successful survey and one that frustrates your circulation. Multiple emails to respondents can encourage response by arriving in their inbox throughout the day. Some of the times the emails arrive will be more convenient for your sample than others. However, you should only send follow-up emails to non-respondents. Out of privacy concerns, sample members should have the opportunity to opt-out with each contact. Outbound emails must also be coded to only allow one response per person, and to prevent others from taking the survey.
Proposals for mailed surveys should clearly outline the data collection series and each component of the survey kit. A sophisticated mailing series can efficiently improve response rates and increase the quality of data. Some cost-effective techniques that can boost response rates include the use of incentives, stamped reply envelopes, follow-up survey kits to non-respondents, alert letters or postcards, and personalization.
- Data Processing
Your proposal should highlight the steps the research company will take to make sure that the data is accurate and representative. Depending on the type of survey, checking logic, consistency, and outliers can take a significant amount of time. You must have some process noted to identify inconsistent answers for surveys that collect a significant amount of numerical data (salary survey, market studies, budget planning). Finally, some percentage of mailed surveys need to be verified for data entry accuracy.
- Analysis
A straightforward analysis of survey data can meet many objectives. In other cases, a multivariate statistical analysis will provide deeper insights to achieve your objectives— making results easier to use. If your objectives include learning about separate segments of your circulation, cross tabulations should be specified.
- Deliverables
A variety of reporting options exist for a survey. These include but are not limited to data tables, a summary of the results, in-depth analysis, and graphed presentations. As a result, you need to understand exactly what you’ll receive following your survey and in what format.
If the report and data table samples aren’t offered, ask for them. You want to make sure that data tables are easy to read and attractive. Consider how well write-ups enhance the clarity and usability of the results. If you plan to use your reports in presentations, make sure they’ll reflect well on you. Don’t forget to consider the time you’ll have to take to reformat the results.
- No Surprises!
Make sure the proposal covers all the bases: what you need to do and provide, what the firm will do when they will do it, and how much it will cost.
There should be no surprises in what you need to supply. No “you need how much letterhead and envelopes?” a week before your survey is scheduled to mail.
Review the price carefully and understand what it includes and doesn’t include. Avoid the “you want how much to report the results?” after your survey is complete.
Proposals with specific timelines will give you an idea of how long each step of the survey takes and illustrate the total time necessary to complete the project. If you have a deadline of when you need results, make sure it’s addressed in the proposal. The annual meeting to go over survey results will be much less engaging if the reports won’t be ready in time.
As with many things, you usually get what you pay for from a research provider. You’ll almost always be able to find a firm that says they can do the same survey quicker and cheaper than another.
One thought on “Evaluating a Research Proposal”