Issue 13: June 2008

Featured Article: Online Surveys - Efficient and Flexible
by Marian Kimball Eichinger

With the technological age upon us, the field of evaluation has not been left untouched. International focus groups can be conducted through organized teleconference calls, interviews with people from across the country can be accomplished via video software, and programs like SPSS make data analysis possible with a few clicks of a button. Another tool that has been increasing useful in program evaluation is the online survey. Online surveys are a very practical and efficient method of collecting both quantitative and qualitative data. This article identifies some program characteristics that may attract or deter you from using online evaluation surveys, outlines some benefits of using online surveys if appropriate to the situation, and provides tips for planning and administering web-based surveys.

When determining whether to use online surveys, consider who your survey respondents will be. Do all program participants have computer internet access making this method of surveying a viable option? The Improve Group has found that it need not be an either/or situation: some participants may complete the survey online and for those who cannot, a paper survey will be made available. Another factor to consider is whether you have a viable email list to use to send your respondents the survey link. If an email list is not accessible, consider the pros and cons of creating one (time and staff availability).

One positive aspect of online surveys which reaps benefits in both time and budget is the lack of manual data entry since the respondents complete their own data entry upon hitting “submit.” In addition, administering an online survey to 100 local participants essentially costs the same as administering the same survey to 1,000 people across the country. A large scale national survey may prove to be logistically complicated when setting up a mailing list, but the actual administration of the survey is blind to population size or location. Other benefits of online surveys include flexibility in type of question; Likert-scale, multiple choice, and open-ended questions, as well as “go-to questions” can all be effectively utilized in an online survey. An additional benefit that we've observed if that online surveys provide a private environment for the respondent to complete the survey, such as in their home or office. Studies show respondents are likely to answer open-ended questions more thoughtfully and comprehensively in an online survey than on a paper survey.

The evaluation plan and survey design for online surveys needs to be as comprehensive as when using paper surveys or any other form of data collection tool: goals, research questions, outcomes and survey items need to be identified and well thought out. Free web-based survey hosts, such as Survey Monkey or Zoomerang are user-friendly and cost-efficient means to conduct, analyze and report on a simple, straightforward online survey. More complicated services and questionnaires, may require purchasing survey software such as Perseus Survey Solutions or Snap. Owning your own software allows you to host the survey on your company’s website (note: a certain level of technological expertise is required). There are a plethora of survey software programs available that offer many styles, services and interfaces at a variety of costs.

When appropriate, the Improve Group uses online surveys, especially for multi-site and school-based evaluation projects. We are finding that people are becoming more comfortable taking surveys online and appreciate their anonymity and convenience. This comfort spans age levels, regions and levels of expertise.

Some information in this article was gathered from the New Directions for Evaluation, Fall 2007 publication, Using Online Surveys in Evaluation; authors Lois A Ritter and Valerie M. Sue.

I am very pleased to be the newest graduate student intern here at The Improve Group for the summer of 2008. In the short period of time I have been here, I have discovered a group of individuals who all bring something special to the table and create one cohesive unit that really gets things done. I am excited to be a temporary member of this team, and contribute my efforts to the various projects I am assigned to throughout the summer. I am currently working on several projects that include the DHS Waiver Review, Department of Corrections Life Skills program, and have done some work with the Hispanic Scholarship Fund project as well. Program evaluation is a discipline that only recently had been brought to my attention through my graduate program at The University of Wisconsin-Stout. Originally I enrolled in the Masters of Applied Psychology with an emphasis in Industrial/Organizational Psychology. However, after sitting down with my advisor I was convinced that adding program evaluation to my curriculum would enrich my experience in the program. I am hopeful that by working with the entire staff this summer, I will be able to enrich that experience even more through gaining perspectives and advice from working professionals.

I spend a good portion of my free-time with my fiance Hilary, planning out details for our wedding (which she insists are important, and for safety’s sake I usually just agree with whatever she likes). We are getting married August 16th, and are very excited for what the future holds. I hope to have the chance to work with all of the staff here at The Improve Group at least once during my time here, and I really look forward to the different opportunities that lie around the next bend in the road.