What a survey is
Most people have many opportunities to experience the joys of filling out a survey. They come in your mail, pop up in your email, and are asked by phone (especially in the months leading up to an election!). Lots of people want to know your opinion, and surveys are a great way of gathering the perspectives of a number of people. If you gather enough surveys in a scientific way, you may be able to make generalizations about the opinions and perspectives of a larger population. A smaller sample will still yield an overview of the opinions and perspectives of those who returned the survey. If you want to know the basics of how people stand on an issue, how they feel or act, a survey can be an efficient and effective way of gathering data. As respondents provide data to describe themselves, analysis can also reveal if there are differences in perspectives, impact, actions, etc. between different groups of respondents.

What a survey isn't
Surveys are often not so good at capturing much depth in people's perspectives or thoughts. While they may offer an opportunity for respondents to write in "long answers", the bulk of a survey usually asks for a short answer or a response selected from a list of options. Surveys also rarely offer much opportunity for respondents to shape the content; you are responding to what a researcher and the client organization has determined is important to study.

As with other kinds of research you may be asked to participate in, surveys should also be transparent. You should always know who is doing the survey and how the results will be used.

What is the value of a survey to the participant?
A survey can be a great opportunity to get your opinion heard. If you do take the time to RETURN YOUR SURVEY to your caring and interested researcher (who only wants the best for you, by the way!), your opinion will be used to shape the research findings. The greater the percent of those who respond, the more valid the picture formed by the survey findings. Survey findings can be used to change programs, continue them as is, or stop them entirely. Surveys require an investment by the organization, so you can usually trust that the results will really be used.

While you are completing the survey, you can reflect on your opinions and experiences. Perhaps you will see new themes in your answers that you had not noticed before. Perhaps you will clarify your thoughts or have a happy reminder of a good experience. Or, perhaps you will see that what you found important about a program or experience was actually not its intended purpose. That's kind of interesting, don't you think?? You should also be able to see the final report of the survey results, so you can see how others felt and how the results were used. Researchers usually welcome requests for more information or the chance to share how the information will be used.

The census (http://www.census.gov/main/www/cen2000.html) is the most familiar survey to many people. But, if you are interested in seeing other examples of how surveys have been used to help understand public opinion about a variety of issues, check out these links (copy and paste into your browser):
http://www.ala.org/ala/acrl/acrlpubs/crlnews/backissues2006/october06/opinionpoll.cfm
http://www.csus.edu/indiv/n/nalderk/Public%20Opinion%20Websites.htm
http://www.library.cornell.edu/olinuris/ref/pollres.html#online.

Posted by Guest author at 01:06 PM | Permalink | Comments (0) | TrackBacks (0)

In our evaluation work at the Improve Group, we regularly collect data from stakeholders. We use a comprehensive approach to evaluation and often collect many different types of data for a single project. If you’ve read our Participant Primer Series, you’ve learned about the different methods the Improve Group uses while working with human subjects and the data they provide. You may now be wondering what we actually we do with the data after we collect it from stakeholders.

After the data is collected, we compile it to help inform our reporting, using quantitative and qualitative analysis. Earlier this month, Leah wrote about the use of quantitative versus qualitative data. In brief, quantitative analysis is used for numerical data, such as survey items where we ask participants to rate items on a scale from one to ten. Qualitative analysis is used for any type of free-response question. We collect this data through interviews, focus groups, open-ended responses on a survey and observations.

I find quantitative analysis to be more of a straightforward, but rigid process. We decide ahead of time what associations we will look for in the data and then run analyses using software. We use analytical software packages such as SPSS, Access and Excel, depending on the complexity of our data. The findings are used to make charts and graphs to provide stakeholders with a concise picture of our findings.

There is more room for creativity and flexibility with qualitative analysis. However, it is important to analyze this data systematically as you would any other data source. A systematic analysis method will also help you manage a large amount of data that may otherwise be overwhelming to tackle. We use qualitative data from interviews, focus groups, surveys and observations and often compile open-ended responses in a spreadsheet which we use to identify themes across respondents and across data sources. The findings are used to write report narratives. Unlike quantitative numbers and charts they often provide a more “human” side to the stakeholder’s story.

Quantitative and qualitative data are both valuable, and often complimentary, information collected during the evaluation process. Used together, they help us provide a more comprehensive description of findings than we would be unable to provide using just one method alone.

Posted by Guest author at 10:49 AM | Permalink | Comments (0) | TrackBacks (0)

The Improve Group has used secondary analysis in conducting quantitative analysis. Like focus groups and interviews, secondary analysis contributes to a comprehensive, in-depth understanding about the issues facing an organization and their participants.

So, the Improve Group calls you to participate in secondary analysis– what exactly are you getting yourself into?

What is secondary analysis?

Many of us have read articles about results of studies using secondary analysis. Secondary analysis uses existing data that may have been collected for previous purposes in order to examine an issue different than originally intended. The data could have been collected for determining eligibility or enrolling participants, managing a program, or for a previous study. This data can range from responses to interview questions and focus groups to individual test scores.

What secondary analysis isn’t.

Secondary analysis is not the re-evaluation of the original study. However, it can use that original study to explore new areas. The process should be straightforward and transparent, especially when dealing with sensitive information. When researchers ask for secondary data from an original data collector, they should be able to provide a detailed answer to the following questions:

• What is the purpose of the research?
• How will this data be used?
• How will the results of the research be shared?
• How will confidentiality be preserved?

In addition to answering questions, the researchers should learn as much as they can about the data, including any issues in validity or consistency of the data. Studies using secondary analysis should be well-grounded in the theories of social science. Otherwise, the validity of the results of the secondary analysis can be seriously questioned.

What is the value of secondary analysis for the participant?

Secondary analysis provides an inexpensive and convenient option in analyzing multiple sets of data that can build on the work of other researchers. Such analysis contributes greatly to answering important societal questions that affect the participant. Don’t be afraid to take existing data as a starting point to evaluate an important current issue. If you are interested in the results of a study or how a study can be used, feel free to ask questions and provide comments about the study to the researcher. Additionally, you can request a copy of the results of the secondary analysis.

Therefore, if a researcher asks you to participate in secondary analysis, you should strongly consider it!

Posted by Guest author at 09:44 AM | Permalink | Comments (0) | TrackBacks (0)

The Improve Group has conducted interviews with a wonderful variety of people for a great variety of disciplines. Interviewees (also referred to as: discussants, informants, subjects, participants or conversational partners) add critical and valuable perspectives about programs. Like focus groups, interviews contribute to a more in-depth, comprehensive understanding of the issue at hand.

So, the Improve Group calls you to participate in an interview – what exactly are you getting yourself into?

What is an interview?

Many of us have done an interview at one time or another. In some situations, it can be a scary and uncomfortable experience. One may consider a typical interview to be an in-person, one-on-one interaction where the interviewer rattles off a list of questions to the interviewee. While there are certainly some interviews that fit this scenario, researchers conduct interviews in many different ways and make efforts to put the interviewee at ease. At the Improve Group, we strive to make our interviewees comfortable and are sensitive to their diverse needs. Interviews can be in-person, over the telephone or even online with one person or a group of people. They can be formal and structured with a predetermined list of close-ended questions, or they can be unstructured “guided conversations” that are meant to elicit in-depth information. Cultural interviews, oral histories and life histories are examples of unstructured interviews.

Interviews usually have an underlying purpose or topic they are meant to address. Sometimes the interviewee is recruited based upon prior knowledge or expertise in the area of interest, while other times an interviewee is recruited without any prior knowledge of the topic to add an outside perspective. Ultimately, interviews help researchers understand how others think or feel about a topic. Interviews should take place in a comfortable setting and although the time length varies, most do not last more than an hour and a half.

What an interview isn’t.

Like focus groups, interviews should not be mysterious. Anyone asking you to do an interview should be able to give you a detailed explanation of:

• who is conducting the research;
• what the objectives of the interview are, and
• how the information you share will be used.

If sensitive information is to be shared you should be given assurances of anonymity and security. Depending upon the nature of the topic a release or consent form could be offered for you to sign. In some circumstances, especially if a direct quote is cited and referenced, the interviewee will have the opportunity to read the report that includes information from their interview to verify accuracy.

Interviewers should never be judgmental of the interviewee; the interviewer should respect your thoughts and opinions.

What is the value to you the participant?

Interviews can be an opportunity for you to talk about something that is important or of interest to you. It can be an opportunity to become involved in research on program or product improvement and to have your opinions heard and appreciated. And, sometimes you are offered an incentive to get involved, like a monetary gift, gift card or donation to a favored program or charity.

As with focus groups, there is real power in getting your ideas and concerns across to a researcher that is charged with the task of planning a course of action or recommending change.

Interviews are an excellent method of gathering rich, contextual information in many areas of research. Please consider the value you will contribute to a project by participating in an interview!

Posted by Guest author at 09:28 AM | Permalink | Comments (0) | TrackBacks (0)

The Improve Group has conducted focus groups for clients from a great variety of disciplines. Focus groups are a good way for researchers to learn about the human experience.

So, the Improve Group calls you to participate in a focus group – what exactly are you getting yourself into?

What is a Focus Group?

A focus group is commonly a meeting of a small, ad hoc group of people led by a facilitator for the purpose of exploring a topic, product or issue. A series of pre-determined questions are presented to stimulate conversation or solicit opinions and reactions from the participants. Groups meet for a designated period of time depending on the complexity of the topic (usually about 1 hour). Sessions are kept on schedule by the facilitator and everyone is given a chance to contribute.

Focus groups have become more sophisticated over the years and more creative methods are being used to gather information. Some methods include participants bringing photographs to discuss, writing in a journal or contributing to an online bulletin board or chat room. If the focus group is for a marketing firm they may offer a product for you to try out or a food to prepare and sample.

What a focus groups isn’t.

Focus groups should not be mysterious. Anyone asking you to be in a focus group should be able to give you a detailed explanation of:

• who is conducting the research;
• what the objectives of the meeting are, and
• how the information you share will be used.

If sensitive information is to be shared you should be given assurances of anonymity and security. Depending upon the nature of the topic a release form could be offered for you to sign.

Focus groups should never be judgmental; everyone’s thoughts and opinions should be heard and respected.

What is the value to you the participant?

A well-run focus group can be enjoyable and fulfilling. Sometimes you are offered an incentive to get involved, like a monetary gift, gift card or donation to a favored program or charity.

Although the incentives are a nice touch you cannot overlook the more personal incentive of being part of a focus group – the satisfaction of having your thoughts and opinions heard and appreciated. If the focus group is on a topic about which you are passionate you get the opportunity to tell others what you think and how you feel. There is real power in getting your ideas and concerns across to a group of peers or a client who is charged with the task of planning a course of action or recommending change.

Should you get the call, we hope you will believe in the importance of your opinions and consider taking part in a focus group.

Posted by admin at 03:24 PM | Permalink | Comments (0) | TrackBacks (0)

In April 2007 we will post the first installment of a series of blogs we are presenting on the roll of the participant in the research and evaluation process. This “Participant Primer” is meant to demystify the different methods the Improve Group uses while working with human subjects and the data they provide. I’ve never really liked the term “Human Subjects,” it always reminds me of looking at people under a microscope or plot lines in an old episode Star Trek. What the term really means is someone like you who may provide us with information or data pertinent to our research. We thought this series would give you a little more background about why we ask for your help and how we handle the information you contribute.

Our upcoming series of blogs will include information on:

• Focus Groups
• Interviews
• Surveys
• Observations
• Secondary Analysis

For more information about working with human subjects see a previous Inside Groove blog posting under the Evaluation category of our blog called Working with vulnerable research subjects - by Deborah Mattila.

-Staff author Susan Murphy

Posted by admin at 12:31 PM | Permalink | Comments (0) | TrackBacks (0)