I was sorely tempted to highlight a questionable federal grant to study Longnose Leopard Lizard Telemetry within Colorado. Now to be fair, the full RFP has not been posted yet, so maybe distance jumped or data transmitted by reptiles with lengthy proboscis will someday be of great benefit (but I doubt it). This is not to say that innovative thinking is not a good basis for granting funds. Today I am revisiting a foundation I highlighted last January. This health funder has a great ability to seek out partnerships and promote new ideas that are of benefit in the field of health.

This week’s pick: In partnership with Robert Woods Johnson Foundation, the social solutions initiative, Changemakers, has come up with a great idea to utilize computer and video games to develop creative approaches for addressing issues in health care. This international competition will fund ideas that use video game innovations which present fun and useful education and interaction regarding health and health care.

Entries need to be submitted by 3pm EST on September 26th, 2007.You must create a user account and fill in the application form provided on the Changemakers website to enter.
Let the games begin!

Posted by admin at 04:19 PM | Permalink | Comments (0) | TrackBacks (0)

In our evaluation work at the Improve Group, we regularly collect data from stakeholders. We use a comprehensive approach to evaluation and often collect many different types of data for a single project. If you’ve read our Participant Primer Series, you’ve learned about the different methods the Improve Group uses while working with human subjects and the data they provide. You may now be wondering what we actually we do with the data after we collect it from stakeholders.

After the data is collected, we compile it to help inform our reporting, using quantitative and qualitative analysis. Earlier this month, Leah wrote about the use of quantitative versus qualitative data. In brief, quantitative analysis is used for numerical data, such as survey items where we ask participants to rate items on a scale from one to ten. Qualitative analysis is used for any type of free-response question. We collect this data through interviews, focus groups, open-ended responses on a survey and observations.

I find quantitative analysis to be more of a straightforward, but rigid process. We decide ahead of time what associations we will look for in the data and then run analyses using software. We use analytical software packages such as SPSS, Access and Excel, depending on the complexity of our data. The findings are used to make charts and graphs to provide stakeholders with a concise picture of our findings.

There is more room for creativity and flexibility with qualitative analysis. However, it is important to analyze this data systematically as you would any other data source. A systematic analysis method will also help you manage a large amount of data that may otherwise be overwhelming to tackle. We use qualitative data from interviews, focus groups, surveys and observations and often compile open-ended responses in a spreadsheet which we use to identify themes across respondents and across data sources. The findings are used to write report narratives. Unlike quantitative numbers and charts they often provide a more “human” side to the stakeholder’s story.

Quantitative and qualitative data are both valuable, and often complimentary, information collected during the evaluation process. Used together, they help us provide a more comprehensive description of findings than we would be unable to provide using just one method alone.

Posted by Guest author at 10:49 AM | Permalink | Comments (0) | TrackBacks (0)

by Susan

Okay Leah, this blog is for you. As you may have deduced from our previous blogs and newsletters, the Improve Group consists of a rabid bunch of readers. We share books and post our favorites on our internal website. You don’t need to be a whiz in evaluation to deduce that there would be a few Harry Potter fans in our midst. Like much of the literate world we have anticipated the arrival of the seventh book in the series. I consider myself to be a real “Potter Geek.” My sons and had I dressed up and attended the book party for the 6th book and so we couldn’t miss the opportunity to be present at the Wizard’s Ball at Barnes and Noble in Roseville, Minnesota for the final chapter. The photos below show my family as Professor Lupin, Mrs. Weasley and Harry Potter himself. My friend, Beth and her son, Jake, also attended as Professor Sprout and the dreaded Lord Voldermort. It is hard to believe that the first book was published in the United States in September of 1998. My sincere thanks to JK Rowling for the wonderful entertainment she’s given us to these many years.

Posted by admin at 08:43 AM | Permalink | Comments (0) | TrackBacks (0)

When gazing I am always looking for something that hits me as particularly useful and often uncommon in the funding world. I was very pleased to discover my subject for this blog. This funder is unique in that they grant money for “tangible items” or things that tend to remain with the grantee past the length of the program or initiative they may be used for. This is a rare find since most grants out there are for programming, capacity building or strategic planning.

This week’s pick: The Ludwig Family Foundation in California has committed their help to a wide variety of groups that “are working to make a positive difference in the world.” Types of tangible items they will fund are:

 New vehicles or equipment
 Equipment replacement and modernization
 Improvements to facilities
 Educational materials

Grants are made for a single year and range from $5,000 to $50,000 with applications accepted for the first of September, January, March and July. Tax exempt, 501©3 organizations within the United States or United States-based non-profits are eligible for support. For guidelines and more, click here.

Posted by admin at 04:14 PM | Permalink | Comments (0) | TrackBacks (0)

Last Thursday (July 19) we launched our new workshop series. In a session on developing good outcome indicators and tools, and another on how to conduct an evaluation as an internal evaluator, participants walked through the different stages of an evaluation and developed their own materials to use in their work. They learned how to create good surveys and focus groups, develop a logic model, and use evaluation to tell their organizations story. The picture below shows one activity in which participants used a road map to describe their evaluation goals.

Our participants said:

My learning goals were to understand the greater evaluation process and purpose, beyond just reporting what funders require – definitely met my goals!
I feel I have a much better base of knowledge in evaluation and will be using this info to improve our processes at our organization.
I learned specific logistical details about conducting evaluations.
My goals were to understanding data collection mechanisms, interpret outcome data and learn about the elements of an evaluation – all of these were met.

We will next be offering our workshops in September. If you are interested in registering, please let us know through our contact form and we will send you additional information.

Posted by admin at 09:37 AM | Permalink | Comments (0) | TrackBacks (0)

We are so appreciative of the enthusiasm our clients have when working with us -- and particularly when they pass on a good word about us to other organizations. We know how important it is to show the same support when we find a vendor or product that provides us with high-quality service. Both Susan and Leah had the chance to "pass it on" recently--in ways that were great fun!

Susan: My experience started when I called in to tell my “Cub Foods Story” in response to TV ad a couple months ago. I shop at Cub and was happy to tell them about my good experiences and shopping preferences for their grocery store; particularly as a single mom I like the great assortment of fresh fruit that is easy to give to my sons when we’re on the run to our next adventure. This month I received a call from an advertising agency asking my family to take part in a photo shoot about the Cub Stories. I took my two boys along with some clothing choices to a studio in Minneapolis. The boys got a kick out of seeing a makeup artist work on me, and they had fun posing for the photographers as part of a professional advertising shoot. You will see us Murphy’s in one of the upcoming coupon papers in your Sunday newspaper.

Leah: Like Susan, I was contacted to help market a company. We use Administaff as an external HR department, and love that we can offer our employees some of the same benefits they would get at a larger organization. We use their payroll services, and our staff also get a choice of health plans, 401K options, and access to training. We have been really impressed with Administaff, particularly because we can spend more of our time focusing on our core business (while not cheap, we calculated that the service actually saves us money based on reduced staff time needed for HR and the negotiated costs they have for insurance). Administaff's marketing department asked if I would record a radio ad for them. It was a lot of fun! I got to go to a recording studio, wear the big headphones, and read a script related to our experiences with their services. I also got to support a company that has really been a great vendor. Listen for the ad on AM radio stations starting in September.

Posted by admin at 01:03 PM | Permalink | Comments (0) | TrackBacks (0)

The Improve Group has used secondary analysis in conducting quantitative analysis. Like focus groups and interviews, secondary analysis contributes to a comprehensive, in-depth understanding about the issues facing an organization and their participants.

So, the Improve Group calls you to participate in secondary analysis– what exactly are you getting yourself into?

What is secondary analysis?

Many of us have read articles about results of studies using secondary analysis. Secondary analysis uses existing data that may have been collected for previous purposes in order to examine an issue different than originally intended. The data could have been collected for determining eligibility or enrolling participants, managing a program, or for a previous study. This data can range from responses to interview questions and focus groups to individual test scores.

What secondary analysis isn’t.

Secondary analysis is not the re-evaluation of the original study. However, it can use that original study to explore new areas. The process should be straightforward and transparent, especially when dealing with sensitive information. When researchers ask for secondary data from an original data collector, they should be able to provide a detailed answer to the following questions:

• What is the purpose of the research?
• How will this data be used?
• How will the results of the research be shared?
• How will confidentiality be preserved?

In addition to answering questions, the researchers should learn as much as they can about the data, including any issues in validity or consistency of the data. Studies using secondary analysis should be well-grounded in the theories of social science. Otherwise, the validity of the results of the secondary analysis can be seriously questioned.

What is the value of secondary analysis for the participant?

Secondary analysis provides an inexpensive and convenient option in analyzing multiple sets of data that can build on the work of other researchers. Such analysis contributes greatly to answering important societal questions that affect the participant. Don’t be afraid to take existing data as a starting point to evaluate an important current issue. If you are interested in the results of a study or how a study can be used, feel free to ask questions and provide comments about the study to the researcher. Additionally, you can request a copy of the results of the secondary analysis.

Therefore, if a researcher asks you to participate in secondary analysis, you should strongly consider it!

Posted by Guest author at 09:44 AM | Permalink | Comments (0) | TrackBacks (0)

We are all passionate about something. There is something we all wish to change or make better for ourselves and the people around us. Finding help to make those changes happen can be very challenging. If you are working to make things happen in your community the following funding opportunity may be of some help.

This week’s pick: The Public Welfare Foundation “The Foundation's grant making supports strategic advocacy that aims to transform public policy at the local, state and national levels.” They fund a wide range of community and neighborhood initiatives as long as they include citizen action and giving people a voice in creating solutions. One of their newest initiatives will fund work helping to create policies for employee paid sick days.

Other their topics of interest are:
Community development

Environment
Human rights
Criminal justice
Health
Civil participation

Typical funding ranges to $3,000. To start the process, submit a letter of inquiry at any time (see inquiry letter checklist on the foundation website). Note: the foundation will be issuing new program guidelines this fall.

Posted by admin at 11:05 AM | Permalink | Comments (0) | TrackBacks (0)

We had a fun time celebrating Susan and Liz’s birthdays at lunch today. At the request of the birthday girls, we enjoyed Panera sandwiches, soup, salad and chips and finished off our celebration with some cookies and birthday apple pie. Yum! We brainstormed on costume ideas for the next Harry Potter book release and laughed about some recent sketches on the Daily Show. We always have a fun time at these celebrations and it was a great break from a busy week in the office.

Posted by Guest author at 02:52 PM | Permalink | Comments (0) | TrackBacks (0)

While conducting research for a recent proposal, I came to a rather hopeful conclusion. I found that the literature on teacher quality offers promise for helping disadvantaged students overcome community barriers. The studies I reviewed showed that schools have greater influence on student achievement than community characteristics.
This presents a great opportunity, but also a great responsibility for teachers.

Linda Darling-Hammond’s (2000) study of the correlation between the existence of institutions that set licensure policies and percentages of qualified teachers has implications for a licensing and monitoring infrastructure that is currently lacking in early childhood education. The findings suggest that policymakers could play an influential role in ensuring the quality of early childhood programs by increasing requirements.

Early childhood providers also need to be aware of what qualities make an effective teacher. As Carroll Helm indicates, caring and empathetic teachers are more able to build students self-esteem, which is important for future academic performance (2007). More advanced teacher screening processes would need to be used in order to take these qualities into account.

Policy makers and school administrators face two challenges in reforming current practices. First, there is a great amount of variance in the qualifications of current early childhood teachers. In standardizing the credentials, administrators need to be sure not to marginalize the experience of current teachers who lack the credentials. Second, costs will increase with demands for more qualified teachers. College students are looking to get a return on their education, therefore, in order to draw graduates into the field, sufficient compensation will have to be provided. Schools will also incur higher costs in recruiting, screening and training highly qualified teachers. Because the costs of early childhood education and childcare are so entangled, the funding streams are complicated. Individual parents, communities, school districts and charitable organizations are all likely to see increased responsibilities to cover these costs.

Sources:
Darling-Hammond, L. 2000. Teacher quality and student achievement: A review of state policy evidence. Educational Policy Analysis Archives 8:1–48.

Helm, C. (2007). Teacher Dispositions Affecting Self-Esteem and Student Performance. The Clearing House, 109-110.

Posted by Guest author at 02:21 PM | Permalink | Comments (0) | TrackBacks (0)

Are you the kind of person who loves a good story and can instantly think about how that experience might relate to other people? Or do you really enjoy seeing a chart or a table and thinking about how all the different numbers explain an issue?

Stories, videos, pictures, recordings all can provide valuable information -- and evaluators usually lump them together in the category "qualitative data". Charts, tables, percentages are equally valuable and usually lumped together as "quantitative data".

We usually use both types of data when helping our clients understand the outcomes or results of a program. While both presents interesting information, using them together provides us with a greater depth of understanding about what happened when someone tried a new program.

Yesterday someone asked me "How do you get someone to see the value of qualitative data?" and this was probably the tenth time I've been asked this question. Some people are just going to want hard numbers and that quantitative data is the best way to get at the truth of a situation. I don't dispute that numbers are great, but you can't guarantee that everyone is going to respond to them.

One way to make qualitative data more responsive to a number-phile is to present both types of information side-by-side. For example, you may have a chart on the left-hand side of a page, with relevant quotes on the right-hand side. (By the way, when I was checking to make sure "phile" was the right suffix to use, I found a number of great resources: the Columbia Guide to Standard American English, the suffix entry at Wikipedia, and the Vocabulary Workshop for people preparing for an American University with its extensive discussion on suffixes. There was also a nice list at http://www.michigan-proficiency-exams.com/suffix-list.html, but it didn't include "phile")

Another way to present qualitative data to those who prefer numbers is to convert stories to numbers. For example, you might do a focus group or two and learn that several people have enjoyed a specific social experiences through a program or organization. You could then design a survey for more people to find out if they similarly enjoyed the social aspect of a program, using some sort of scale. We used this approach in our work with United Jewish Fund and Council, in which we first explored some issues through focus groups and then conducted a larger community survey.

There is a related Dilbert cartoon; I am waiting for permission to publish it.

Posted by admin at 02:01 PM | Permalink | Comments (0) | TrackBacks (0)

by Leah

I want to sincerely congratulate John Bryson on his recent receipt of the McKnight Presidential endowed professorship. Prof. Bryson taught me a great deal when I was a student at the Humphrey Institute, including to have fun when working complex issues involving a lot of players with divergent interests. Dean Atwood asked if I would write a letter about Prof. Bryson for his nomination for a faculty award. In my letter, I said (among other things):

"I particularly enjoyed learning from Professor Bryson’s broad view of stakeholders and their importance in strategic planning initiatives. He had an intuitive understanding of how stakeholder engagement can strengthen a strategic planning process by ensuring goals are broadly based in the current context, building support for resulting strategies and ensuring a level of accountability for the benchmarks or outcomes set in the process. This lesson in particular has stayed with me in my professional years; stakeholder engagement is one of the areas I emphasize in my own professional work and continue to search for creative ways of building those relationships between organizations and their stakeholders... "

“Because I learned so much from Professor Bryson, I am truly grateful to have the opportunity to support him for the Regents Professorship. I owe much of the philosophy that underlies my work to his teaching.”

Posted by admin at 01:23 PM | Permalink | Comments (0) | TrackBacks (0)

Hello readers. I am happy to be back from a great Fourth of July holiday (I’m a real sucker for fireworks). I hope you had a good Independence Day as well.

It was hard to pick a topic this week since there are many interesting RFP’s out there. I decided to go with a Federal opportunity to shake things up a little. I have mentioned before that the Improve Group is ready to partner with groups to apply for grants that have an evaluation component. This offering is all about evaluation.

This week’s pick: The Department of Health & Human Services Administration for Children and Families’ Office of Community Services (OCS) is granting funds to conduct ongoing evaluations of two programs administered by the agency: Community Economic Development (CED) and Rural Community Facilities (RF). If your organization is involved in these programs you can receive funds to evaluate the program effectiveness in the areas of:

(1) program purpose and design;
(2) strategic planning;
(3) program management; and
(4) program results and accountability.

Some of the funding goals of this Evaluation of Community Economic Development (CED) and Rural Community Facilities (RF) Programs are to develop a data collection system, assist in design of evaluation instruments and identify and document best practices. The Funding Opportunity Number is HHS-2007-ACF-OCS-EC-0046 and the estimated level of funding available under each area is: Community Economic Development (CED) for $200,000 and Rural Community Facilities (RF) for $75,000.
To learn more about it please click here.

Posted by admin at 11:02 AM | Permalink | Comments (0) | TrackBacks (0)

Last February, I wrote about using blogs as an evaluation tool for the Girl Scouts of the USA's uniquely ME! program. One month later, we launched the uniquely ME! blog and I have been learning some valuable lessons about this evaluation method that I would like to share with you.

THE CHALLENGE: To effectively advertise the blog to girls and recruit girl authors.

In designing the blog, we envisioned a place where girls would post entries about their experiences with uniquely ME! and other girls would freely respond to their postings. Because many of the girls served in the program are under the age of 13, we had to make sure that our blog addressed all federal regulations to protect the girls’ privacy (see my entry for more details). Unfortunately, our original vision did not comply with all required regulations.

The great thing about blogs is that they are always evolving. We revamped our plans and decided to recruit girls to write for the blog through our contacts at the 32 Girl Scout councils implementing uniquely ME! We created a FAQ sheet about writing for the blog and asked them to distribute it to girls who might be interested. While we had a number of “bites,” they did not turn into actual entries authored by girls.

We needed a formal method of advertising the program to girls; we asked our council contacts to do this, but discovered more guidance was needed for these efforts. We decided to create a simple sheet advertising the blog that our contacts could give to girls. In the future, we may include this as part of our evaluation requirements.

Presently, girls can contribute to the blog by visiting the site and clicking on a link with instructions. Submission options include mailing, emailing or faxing their responses. This month, we will offer an even easier method to contribute by adding a link to click on for an online form.

The uniquely ME! blog is still a work in progress, but by being flexible and rethinking our original plans, we have created a fun and safe space for girls online!

Posted by Guest author at 01:09 PM | Permalink | Comments (0) | TrackBacks (0)

Improve Group staff are devoted to using evaluation to improve outcomes, so conducting a test presentation of our new workshop series seemed natural. On June 14th, 2007 we gave a pilot run of How Do I Know That: Creating tools to understand program outcomes and Inside Knowledge, Outside Perspective: Evaluating your organization form within. The series is aimed at those who want to begin or expand their evaluation knowledge and feel empowered to use it in their organization.

Presenter Leah Goldstein Moses taught participants how to develop evaluations that fit smoothly with program activities while maintaining validity and accuracy. Presenter Marian Kimball Eichinger gave insight into how to improve the evaluation work within organizations and how to select the right tools to get legitimate results.

Attending the pilot was Cassaundra Adler - Financial Educator with Lutheran Social Services, Trisha Lee Cook – Member Services Associate for the Minnesota Council on Foundations and Nathan Salzl – Evaluation Specialist for the Greater Twin Cities United Way 211 Call Center. Also attending was Improve Group consultant Jules Goldstein (who is a member of the Board of United Jewish Fund and Council) and staff member Susan Murphy.

Our thanks to Cassaundra, Trisha Lee and Nathan who were a perfect blend of non-profit experience, foundation background and small and large organization perspectives. The attendees gave honest and valuable feedback on what were the most useful aspects of the training, what could be added or deleted to better serve the needs of the target audience and how to enhance specific exercises and teaching materials.

The pilot group gave a unanimous thumbs-up to the new series, saying:

• The workshops helped me figure out where to start with my organization's evaluation. I felt overwhelmed before and unsure whether I was representing my own opinions or had solid facts; now I know how to collect, analyze and present accurate data.
• I gained a great vocabulary for talking with funders, program managers and staff; the tools provided during the workshop can easily be adapted for our programs.
• I am excited to use the information for my program; I like learning and continually improving my program and was never sure how to ask questions to get honest and fair responses. I am going to review our surveys to make sure they are gathering the information I need.
• The workshops were very fun, very informative and I feel lucky to be invited.

Space in the July 19th sessions is available; to download a registration form click here.

Posted by admin at 11:33 AM | Permalink | Comments (0) | TrackBacks (0)