Issue 22: March 2009

Using Evaluation Data for Marketing: the ethics and logistics of sharing data
by Leah Goldstein Moses

If you have been evaluating your programs, either internally or in partnership with an evaluator, you may wonder whether it is ethical or even advisable to use your data in grant proposals or other marketing efforts. The short answer is “YES”. Evaluation data can be used in a narrative about the need for your program, when describing the quality of your services, or in making the case for a specific aspect of funding. However, here are some things to keep in mind when sharing your evaluation data:

1. Integrity. When you share evaluation results, it is important to do so with integrity and honesty (see p. 3 of the Guiding Principles for Evaluators). Methods, data and findings should be represented accurately and fully. This means sharing all findings—those that put your organization in a positive light, as well as those that show any outcomes or goals that are not achieved.
2. Respect for privacy. If there is any risk that your participant’s private information could be discerned from your evaluation results, don’t share them. An example we recently encountered was when sharing summary results with Regional Arts Councils from our statewide arts study. Because some of the regions have relatively small populations, it might be possible to identify an individual and their responses from a few demographic questions; for example, if the person reported that they own a café, but there is only one café in the town. To protect respondents, we grouped several together in categories like “business owners and professionals”.
3. Interpreting data appropriately. When you review your evaluation data, what story does it tell you about your program’s successes and challenges? Share those for the reader. Also help your reader understand any challenges you had in conducting your evaluation. It is important not to overstate your case. If you do not have a comparison group, but have participants’ stories of how your work changed their lives, you can confidently say that participants attribute improvement to your program, but not that your program caused the changes.

Sharing both positive and negative findings is not only ethical; it is also a great way to engage your readers in finding new ways to support your program. For example, if you have found that your after school tutoring program has helped students improve state test scores, share that and encourage people to help you to continue to deliver the program. If a parent engagement component has not worked as well, and you think you need an additional staff person to help reach parents, describe the challenges you experienced and how readers can help you improve.

RFPs that Get Reults
by Susan Murphy and David Rothstein

With the Stimulus Bill requiring a speedy distribution of funds for projects, it is important that the gatekeepers to these funds make the Request for Proposals (RFP) process efficient.  A well-written RFP can provide better information, which can also save time for everyone involved and allow the issuer to make an informed hiring decision.

Some of the toughest work on a project starts in-house and long before anyone is hired to carry out the job – this is the effort put into writing the RFP.  The RFP is a document that lays out exactly what your project or program is about, what you have done so far, who is involved, what you hope to accomplish, and what steps it will take to achieve your goals.  The typical goal of the RFP is to solicit strong proposals from vendors or consultants who will carry out the needed tasks in time- and cost-efficient ways. A lot of thought needs to be put to creating an effective RFP, and the following suggestions come from the Improve Group’s many experiences with the RFP proposal process:

• Don’t make it hard for prospective bidders to find your RFP.  There are a lot of sites willing to publish RFPs.  Think of who your target audience will be.
• If your organization is a frequent funder, consider a common RFP document for all the areas you fund.
• Have those who will be working on the project or program be involved in creating the RFP.
• Articulate a clear statement on why this RFP is being issued and what you hope to achieve by hiring a consultant.  This should be at the beginning of the document so those reading your request will know if it is a fit for their services.
• Be as specific as possible in your request.  There is a big difference between needs and wants.  If the RFP is all over the place asking for too many things, it will be hard to find an organization that can fulfill all your requests.  If it is a project that is very complex and multi-leveled, you can do one of three things:
• Issue a series of RFPs or note that the project may be accomplished in stages
• Issue one RFP which clearly explains that the responder can chose one or all of the project tasks to apply for
• Issue one RFP with provisions for using subcontractors for portions of the work
• Put your RFP in a logical outline form with the content you want submitted in the applicant’s proposal. This gives your respondents a clear view of how you want to see the information laid out and provides a standardized and comparable response for your proposal reviewers/evaluators.
• State clearly your due date for questions from applicants and how and when questions regarding the RFP will be answered.
• Please put in a budget range even if you cannot commit to a specific amount. This simple act will save time by giving your applicants a strong sense of how to formulate their costs. A simple statement to use is, “Budget not to exceed X amount of $”
• Ask for budget options if you wish to see how much work is possible with different budget amounts.
• Another time saver for applicants is noting whether vendors or consultants need to meet specific qualifications. For Example: Must have experience working with Medicaid regulations.
• For RFPs that deal with databases, previous research, or secure information, let the applicants know the amount of access they will have to theses resources.  If it will help them in preparing a response, publish a link to the data in your RFP.
• State how much time your staff will have to assist on the project, and if possible, what role/s you see them playing.
• Clearly define expected tasks and deliverables.
• Make the submission electronic.  This is good for those thinking green and it helps save some trees.  Also, if you do have a short response time, this allows the applicant more time to work on the proposal and less time printing, binding, and mailing.
• Be clear about the format of the RFP if that is important to you.  For example, those who will be reading and evaluating the proposals may not want to see a lot of 10 pt font, or may prefer a double space for easy reading.  In this same context, be sure to let the respondent know if you want color printing, bindings, tabs, disk copies, etc. as all of this increases preparation time.
• It is useful to put an evaluation scale in your RFP.  That way your respondents will know where to concentrate the most detail in their proposals.

Example:

Statement of Understanding	10 pts
Work Plan	35 pts
Budget & Budget Narrative	25 pts
Relevant Experience	30 pts

Finally, look at your calendar carefully when setting a due date for the proposal, taking into account postal holidays and how many working days you have allowed between publishing the RFP and expecting a response.  Some RFPs come out with astonishingly few days to put together a proposal.  Give the applicant time to formulate a strong work plan and to do the research that will generate a comprehensive reply.

Editor’s Note: If you are responding to an RFP that will require evaluation as one of the required components, please call us at the Improve Group as a possible partner.  We can provide assistance in creating the appropriate evaluation response.