« November 2007 | Main | January 2008 »

December 21, 2007

U.S. Government increasing evaluation requirements

On Dec. 12, 2007 a new executive order initiated a review of nearly every federal agency to determine:

1. If the agency targets were suitably outcome-oriented; and
2. If measures are appropriate for each target

The executive order classifies targets in three categories: (1) Long-term Measures: Program outcomes that fulfill the program’s purpose; (2) Annual Measures: Implementation of plans and efforts to achieve long-term and strategic goals; and (3) Efficiency Measures: Efforts to provide the most benefits (outcomes and outputs) for the taxpayer dollar spent.

The timeline of this review is very aggressive; each agency will be reviewed by February 22, with revised measures developed and adopted by June 30. What is interesting is the focus on reliability, validity and quality (see quote below) -- which align closely with the Guiding Principles for Evaluators.

From the Order:

The Government’s ability to determine a program’s effectiveness, and to direct attention to genuinely desired outcomes, is largely dependent upon the quality of the programs’ performance and efficiency goals, i.e., their measures and targets. PART goals aren’t always as outcome-oriented as they can be. We should continue to make PART program goals more outcome-oriented and aggressive as well as ensure measures are characterized correctly in PARTWeb.

December 20, 2007

NCLB Renewal: “Growth Model” Gaining Traction in Accountability Debate by Eric Wong

Books.gif

The December 19 edition of Education Week published an interesting article regarding the U.S. Department of Education’s recent policy shift towards allowing schools to be sanctioned or rewarded based on student’s academic growth. The article states that this policy change enjoys strong support from researchers, policymakers and educators as the accountability debate for the renewal of the No Child Left Behind Act (NCLB) continues to fester.

This measure of accountability, called the growth model, is a stark contrast to the current policy, called the status model, which judges schools based on the number of students deemed proficient in a specific subject area. Critics of the status model argue that the model overlooks students who are demonstrating growth towards proficiency and occasionally failing to penalize high-achieving schools where significant numbers of students are not proficient. Additionally, accountability decisions are based off a comparison of test scores of a group of student compared to the previous year, a metric that is open to sampling bias. As a result, the growth model is gaining support among policymakers, educators and researchers because it tracks the academic progress of individual students over time. Growth model supporters argue that its “value-added” measurement of academic progress is a more accurate measure of whether schools are helping its students.

However, while the U.S. Department of Education recently announced it would approve all growth models that met the criteria for a 2-year pilot project while gaining widespread support among education professionals, implementation of these growth models exposes a number of significant concerns, including the following items.

1. To be able to implement the growth model, states must have data systems in place that are able to track individual students’ progress via test scores over time. Specifically, data systems need three items to implement the growth model: a unique student ID number, assurance that proficiency levels are consistent across grade levels and information why certain students were not assessed. Currently, only 34 states (including Minnesota) have data systems that have those items to track every student. It may take states several years to develop data systems able to collect this data adequately to implement growth models.

2. For these growth models to achieve the NCLB mandated goal of universal proficiency by the end of the 2013-2014 school year, experts believe that many schools will need large rates of annual growth to meet that goal. Under the NCLB, schools need to demonstrate progress towards achieving the proficiency mandate each year for all students.

3. While growth models may show the significant gains for individual students in schools with low achievement levels, designing growth models for high achieving schools presents significant challenges. These schools generally fare better under the status model since a high proportion of students are deemed proficient according to the mandated benchmarks. However, under the growth model, high-achieving schools will likely have low rates of academic growth. Education experts disagree on how to address this issue.

While the NCLB reauthorization continues beyond its original due date in 2007, accountability measures are here to stay. In my opinion, the introduction of growth models in education presents challenges and opportunities that evaluators should explore. Some areas to explore include the effects of these measures on minority and disadvantaged students, the effects in different regions and the specific elements of education programs where large rates of academic growth has occurred.

Do you have any reactions to what’s going on with accountability in education?

December 17, 2007

New grant opportunities - from guest author Leah Goldstein Moses

Two new grant initiatives came since my last post.

The first is very exciting - notice of a new Safe Schools/Healthy Students solicitation will be released on January 24, 2008, with a 3-month turn-around period. Several Minnesota districts have previously submitted applications. The 2007 solicitation is still available online and is not expected to be modified considerably. One of the interesting things in the current cycle is that grantees are required to include a logic model--see several Improve Group resources for information about this tool for managing your initiative and evaluation.

Healthy Together: Creating Community With New Americans is a grantmaking initiative by Blue Cross/Blue Shield of Minnesota designed to reduce health inequities for immigrants and improve the health and vitality of communities in Minnesota. Applicants may propose projects related to one of the initiative’s three objectives: 1) Foster facilitated exchanges between immigrants and the receiving community, leading to greater social connectedness, healthier communities, and increased opportunities available for all. 2) Build the capacity and viability of immigrant-led organizations. The foundation invites Letters of Inquiry for one-year organizational development grants of up to $10,000 each. 3) Promote the mental health and social adjustment of new Americans.


December 14, 2007

Grant Gazing 12.14.07 by guest author Leah Goldstein Moses

a href="http://www.theimprovegroup.com/weblog/clip_image002.gif">clip_image002.gif

Leah here. I am going to do my best to keep up Susan's tradition of providing valuable grant resources to organizations. The Robert Wood Johnson Foundation (RWJF) has opened a new round of competition for its 2007-08 vulnerable populations grants. The new grants competition, Young Men at Risk: Transforming the Power of a Generation, looks to identify the most innovative approaches to help young men ages 15 to 25 fulfill their potential and become healthy, successful adults. From dancing as means to address domestic violence to using music to promote improved mental health, an increasing number of programs are finding new ways to meet pressing societal problems faced by today's young men, the foundation notes. RWJF especially is interested in providing support to organizations that are outside the formal network of health-care providers, such as grassroots organizations, faith-based organizations and advocacy organizations (rather than traditional providers such as hospitals and medical clinics). Awards will be up to $5,000 for up to three competitors. Plus, winners get a chance to enter the larger vulnerable populations competition with a shot at $1 million grants.

Bernard Foundation
Deadline: January 31, 2008

The Viola W. Bernard Foundation was established initially in 1968 as the Tappanz Foundation to provide seed money for innovative mental health programs with a particular emphasis on families and children. The mission of the foundation remains to support innovative programs that address the interplay between social conditions and the psychological health of children and families. The foundation is a $5 million estate making grants of $200,000 each year. Grant applications are considered twice a year and Jan. 31 is the deadline for the March meeting. Applicants will be notified as to whether their application has been granted by written notice within approximately 30 days after board action. Grant award amounts will vary and depend on the scope of the project proposed. The foundation was created in the memory of Dr. Viola Bernard (1907-1998), who dedicated her life to a conception of psychiatry that broadened the traditional definition of the patient and therapist.

Data related to publicly-funded health care

The Centers for Medicaid and Medicare Services (CMS) manage and monitor eligibility for the federally funded Medicaid and Medicare programs. We recently found their data on Medicaid and Medicare enrollment, use and payments very valuable in preparing a proposal to study senior's use of in-home vs. residential services. Most data is current through 2004.

Medicaid Statistical Information System (MSIS data) is available in .pdf summary form or you can download Excel files directly by clicking on the zip versions. There are state-by-state comparisions of eligibility, beneficiaries, vendor payments and managed care. Because each table is presented in raw numbers, researchers may want to blend the Excel files with other sources of data (such as census data) to determine use levels within given communities.

Medicaid Analytic eXtract (MAX) allows you to actually search and summarize data yourself (although limited choice options are available). It is based on the same set of data as MSIS but is compiled annually rather than quarterly so lags a couple of years behind. There is generally more detail; for example, I ran a list of the U.S. top 10 drug groups ranked by total reimbursement (anti-psychotics are ranked highest in almost every state including Minnesota).

Data related to publicly-funded health care

datayoucanuse.JPG
The Centers for Medicaid and Medicare Services (CMS) manage and monitor eligibility for the federally funded Medicaid and Medicare programs. We recently found their data on Medicaid and Medicare enrollment, use and payments very valuable in preparing a proposal to study senior's use of in-home vs. residential services. Most data is current through 2004.

Medicaid Statistical Information System (MSIS data) is available in .pdf summary form or you can download Excel files directly by clicking on the zip versions. There are state-by-state comparisions of eligibility, beneficiaries, vendor payments and managed care. Because each table is presented in raw numbers, researchers may want to blend the Excel files with other sources of data (such as census data) to determine use levels within given communities.

Medicaid Analytic eXtract (MAX) allows you to actually search and summarize data yourself (although limited choice options are available). It is based on the same set of data as MSIS but is compiled annually rather than quarterly so lags a couple of years behind. There is generally more detail; for example, I ran a list of the U.S. top 10 drug groups ranked by total reimbursement (anti-psychotics are ranked highest in almost every state including Minnesota).

Our thoughts are with Susan Murphy

You may have noticed a "dip" in the quantity of new blogs over the last week, or be missing our regular Grant Gazing column in particular.

Sadly, our star author, Susan, has been dealing with a sudden and serious family illness this week. We've all been thinking of her and wishing her family well. If you have a personal relationship with Susan, I'm sure that she's included you on her Caring Bridge contact list.

Susan is hoping to be back at work in a couple of weeks. Until then, we will be thinking of her (and we will also be organizing something to help her - more about that in the coming days). I know she'd appreciate your kind thoughts, too.

untitled.JPG

December 13, 2007

Technology Tips: user-defined charts by Liz Radel

research tidbits.jpg

Welcome back to the Research Tidbits series!

Albert Einstein was quoted as saying that the definition of insanity is doing the same thing over and over again and expecting different results. This quote brings a smile to my face as I am reminded of my occasional trials and tribulations with technology. I have a feeling I’m not alone, although perhaps ‘insanity’ is a bit strong characterization! In the interest of preserving my fellow evaluators’ sanity, I’d like to share a tip that I recently picked up about importing and exporting charts.

Recently, I was working on a PowerPoint presentation and needed to import some charts from an Excel spreadsheet. These were user-defined charts with our specific color scheme (For some tips on how to make these types of charts, click here.) I found that every time I imported the charts, the labels disappeared. I first tried to alter the chart size in Excel before I copied them over, but to no avail. Try as I might, I still had the same result. It struck me that perhaps others experienced this problem, too. I typed my problem into a quick web search and found out that this was indeed a problem with some user-defined charts. However, I learned that I could make a chart into a picture by clicking shift, accessing the edit menu and selecting “Copy Picture.” Voila! Problem solved.

My take-away messages from this experience are: 1) Yes, there IS an easier way to import Excel charts; and 2) if you’re having a problem with technology, chances are you are not alone. Type your problem into a web search and with luck, your problem will soon be solved.

December 06, 2007

Help for a Worthy Cause by Susan Murphy

I have a son in the Cub Scouts and last Monday night Pack 90 delivered food for the Neighborhood House food shelf. Christi Anderson, Volunteer Services and In-kind Donations Manager for Neighborhood House met us and told us all about how the food shelf works to benefit our community. The Scouts’ food contributions weighed over 400 pounds which is about one afternoon of grocery distributions for the food shelf. Christi is an old friend of mine and I was happy to know she was there working so hard for those who really need it.

I want to thank fellow staff members for donating food items to for Nick to bring in. The Improve Group staff is always so great at watching out for others. If you are thinking of helping out those who have less than you please look at my Grant Gazing 11.28.08 blog for ideas of where to give.


Winter 2007 008.jpg
Alex and Nick with Christi Anderson

December 05, 2007

GRANT GAZING 12.5.07 by Susan Murphy

clip_image002.gif

At the elementary school my sons attend they have a unique opportunity to select two “theaters of learning” each school year. These are short, elective courses where you can learn more about things that are of interest to you but are not part of the regular school curriculum. My youngest, Nick, chose a nutrition course and he really benefited from it. His consumption of vegetables grew as did his knowledge on nutrition and exercise. As a mom, I was very pleased.

The funder I’d like to highlight today is also concerned with children’s health. The timing of this grant coincides with the United Nation’s International Year of the Potato 2008. This effort by the U.N. is aimed at raising awareness of the importance of the potato, and of agriculture in general while focusing in on reducing global poverty and hunger.

The effort by my grant pick this week has a more local focus: helping kids to learn about better nutrition choices in their own schools.

This Week’s Pick: The U.S. Potato Board and the School Nutrition Foundation are working together to launch the School Wellness Grant Program. The Program will award ten U.S. Public elementary schools funds for equipment and/or educational programs that will aid in improving health for students. Eligible applicants include school administrators, foodservice staff, teachers and parents. Grants total $2,500 each and applications are due by April 15, 2008. Click here for more details.

©The Improve Group    Phone: 1-877-IMPRVGP (467-7847)    Fax: 1-877-IMPRVGP (467-7847)