Whoever thinks data nerds are no fun clearly needs to work with the Improve Group. In the past six months we have had the wonderful opportunity to partner with the Hennepin County Library system to pilot some tools among youth and adult patrons. What makes this project so special is that in negotiating the challenges of drop-in patrons, we get to think through and test some of the more innovative approaches to data collection.
We have so far tested the following different tools (adapted):
- Photo Voice
- Video
- Ticket canisters
- Sticky notes on poster board
- Stickers on poster board
- Verbal survey
- Paper survey
Most of these tools are variations on the traditional survey, but have been adapted to make evaluation fun! We are still in the midst of the evaluation so cannot yet share which tools are most successful among patrons, but the following paragraphs detail what some of the tools are and how they were implemented.
The Improve Group/Nokomis Library version of Photo Voice was a means to capture examples of how parents were engaging with their child in the early learning stations at the library. Library volunteers asked parents to sign a consent form for photographs during a story time session, and the families were later approached to take the picture. We asked a few follow up questions about whether or not the family had participated in an activity before this station was put up in order to get a sense of impact. These photos can now provide a very descriptive picture of how the stations are affecting families, and may be used to demonstrate to funders and others why it is important. Conversely, we learned that families do want the opportunity to verbally share why they use the stations over other opportunities, so building in this storytelling may enhance the analysis.
The adolescents participating in the Homework Help program helped us gather data through a different multi-media technique, the video booth. The Improve Group set up a curtained off private corner of the library with a table and iPad to have students record “selfies”; talking about what they have gained from participating in the program. Again, consent forms were required from the youth parents which created a bit of a challenge in recruiting for the video. These 1-2 minute clips will provide unique testimonials about the program that will be promoted internally, and serve as a different format for youth to express their experience in the program.
Sticky notes and stickers on poster board are fairly straightforward collection methods. For the former, we asked one question about how people feel about the library. It turns out that some families love writing and talking about their experience with the library; in a few cases we had several notes put together to form a response! In contrast, the stickers allow for a bit more complexity. We asked a few questions through the use of a grid scale (less, same, or more use), similar to a multiple choice grid on a survey. Youth could indicate how their use of certain things at the library changed in frequency since attending the Homework Help sessions. We had tutors give out the stickers to the students, color coding based on grade and thereby allowing us a second layer of analysis.
Lastly, in heavy collaboration with the library evaluation pilot team we designed a set of ticket canisters to represent response options to a set of questions. Each question had 3-4 possible responses. Tickets were numbered as a set (so, five tickets that all had an “ID” number) and placed in baggies next to the canisters. Caregivers could take a baggie with their child and insert a ticket into the canister that best reflected his/her answer. With five questions, and five tickets, we are thus able to track an individual’s responses across questions. Although a complex way to ask questions about confidence and knowledge changes—something traditionally asked in a survey—many families used the colorful canisters and tickets as an activity to do with their child.
Want to learn more? I am excited to share that myself and another staff member at Improve Group have submitted an application to present on a panel on this topic at the American Evaluation Association (AEA) Annual Conference in October 2014. The presentation will cover how we have worked with several different types of cultural institutions to conduct evaluations using data gathered from drop-in patrons and the challenges of gathering information from a rotating audience not consistently available for data collection. We will talk about lessons learned and insights on creative and traditional data collection methodology in these environments.