Have you heard of big data? If not, a recent article in the New York Times provides an excellent overview. If so, you might be thinking, like I am, what the implications are for you, the way you work and the way we understand our world. You might also be wondering why this is a hot topic now.
A few major factors are leading big data to be a current hot topic – and these same factors also greatly affect evaluation, research and planning:
- We live our lives in ways that leave bits of data behind. From purchases made with a credit/debit card, to books checked out from the library, to visits with doctors, there are records. Even when your records are anonymous, they can be linked together OR combined with hundreds of other people to understand patterns. A recent blog and a video describing how your data is used, even when it isn’t linked to your name.
- Computers can do massive amounts of data crunching for us that wasn’t possible years ago. All of that data was cumbersome to manage and analyze without that computing power. Now, if you want to know the likelihood that someone who purchased matches did so because they’ve taken up smoking or because they are going camping, you can come up with a fairly good guess with a few clicks of a mouse. And, while we purchase fairly expensive licenses for statistical software to make our lives easier, you can have the same capabilities through free, open-source options such as R if you are willing to invest in learning how to use them.
- If you have big questions, big data is a goldmine. Imagine a question like that posed by Target – can we know if someone is pregnant before they even tell us? While that might seem a little creepy, there are other questions that the answers could be put to very good use by people in the social sector. Imagine answers to questions like: what circumstances lead people to binge drink? What triggers people to start smoking after successfully quitting? What communication patterns help or hinder us in the workplace?
- Big data can also help if you want to explore options. For example, split tests, with a long history in the commercial sector, are becoming more common in the social sector. In split tests, you try a few different messages or approaches and find out the results. The recent Crisis Connection efforts to try text messaging as a strategy to prevent suicide is a great example; they can see differences in who accesses their services, when, and the ultimate result when people reach them by phone or email.
- Would you be willing to opt-in to have even more data gathered if it meant improvements in health or other services? Each year, hundreds of people sign up for medical trials. If some data was gathered automatically – for example, your scale transmitting your weight or your phone tracking how many steps you’ve taken in day.
