PolicyMap’s Use of Big Data and Open Data
Here at PolicyMap, we often get questions about what kind of Big Data we have and what our role is in the Open Data movement. After reading a recent Guardian article, we decided that it was high time we addressed these questions for our blog readers.
In the Guardian article, Joel Gurin at NYU’s GovLab addresses some of the differences between big data and open data. He explains that, “While big data is defined by size, open data is defined by its use. Big data is the term used to describe very large, complex, rapidly-changing datasets.”
The way we think about it here at PolicyMap is that big data is much larger than the aggregated data that we generally provide to our users. Big data often includes rows and rows of individual transactions, for example, whereas the administrative data we use includes rows that represent aggregated information like groups of transactions at a geographic level, such as a county. Our data doesn’t target individuals, but helps describe communities.
Gurin goes on to describe open data: “Open data is accessible public data that people, companies and organisations can use….the data must be publicly available for anyone to use, and it must be licensed in a way that allows for reuse.” This is what makes up much of what we provide on PolicyMap. For us, it’s data that the government has provided through free and publicly accessible online downloads, and it’s available without restrictions on us reusing it on PolicyMap.
Gurin’s article provides a very informative Venn diagram on the relationship between big data and open data:
Two of our Data Team members had the chance to see Joel Gurin speak at the Economic Impact of Open Data event earlier this month, an event hosted by the Center for Data Innovation. At the event, they also attended the official release of Open Data 500. PolicyMap is one of the Open Data 500 featured companies that use government data as a primary business resource. We look forward to reporting back on our continued work with the Open Data 500 initiative. In the meantime, feel free to contact us with any questions about these trending data concepts.