June 1 was the National Day of Civic Hacking. Over 11,000 civic activists, technology experts, and entrepreneurs in 83 cities developed software to help others in their own neighborhoods and across the country. The White House hosted more than 30 developers and designers to work with the We The People data API – the API that powers the We The People petitions website. Attendees had submitted ideas and portfolios to be considered back in April and I was one of those selected to participate.
The day began with welcome notes from our hosts, the White House Office of Digital Strategy. Representatives from NASA lamented that this day would already beat their record for the largest Hackathon held only a few weeks prior. From there, each participant gave a brief introduction of who they are, the skill sets that they bring, and the any project ideas that they had. The room included talent from local and federal government agencies, private industry, and even representatives of technology giants such as Facebook and Google. Many had travelled across the country just to take part in the event.
I formed a team with Bryan Braun, a contractor with the White House Office of Digital Strategy, and Ben Damman, an entrepreneur from Minnesota to work on Petitions NewsLink. Our goal was to display aggregate data about popularity of petitions over time by issue and allow the user to dig deeper by viewing related news from a point in time that is related to the most popular petitions within the selected issue. Our hypothesis was that many petitions were formed in response to major news events. One of most obvious examples would be a spike in submitted petitions in the “Firearms” issue category after the tragic shootings in Newtown, CT.
In order to put this puzzle together, our team decided to utilize:
- Highcharts JS library for data visualization
- We The Entities API to perform textual and sentiment analysis on petition text. This API was actually created just in advance of the Hackathon by a fellow participant and was under development throughout the day.
- New York Times Article Search API in order to search articles by keyword in particular sections
We also quickly learned that we needed a caching layer in order to perform aggregate look-ups on the data. Ben took the role of creating this layer, Bryan took the user interface role, and I took the role of communicating with the We The Entities and New York Times APIs. By this time everyone had settled into teams and set down to work and the room was quiet.
My first thought had been to also utilize the New York Times Tags API in order to correlate We The People issue categories with New York Times tags that could then be used to limit new results to only those most relevant. After spending a good deal of time working through their API, however, I began to realize that the data set that was returned limited by tag became to limited when faced searching on a short window of dates. With that work tossed aside, I began to focus more on a workable demo by our 4:30 demo time. We agreed that our goal for the first demo was to have a broad resultset and not necessarily spend our time refining the searches further – that work could be saved for the next phase. As our demo time approached we worked feverishly to finish up.
The demos showed us a broad set of projects that had been created given a small, but powerful, We The People API. It included two WordPress plugins for embedding petitions, a data visualization that showed party affiliation of signers, mapping visualizations, and type ahead tools to enhance searching and filtering of the data set. Of particular interests to journalists was a tool created to import data directly into Google spreadsheets to allow non-technical users to analyze and chart data within a familiar toolset. The full API gallery is available online. Most projects include links to demos and/or source code.