TheVine story: “Numbers Game: Will Predictive Analytics Decide September’s Federal Election?” June 2013

The biggest feature I’ve worked on to date — a TheVine story looking at how Barack Obama’s predictive analytics techniques might be applied to the upcoming Australian federal election. Excerpt below.

imageA week after November’s United States presidential election and Nate Silver was everywhere. Already an established member of the American political commentariat via his blog FiveThirtyEight.com, Silver’s name went into the stratosphere after he correctly predicted the winner of all 50 US States and the District of Colombia in the 2012 race for the White House.

A week after November’s United States presidential election and Nate Silver was everywhere. Already an established member of the American political commentariat via his blog FiveThirtyEight.com, Silver’s name went into the stratosphere after he correctly predicted the winner of all 50 US States and the District of Colombia in the 2012 race for the White House.

Silver went from being a political analyst on news television to a regular guest on mainstream talk shows across the United States. How had his predictions been so accurate? And what did it mean for future elections in the US? Silver had become the country’s political oracle, in the process doing a bunch of entrenched pundits out of a job, and his profile was rammed home with the release of his own book, The Signal and the Noise.

But as the northern autumn turned to winter and the United States settled in for the January swearing in of Barack Obama for a second four-year term as president, a new story began to emerge. Silver had become a lightning rod for the statistical community and ‘big data’ believers the world over. But by January it was clear his statistical predictions were actually a secondary story – a sidebar to the most phenomenal harnessing of information in electoral history.

“He was doing forecasting,” says Eric Siegel. “So for an entire state of voters – especially swing states – it was interesting. Whereas the Obama campaign’s use of predictive analytics was making predictions for each individual voter in the state.”

Predictive analytics. Persuasion modelling. Machine learning. These are the terms that have emerged out of the fog of numbers documenting the election. Siegel, Ph. D., is the president of Prediction Impact, a US-based analytics and data mining firm, and the author of Predictive Analytics: The Power to Predict Who Will Click, Buy, Lie, or Die. He watched with fascination as the real story behind the presidential vote became apparent.

“Which is more powerful? Winning at a prediction, which Nate Silver competed publicly to do. Or winning the election itself, which by its nature of course is going to be done less publicly. Predictive analytics: a defining characteristic is that it’s predicting for each individual customer or person or patient or voter. That means it gives it the power not only to predict but to influence the future.”

Predictive analytics has already been widely used throughout the commercial sector, in situations ranging from the curious to the profound. Prison parole boards predict which prisoners might be repeat offenders. HP predicted employees who were likely to quit their jobs. In one particularly infamous case, Target used purchase history and other customer information willingly disclosed to the company to predict pregnancy among their female customers.

In each of these instances, the value of predictive analytics to change the future is clear: don’t release the prisoner, adjust working conditions for the disaffected employee, lift sales with the targeted marketing of baby products. But the Obama campaign took such techniques to a whole new level. Using a particular brand of predictive analytics known as persuasion modelling, a team of 300 data analysts went to work augmenting data sets from the Democratic National Committee with reams upon reams of polling data. A poll on its own is a method of forecasting, but combined together with other data – both from the DNC and in some cases acquired from private sources – they allowed the Obama team to make predictions per voter and run models on whether each particular person was persuadable.

Now, the Democrats wouldn’t be wasting time talking to people who had no chance of voting for Obama. Working from one single data source, they could utilise all of the channels available to them – email, Facebook, TV and mail drops – to target their list of ‘persuadables’.

Mitt Romney had his own analytics operation – Project ORCA – which he’d used to give himself a clear edge in the Republican primaries. But it was nothing compared to the scale of data collection being undertaken by Obama’s campaign team.

Siegel tells of shooting the breeze with a colleague months before the presidential election: “We were speculating, ‘Gee, I bet you within four or maybe eight years we’re going to find out one of the presidential campaigns has used this stuff,’” he recalls. “Then, a month after the election, that’s when I really got to interview the campaign’s chief data scientist and found out, ‘Oh my God, they actually did this!’ And they did show that it was a factor that got them significantly more votes.”

Extrapolating the data

Obama’s chief data scientist was Rayid Ghani. Ghani neatly encapsulates predictive analytics as the ability to personalise at scale. “There’s no other way,” he tells TheVine. “This is the only way you can do it for millions of people and still keep it very personalised.”

Ghani is talking from Sydney, where he’s travelled as a guest of CeBIT Australia, a global business meeting, exhibition and networking event. Ghani’s not the only member of Obama’s analytics team who’s been in the country recently – Joe Rospars, the president’s principal digital strategist and adviser, appeared at last month’s Sydney Writers’ Festival – and he says there’s been plenty of Australian interest in the big data techniques used during the presidential election. “Not just Australia, but pretty much any country where there are elections happening – they’re all interested in learning about how we did certain things and how it transfers to their electoral system and how they can apply some of these techniques.”

For the full article, visit TheVine.

Leave a Reply