BoE’s Underground blog has a nice post on how to use (and not be mislead) by big data ideas.
Could Twitter help predict a bank run? That was the question a group of us were tasked with answering in the run up to the Scottish independence referendum. To investigate, we built an experimental system in just a few days, to collect and analyse tweets in real time. In the end, fears of a bank run were not realised, so the jury is still out on Twitter. But even so we learnt a lot about social media analysis (and a little about American Football) and argue that text analytics more generally has much potential for central banks.
While anaylisng tweets for bank runs the analysis got mixed with home runs as a popular match was being played during the time of real analysis!
For example, our attention was drawn to a spike in the early hours of Monday 15th September. However, we quickly realised that this was driven by American Football, rather than events closer to home!
On closer examination, it transpired that we were looking at a series of tweets and retweets involving the Minnesota Vikings. This had been captured because they combined the term “run” and the abbreviation “RBs”. But in this context, the reference was to Running Backs and not the Royal Bank of Scotland! To avoid this pitfall, the search terms were subtly changed to avoid that particular pattern of upper and lower case characters.
Tweets were monitored for a full week, with particular attention on the night of the referendum vote. In the end, there was little traffic, as the ‘no’ result became increasingly apparent throughout the morning. Even so, this was a valuable exercise, building capabilities and knowledge to serve as a foundation for future projects.
In particular, the project presented a number of IT challenges. Our search terms needed to be securely transmitted to the data provider; if it was revealed that we were interested in particular firms this could cause the very run we were concerned about. We also needed to stream, store and query these data in real-time, and achieved this using novel technologies for the Bank, opening up new avenues for analysis and research.
Social media provides a rich vein from which to mine information, and text data more generally can reveal trends in people’s opinions and sentiment on specific topics or events. The Scottish Referendum gave us a great opportunity to do this in a time-critical context, and provided valuable live insights.
Kevin Warsh, commenting on transparency and the Monetary Policy Committee, remarked that while “studies seeking to make sense of millions of spoken words” are “daunting and imperfect”, text mining has “meaningfully advanced our understanding” of central banks. And as this post illustrates, central banks are themselves adopting these tools and techniques to address a wide range ofpotential applications across central banking, and build more agile and wide-ranging data analysis capabilities for the future.
BoE blog is coming out with some really interesting posts. Say such analysis is done in India and a cricket match involving Indian team as well. Just imagine how messy it will be..