A terrific post by Prof. Cóilín Parsons who is a Professor of English at Georgetown University, in Washington, DC.
She points how big data is hardly a 21st century thing. It actually can be traced to 19th century when British decided to make Gazetteers documenting small details about parishes in Ireland.
Initially, they had envisaged dozens of them: slim booklets that would handily summarize all of the important aspects of every parish in Ireland. It was the 1830s, and such a fantasy of comprehensive knowledge seemed within the grasp of the employees of the Ordnance Survey in Ireland. These fantasies were, in fact, to be found all over Europe and its colonies – it was a time of confidence in the availability of the world to be turned into facts (to be gathered in maps, censuses, encyclopedias, and statistical reports), and in the capacity of humankind to describe that whole world. Nineteenth-century Europe saw the invention of big data as a tool of effective government. But the fantasy of comprehensive knowledge found its limit in the colonies, and nowhere more clearly so than in Ireland.
As part of the improving zeal of the British government in 1820s, a parliamentary commission was set up to investigate the necessity of re-surveying the island of Ireland, with a view to establishing new and more accurate land values (The classic study of the Survey in Ireland is J.H. Andrews, A Paper Landscape: The Ordnance Survey in Nineteenth Century Ireland, OUP 1975. Much of the historical detail below comes from this book). The commission tasked the Ordnance Survey, a branch of the British Army, with making an accurate and comprehensive map of Ireland at a scale of six inches to one mile, or 1:10,560. The scale might seem unexceptional to anyone alive now who grew up with the Ordnance Survey’s maps of Ireland and Britain, but at the time it was nothing short of revolutionary – it called for enormous maps of frequently sparsely inhabited areas, and at a level of detail never before seen across such a vast expanse of land. How was the Survey to gather the information to fill in such detailed maps? The answer was to task a crew of fieldworkers, not only to map the physical features of the landscape, but to record every possible aspect of the landscape from its placenames to its productive economy.
With all of these data being gathered, there was no room and no protocol for putting them onto a map. There was simply too much information – a placename might be included, but not its etymology; a mill might find its way onto the map, but not its history and ownership. The solution was to publish a series of printed gazetteers to accompany the map, which would record all of the extra information that the Survey officers would be able to gather in the course of their fieldwork. Col. Thomas Larcom, in laying out instructions to the officers about what kind of information to collect for just one of the many sections of their reports, asked them to note:
Habits of the people. Not the general style of the cottages, as stone, mud, slated, glass windows, one story or two, number of rooms, comfort and cleanliness. Food; fuel; dress; longevity; usual number in a family; early marriages; any remarkable instance on either of these heads? What are their amusements and recreations?
While not all officers filled out these reports, and some were more thorough than others, it is easy to see how the anticipated slim volumes generated a mountain of information, at a scale that was practically impossible to manage. Reports on the geology, meteorology, history, archeology, literature, and culture of parishes created scenes straight out of the fiction of Jorge Luis Borges, as the available information rapidly eclipsed the Survey’s capacity to edit and publish it. One Survey officer wrote that, to study the placenames of Ireland alone was “to pass in review the local history of an entire country” – a paradox of scale, and almost a parody of the fantasy of comprehensive knowledge that drove the British colonial administration in Ireland. How could a Survey be both intensive and extensive at once?
In the end, only one publication emerged from the whole project, covering just one parish (Templemore) in Co. Derry. At 350 pages in length, the ‘memoir’ as it was now called, had ballooned in size, and cost more than three times the estimated budget for the entire county. Begun in about 1834, it wasn’t finished until November 1837.
The volume had its own unintended consequences in stoking anti-imperial feeling. Read the post for more details. Even Census was a big data project.
One is wondering how India is missed from such posts. The British wrote just many many Gazetteers documenting in great detail several places of the country. These documents continue to serve as the major reference points about India’s locations even till date. Much of work on knowledge about a region was an eventual follow up of these gazetteers. In India’s case British did not stop at one gazetteer but made many more (forced rather) in order to know what was really a diverse and mysterious country. This was obviously followed by the Census as well which must have been a huge exercise back then.
This gazetteer writing could have easily been the biggest big data project ever undertaken and likely to remain in future as well. The amount if detailed data in these is plain shocking and tells you the pains taken to document such micro information which is missing these days. It will be interesting to figure this history of gazetteer writing in India as well.
This is again a reminder of importance of history and connecting the dots. Most of the time all the 21st century big ideas have an equally big and eventful past as well. All we do is keep changing names and make things fancier..