People are excited by how history can be presented online and the insights that can be revealed by innovative use of digital tools. While many say “wow!” they often follow it up with “but I can’t do that”. I started Stumbling Through the Future because I realised that there was a shortage of online digital history tutorials and blogs aimed at the beginner in digital history. We know that teaching a skill is the best way of cementing what we have learned. I knew that if I could explain how to do things in clear and simple steps to others I had developed the level of competency I desired.
At the recent ‘Working History’ conference organised by the Professional Historians Association of Victoria I presented a paper about how I use digital tools in the research process. This conference focused on our professional practice rather than the outcomes of our work. I titled my paper ‘Life of a Fragment of a History’, but really I could have named it ‘You Can Do It Too’. In this post I will write about the paper and some of the thinking behind it rather than replicating it. As it turns out, this post has taken a rather different track to my presentation. Rather than tracing the life of a fragment of history as I capture it, process it and file it, this post is largely about the IT learning process. If you are interested, you can flick through my presentation slides (spot the unintended steam punk and a cultural reference courtesy of some back-channel fun):
Earlier in the conference digital historian Tim Sherratt delivered a keynote presentation about ‘Telling Stories with Data’. I owe Tim a great deal as I have learned so much from him. He is very generous as can be seen by him making his paper freely available online. Continue reading
Photo by Julie-Ann Robson.
Mining Big Data is very useful for producing longue-durée history as Jo Guldi and David Armitage note in their book, The History Manifesto (read my post on this aspect of their book). However Big Data mining techniques have been equally productive for recovering the history of the millions of people in the past whose lives have before now, been largely hidden from view. We can home in on a chance comment about one person who does not appear in any archival indexes and using a multiplicity of sources build up a fuller impression of their life. Alternatively we can collect the hundreds of scraps of evidence and gain a greater sense of communities which have been previously ignored.
Over one million men from the Indian subcontinent were involved in World War I, whether as soldiers or in support roles, yet their role has rarely featured in the histories that we have told of this War. This is large-scale history that has been ignored until recently.
Over the last few months I have become interested in the interaction between Indians and Australians in World War I. Alerted to the participation of Indians in the War by Australian historian Peter Stanley, I had another look at the World War I diaries I am working with and found numerous references to them by Australian soldiers. Peter Stanley has recently published a book about the Indians at Gallipoli, Die in Battle, Do Not Despair: the Indians on Gallipoli which I am looking forward to reading. I have written about Indian soldiers at Gallipoli on my Stumbling in the Past blog in a post titled, ‘Indian Soldiers Fought at Gallipoli‘.
‘The Needle in the Haystack’: My #OzHA2015 Paper
We are in a time of extraordinary change in research practices in history and are only beginning to discover the potential of even the most simple technical tools. At last week’s Australian Historical Association Conference in Sydney, I delivered a paper where I discussed the methodology that enabled me to find previously uncovered references to Indian soldiers at Gallipoli.
I would not have been able to find out about Indians from these diaries or make the gains about the research on the beliefs of Australian soldiers on the front line, without the generosity of the digital humanities community. Everything that I have learned about using technology in history I have learned from digital humanists on Twitter, writing in blogs, THATCamps and most recently the Global Digital Humanities Conference in Sydney.
In the spirit of giving back, I have attached the paper I delivered at the conference at the bottom of this post. It is not exactly the same as the paper I delivered because I ran out of time so cut some of the comments towards the end of the paper. I have added hyperlinks, lots of references and some of the images I included in the slides which accompanied my talk. Continue reading
At some stage or another any historian in the twenty-first century will consider embarking on a digitisation project of their own. Back in 2010 I briefly explored the possibility of organising the digitisation of some old school text books that I had been researching as part of my work on the Teaching Reading in Australia project. If I was to organise this I wanted to do it properly and ensure the resulting data could be linked to other similar historical data and be useful for other researchers. I did not want to do another project that merely reproduced pretty pictures of text (pdfs) which were not machine readable.
I was quickly confronted by the sad fact that my ambitions exceeded my skills. From attending THATCamps, reading blogs and following digital humanists on Twitter I knew that I should encode the data in XML using the framework provided by the Text Encoding Initiative (TEI), but I didn’t know how to do that. I don’t like doing something unless I do it properly, and I always have too much to do, so I dropped the idea.
Like all historians I have transcribed many hand-written documents from photos of primary sources I have taken for research purposes in the archives. Each document is idiosyncratic. The relevant items on a page are not restricted to words. There are underlines, crossed out words (who did the crossing out?), notes scribbled in the margins by the original author at a later date or someone else. There are arrows, drawings or diagrams. Too often the writing may be illegible. Each of these important bits of information needs to be recorded in the transcription. Quite often I will use markup borrowed from html or make up my own methods to signal a type of message in a transcription.
Since then I have been fascinated by a project of Dr Melodee Beals who is a Senior Lecturer in History at Sheffield Hallam University. Beals is marking up her transcriptions of historic documents in TEI. Separating the design from the text is a fundamental principle of web design. TEI enables us to prepare the transcription in a way that can be easily formatted for display on websites via XSLT. Beals’ project makes so much sense for historians. Why not incorporate some basic TEI markup in our transcriptions from the moment we start transcribing documents?
I needed to learn more about this mysterious TEI.
Fortune smiled and one of the workshops offered at this week’s Global Digital Humanities Conference covered basic TEI. For the last day and a half I have been learning about TEI and manipulation of images in the workshop, ‘Introduction to Digital Manuscript Studies‘ conducted by Elena Pierazzo, Professor of Italian Studies and Digital Humanities at the University of Grenoble 3 ‘Stendhal’, and Peter Stokes, Senior Lecturer in Digital Humanities at Kings College London. (Have a look at the impressive results of Pierazzo’s TEI transcription work on Proust’s notebook).
I now have the kickstart that I need. Last night I worked on marking up a transcription I had done of a document from my own project to reinforce what I had learned. One thing that has been bothering me about some transcriptions available on the internet is the lack of consistency with date formatting. There are many ways we can write dates and authors of handwritten documents use all sorts of approaches. Last night I discovered ‘13 Names, Dates, People, and Places’. This is the TEI chapter for me! I discovered how to encode a consistent, searchable date format while preserving the idiosyncratic way it was recorded in the original document. Oh, the potential of this! Continue reading
Are you tired of downloading items one by one after doing a successful search in Trove? Do you want to get an overview of what the entire search looks like? Do you want to connect items which are stored in Trove with like items from another archive?
The Trove API (Application Programming Interface) helps you to do these things and more. People who can program can program can use this in an imaginative way and explore its limits. Yet despite its technical sounding name this is also a tool which people who have no programming skills can use. This introduction to the Trove API is written for those with no background in programming. Continue reading
A few weeks ago I attended the 2013 annual conference of the Australian Historical Association in Wollongong. There were many tweets from the conference and a few blogposts were written. Yet many sessions were not reported online.
In order to give a view of the whole conference, not just the sessions I attended, I have analysed the conference program and shared the results on my history blog, Stumbling Through the Past. I also analysed the conference tweets in order to understand more about the live reporting of the conference using this medium.
This post explains the methodology underpinning my analysis. I do this because I believe that it is important that researchers allow themselves to be accountable for their work by allowing others to check what they have done. I also believe that it is important that researchers share their data where possible so that other researchers can use it to do their own analysis.
There is also a third reason for this post. I hope it will help other people learn some simple but powerful techniques that they can use in their research. This blog is aimed at people starting out in digital humanities so the explanations will be basic. Continue reading
After I finished my thesis at the end of last year, I used my free time to start exploring the world of digital humanities. I started out my professional life as an accountant and as part of our university degree back in the 1980s we were required to do programming in COBOL, dabble with databases (structured, not relational!) and we were exposed to spreadsheets in the days before excel. In year 12 at school I did computer studies which included a fair amount of programming in Basic. Throughout my career in accounting and later working in public relations, I have had to work with spreadsheets, databases etc, but I have not done any programming since the 1980s.
Then I went back to university and studied history. I was astonished. Continue reading
In my spare time over the last few weeks, I have been experimenting with the tools developed by Tim Sherratt to extract data from Australian digitised newspapers available through Trove. In a previous post I discussed how we can produce graphs showing the frequency of the use of particular words in Australian newspapers over time using Sherratt’s tools. In this post I will look at other methods of text analysis and explain how I used Sherratt’s tools to extract a large number of articles from the Trove database and used a text analysis tool to further analyse the articles. This post is about possibilities, not conclusions. It is a work-in-progress, so I am keen to hear your suggestions and experiences. Continue reading
Over the last week I finally got a chance to try out the tools that Wragge (aka Tim Sherratt) has devised to mine digitised historic Australian newspapers accessed through Trove. This post is about the results of applying his tools. If you want to do this yourself check out Wragge’s posts, Mining the Treasures of Trove (Part 1) and (Part 2). Firstly let’s look at Wragge’s graph of a topic that I have been writing about this year – floods.
Wragge has produced the graph above showing the occurrence of the word “floods” in Australian newspapers digitised and accessible on the Trove website. As we would expect the word is mentioned more in years when there was severe flooding such as 1893. Continue reading
Essential backup devices: external hard disc and the Cloud,
Computer backup is vital. This was brought home to me when we suffered the Great Perkins Computer Glitch of 2010. Every computer directory that contained my work for the last year disappeared, including all my research for my thesis. Yes, it vanished! I was having trouble opening a directory; we left it to watch a movie and when we came back we could not see or access a single file. Fortunately my other half, Hubble, had set up not one, but two backup systems. Now we were to find out how good our backup systems really were…. Continue reading