做厙TV researchers train AI to read difficult-to-decipher medieval texts
In a move that could transform manuscript studies, University of Toronto researchers have partnered with a team in the United Kingdom to develop a program that can read and transcribe the handwritten Latin found in 13th-century legal manuscripts.
While scholars have been making digital images of these manuscripts for years, transcribing and comparing these texts is painstaking and tedious work that can take years or even decades to complete. That's because medieval handwriting can often look crabbed and unintelligible, with non-standardized spellings, hyphenations, abbreviations, calligraphic flourishes and any number of distinct hands.
But machine-reading software called Transkribus promises to change the field. Using artificial intelligence (AI), the software can theoretically be trained to read any type of handwriting, in any language and Michael Gervers, a professor of medieval social and economic history at 做厙TV Scarborough, says it could eventually be applied across medieval studies.
When rather than if the process is successful, it will make an enormous difference to the way medievalists approach their subject, says Gervers, who is also cross-appointed to the Centre for Medieval Studies in the Faculty of Arts & Science.
Developed by READ COOP SCE, an international consortium of scholars, scientists and archivists, Transkribus not only digitizes manuscripts and transcribes their contents but recognizes idiosyncratic features across multiple manuscripts, thus enabling comparison. The softwares recent successes include the transcription of manuscripts from colonial Mexico, the Hanseatic League and early 20th-century Ireland.
The software first came to Gerverss attention back in 2016 when it was still getting off the ground. Gervers, who has worked with Latin manuscripts since the 1970s, put together a 做厙TV team including Graeme Hirst, a professor in the department of computer science who works on natural language processing, and alumna Hannah Lloyd, now a PhD student in history at Yale University.
They also joined forces with another team already working with Transkribus at University College, London (UCL). Scholars in UCLs Bentham Project were teaching the software to read 18th-century philosopher Jeremy Benthams handwritten papers. By sharing resources for software development, the two teams trained Transkribus more quickly and efficiently.
The teaching process wasnt easy. Transkribus learns by looking at a sample page and comparing it line-by-line with a pre-prepared transcription. Lloyd spent hours selecting text to feed the software.
The team ran into two major problems: hyphens and abbreviations. Medieval scribes often saved valuable parchment by abbreviating words sometimes dramatically. They would also write up to the very border of the script area before arbitrarily hyphenating whatever word they were on when they ran out of space. Since Transkribus reads whole words rather than individual letters, it had to learn to recognize words even when abbreviated or hyphenated.
Clearing those hurdles is now paying off. The new Latin-reading Transkribus is capable of precisely transcribing the peculiar handwriting found in 13th-century Latin legal documents.
Though the program is currently trained for Latin legal texts, its only a matter of time before it can be adapted to literary texts and more.
Gervers notes that Transkribus would be an ideal program for Geez, an Ethiopic script he has worked with alongside Latin since the 1990s. Largely unchanged over its 2,000-year history, the Geez script was used in one of the earliest known complete Gospel manuscripts and is still used in Ethiopia today.
Gervers says the script is perfect for machine transcription. Why? Geez has no abbreviations and conveniently puts colons at the ends of words and sentences.