����TV

AI or not AI: that is the question.

Computer scientists in Australia teamed up with an expert in the University of Toronto’s department of English to design an algorithm that writes poetry following the rules of rhyme and metre. To test their results, the researchers asked people online to distinguish between human- and bot-written verses. The best version of the algorithm fooled people nearly 50 per cent of the time. 

In some ways, the computer's verses were better than Shakespeare's. The rhymes and metre in the machine-generated poetry were more precise than in human-written poems. But “Deep-speare” still has a long way to go before it writes anything worthy of the Western canon. 

“It's very easy for me to tell what's by a computer or not – ridiculously easy,” says Adam Hammond, an assistant professor of English at ����TV and co-author of the paper outlining the algorithm.

“We solved two out of four problems,” Hammond says, referring to rhyme and metre. “The other two are much harder: making something that's readable and something that can evoke emotion in a reader.”

Adam Hammond, an assistant professor of English at the University of Toronto, co-authored a paper in which he assessed quatrains written by an algorithm (photo by Geoffrey Vendeville)

Computer scientists at IBM Research Australia, the University of Melbourne and Thomson Reuters trained a neural network using nearly 2,700 sonnets in Project Gutenberg, a free digital library. The computer uses three models – language, metre and rhyming – and probability to pick the right words for its poem. It produced quatrains, or four lines of verse, in iambic pentameter.

The researchers assessed their results by asking people online to tell the human and algorithm poetry apart. Most laymen – maybe confused by Elizabethan English – couldn't tell that verses like this one were the work of a programmed poet: 

With joyous gambols gay and still array
No longer when he twas, while in his day
At first to pass in all delightful ways
Around him, charming and of all his days

But Deep-speare didn't fool the expert. Hammond says it was easy to spot the computer’s verses because they were often incoherent and contained grammatical errors like the one above: “he twas.”

He rated Deep-speare's quatrains on rhyme and metre, finding that they were better in these respects than poems written by humans. The reason? “You have metre in a poem to create expectation,” he explains. “You do that so you can break the pattern. It's about creating expectations and then violating them at some point.”

The break signals a change in the emotional state of the speaker, Hammond adds. 

And as further comfort to poets who worry about being replaced by AI, Hammond says there is much more to a sonnet that a computer can't imitate – at least not yet. A sonnet contains three quatrains and ends with a couplet, two lines of verse. “It's actually a really complicated form because usually the first bit proposes a problem and then there's a turn, then the couplet that offers a solution,” he says.

“Imagine teaching a computer to come up with a problem, express that problem and then offer a solution to it. It's impossible to imagine.”

And yet his co-authors appear undaunted. Hammond says his fellow researchers want to fine-tune the algorithm so that it sticks to a single topic, or design an algorithm that can write short fiction. “Our results suggest that future research should look beyond metre and focus on improving readability,” the researchers say.

in the proceedings of the Association for Computational Linguistics conference.