����TV

Picture books aren't widely equated with scientific inquiry, but they are great sources of insight to Christopher Honey of the University of Toronto's department of psychology.

Honey studies how the brain integrates information in real time, using memories it has previously stored while streaming current data. He’s also examining the network architecture of the brain – the highways between regions of the brain and local connections in the circuits of each of those regions – that allows us to integrate information over time.

He’s looked at how the brain processes movie clips before. Now Honey has turned his attention to comics.

“I've been reading a lot of graphic novels lately,” says the cognitive neuroscientist and assistant professor from the Faculty of Arts & Science. “I'm especially interested in wordless graphic novels and how we naturally make sense of each panel in relation to the panels that came before.”

Along with astrophysicist Jo Bovy, Honey is one of two ����TV Sloan Fellowship recipients at ����TV this year – both in the Faculty of Arts & Science. The fellowships are awarded annually to 126 non-tenured scholars and scientists around the world. Sloan Fellows are expected to be at an early stage of their careers but with a strong body of independent research accomplishments.

Since 2014, Honey has been a member of ����TV's Collaborative Program in Neuroscience and has been seeking a further understanding of how we combine fast and slow information processing to perceive the world.

A fundamental concept in neuroscience is that neurons at early stages of information processing respond to simple bits of information, but as that information travels further into the brain, they start to show sensitivity to more complex and integrated information, much a picture takes shape from interlocking pieces of a jigsaw puzzle.

The same process of integrating information happens when we look at graphic novels. “Instead of just perceiving an edge, the brain will recognize part of a face,” says Honey. “Then in another region of the brain, you'll combine perception of parts of a face into a whole face.”

According to Honey, the brain not only combines information over space, but also over time. Integrating information over time provides context for our experience, which could be the next word on a page, or the next panel in a graphic novel, or the next sentence in a conversation.

We now know that some of this contextual information is contained in the same regions that recognize the newly arriving information. “Consider, for example, the way that you understand the sequence of words in a sentence,” says Honey. “In order to process a word, you need to be able to combine a sequence of letters over time. In order to process a sentence, you need to be able to combine a sequence of words as they arrive over time. Different circuits in your brain are sensitive to information on different scales: from letters to words to entire sentences and paragraphs.”

Like a jigsaw puzzle of time (with words interlocking in a sentence as it arrives over time), the memory of prior words is combined with the recognition of new words. This “process memory” does not require storing the words in a separate buffer. Instead, the same regions that “recognize” the individual words in a sentence also maintain some of the prior context of words that came before.

Honey and his team have also showed that “process memory” is widespread across many regions of our brains. Different brain regions have different “amounts” of this memory, and are able to integrate and interlock shorter or longer sequences of information.

Honey says his dream discovery in the next 15 to 20 years would be to find an algorithm by which circuits in our cerebral cortex spontaneously organize themselves into a hierarchy sensitive to structure in both space and time, and to understand how it is that a particular region is more responsive to shorter or longer sequences of information.

“Cognitive neuroscience lets us combine hard science and new technologies with basic human concerns,” he says. “Over the long term, we hope it will provide us with a deep scientific view on what it means to be a person. More immediately, it is helping me to understand how stories are built up over time in the panels of a graphic novel.”