A real life experiment illuminates the future of books and reading

Books are always transforming. The book we hold today has arrived through a number of materials (clay, papyrus, parchment, paper, pixels) and forms (tablet, scroll, codex, kindle).

The book can be a tool for communication, reading, entertainment, or learning; an object and a status symbol.

The most recent shift, from print media to digital technology, began around the middle of the 20th century. It culminated in two of the most ambitious projects in the history of the book (at least if we believe the corporate hype): the mass-digitization of books by Google and the mass-distribution of electronic books by Amazon.

The survival of bookshops and flourishing of libraries (in real life) defies predictions that the "end of the book" is near. But even the most militant bibliophile will acknowledge how digital technology has called the "idea" of the book into question, once again.

To explore the potential for human-machine collaboration in reading and writing, we built a machine that makes poetry from the pages of any printed book. Ultimately, this project attempts to imagine the future of the book itself.

A machine to read books

Our custom-coded reading-machine reads and interprets real book pages, to create a new "illuminated" book of poetry.

The reading-machine uses Computer Vision and Optical Character Recognition to identify the text on any open book placed under its dual cameras. It then uses Machine Learning and Natural Language Processing technology to "read" the text for meaning, in order to select a short poetic combination of words on the page which it saves by digitally erasing all other words on the page.

Armed with this generated verse, the reading-machine searches the internet for an image—often a doodle or meme, which someone has shared and which has been stored in Google Images—to illustrate the poem.

Once every page in the book has been read, interpreted, and illustrated, the system publishes the results using an online printing service. The resulting volume is then added to a growing archive we call The Library of Nonhuman Books.

From the moment our machine completes its reading until the delivery of the book, our automated-art-system proceeds algorithmically—from interpreting and illuminating the poems, to pagination, cover design and finally adding the endmatter. This is all done without human intervention. The algorithm can generate a seemingly infinite number of readings of any book.

The poetry

The following poems were produced by the reading-machine from popular texts:

"deep down men try there
he's large naked she's even
while facing anything."

from E.L. James' "Fifty Shades of Grey"

"how parties popcorn
jukebox bathrooms depressed
shrug, yeah? all."

from Bret Easton Ellis' "The Rules of Attraction"

"Oh and her bedroom
bathroom brushing sending it
garter too face hell."

from Truman Capote's <"a href="https://www.goodreads.com/book/show/251688.Breakfast_at_Tiffany_s">Breakfast at Tiffany's"

My algorithm, my muse

So what does all this have to do with the mass-digitization of books?

Faced with growing resistance from authors and publishers concerned with Google's management of copyright, the infoglomerate pivoted away from its primary goal of providing a free corpus of books (a kind of modern day Library of Alexandria) and towards a more modest index system used for searching inside the books Google had scanned. Google would now serve only short "snippets" of words highlighted on the original page.

Behind the scenes, Google had identified a different use for the texts. Millions of scanned books could be used in a field called Natural Language Processing. NLP allows computers to communicate with people using everyday language rather than code. The books originally scanned for humans were made available to machines for learning, and later imitating, human language.

Algorithmic processes like NLP and Machine Learning hold the promise (or threat) of deferring much of our everyday reading to machines. History has shown that once machines know how to do something, we generally leave them to it. The extent to which we do this will depend on how much we value reading.

If we continue to defer our reading (and writing) to machines, we might make literature with our artificially intelligent counterparts. What will poetry become, with an algorithm as our muse?

We already have clues to this: from the almost obligatory use of emojis or Japanese Kaomoji (顔文字) as visual shorthand for the emotional intent of our digital communication, to the layered meanings of internet memes, to the auto-generation of "fake news" stories. These are the image-word hybrids we find in post-literate social media.

To hide a leaf

Take the book, my friend, and read your eyes out, you will never find there what I find.

Ralph Waldo Emerson's Spiritual Laws

Emerson's challenge highlights the subjectivity we bring to reading. When we started working on the reading-machine we focused on discovering patterns of words within larger bodies of texts that have always been there, but have remained "hidden in plain sight." Every attempt by the reading-machine generated new poems, all of them made from words that remained in their original positions on the pages of books.

The notion of a single book consisting of infinite readings is not new. We originally conceived our reading-machine as a way of making a mythical Book of Sand, described by Jorge Luis Borges in his 1975 parable.

Borges' story is about the narrator's encounter with an endless book which continuously recombines its words and images. Many have compared this impossible book to the internet of today. Our reading-machine, with the turn of each page of any physical book, calculates combinations of words on that page which, until that moment, have been seen, but not consciously perceived by the reader.

The title of our early version of the work was To Hide a Leaf. It was generated by chance when a prototype of the reading-machine was presented with a page from a book of Borges' stories. The complete sentence from which the words were taken is:

"Somewhere I recalled reading that the best place to hide a leaf is in a forest."

The latent verse our machine attempts to reveal in books also hides in plain sight, like a leaf in a forest; and the idea is also a play on a page being generally referred to as a "leaf of a book."

Like the Book of Sand, perhaps all books can be seen as combinatorial machines. We believed we could write an algorithm that could unlock new meanings in existing books, using only the text within that book as the key.

Philosopher Boris Groys described the result of the mass-digitization of the book as Words Without Grammar, suggesting clouds of disconnected words.

Our reading-machine, and the Library of Nonhuman Books it is generating, is an attempt to imagine the book to come after these clouds of "words without grammar." We have found the results are sometimes comical, often nonsensical, occasionally infuriating and, every now and then, even poetic.