Every few nights throughout the fall of 1977, my grandmother would disappear to her attic and tell stories to a picture of my mom on her desk. With a tape recorder rolling, she weaved what she called “the real, the historical, and the intuitive tapestry” of her life — from her childhood in Boryslaw, Poland, to her entire family’s deportation to Auschwitz-Birkenau concentration camp, where only she survived.
Recorded alone, without an interlocutor present, my grandmother’s tapes are an unusual form of Holocaust testimony, interrupted only by the cassette tapes themselves running out every half hour. Listening to the tapes 40 years later, I couldn’t help but notice how much these interruptions fractured my grandmother’s intimate thoughts and narrative cadence. Though I found myself picturing my grandmother speaking directly to me, each discontinuity was a visceral reminder of the specific context in which she had recorded those tapes: not as my Oma conversing with me in the present day but as a mother recounting to her daughter decades prior.
Where testimony is grounded in specificity, machine learning aspires for generality
Ever since David Boder recorded audio testimony of Holocaust survivors in 1946 using an Armour wire recorder, Holocaust testimony has been inextricably tied to the technology used to capture it. From the optics of the camera lens to the datafication of recountings during digitization, technological interventions frame not only the survivor’s experience of recounting but also our experience of listening. These mediations reveal just how much Holocaust remembrance is shaped by institutional choices surrounding technology in the context of testimony.
Historically, initiatives to record testimony have favored interview styles that encourage interlocutors to ask guiding questions but otherwise let the survivor steer the recounting. In this regard, the interlocutor’s primary role has traditionally been to listen. In 2012, the USC Shoah Foundation took a different approach. It launched a project called Dimensions in Testimony with the goal of creating interactive, AI-powered holograms of Holocaust survivors that visitors could interact with, as though they were helping guide the recollections. After pre-recording question-intensive testimonies with survivors, the interviews were segmented into short clips to produce banks of responses. When a visitor approaches a hologram in the exhibit and asks a question, a machine-learning system determines the most relevant pre-recorded clip and plays it back, thereby simulating a conversation. “Now and far into the future, museum-goers, students and others can have conversational interactions with these eyewitnesses to history to learn from those who were there,” the project description page announces.
Over the past eight years, Dimensions in Testimony has toured museums and institutions across the U.S., making it as far as the floor of the United Nations. The project has generated significant press as well as controversy. In the Los Angeles Review of Books, pre-eminent Holocaust scholar Marianne Hirsch called Dimensions in Testimony “the most recent symptom of a longstanding anxiety: What happens to the story of the Holocaust when the last survivors die out? How will it be remembered?” In a 15-minute documentary published by the New York Times, Davina Pardo found a project with “no clear answers” to its motivating questions but nonetheless believed the project to be an inspiring vision. In Real Life, Linda Kinstler argued that Dimensions in Testimony and other such projects concerning virtual reality representations of the Holocaust carry with them an immense “moral burden” in the abstractions they present.
Is Dimensions in Testimony kitsch? I wondered. As a computer science Ph.D. candidate and the grandson of a survivor, I questioned the idea that an algorithm could interact on behalf of a survivor. How much of a role should machine-learning processes play in collective remembrance? And what does this tell us about how we are expected to remember?
Holocaust remembrance has long grappled with the question of what will happen when the last survivor passes away. Without living survivors to recount firsthand, stories and histories will forever be lost. The testimonies and other documentation are in part a bulwark against such atrocities happening again, which seem as necessary as ever as we face strengthening currents of disinformation and a rise in fascist organizing.
Over the past 30 years, the push to document and record testimony has been monumental. The USC Shoah Foundation’s own Visual History Archive contains more than 54,000 recorded Holocaust testimonies (including one conducted with my grandmother). When considered in conjunction with the tens of thousands of other recorded testimonies preserved in archives across the world, the scale of ethnographic documentation is remarkable.
Though Holocaust recountings have taken on a diverse range of formats — from memoirs to paintings — these efforts surrounding the collection of testimonies have largely focused on oral histories recorded with an interlocutor present. This approach is tried and true. Ethnographers have refined this practice over decades; moreover, the format is inviting for survivors, and recording can be completed in a single day. In its scalability, reliability, and welcoming nature, the oral history approach adopted by countless institutions has much to offer.
Most testimonies are static artifacts, the Dimensions project reminds us, mediated by an interlocutor whose questions may differ from the questions you or I might ask
Dimensions in Testimony situates itself among this collective effort by highlighting the presumptive deficiencies of its peers. Most testimonies are static artifacts, the project reminds us, mediated by an interlocutor whose questions may differ from the questions you or I might ask. Implicit in this provocation is the belief that canonical testimony will eventually become unrelatable and grow stale, remnants of the distant past. Dimensions in Testimony’s AI-powered holograms address this supposed interactive deficit and disintermediate those static testimonies: According to the promotional material, the project “ensures that future generations will still be able to speak with and learn from survivors.” To keep Holocaust memory alive, the people of the future must be able to seem to directly converse with the victims for perpetuity, this project suggests. Dimensions in Testimony thus positions itself as a corrective that ostensibly humanizes the survivor and reminds the visitor that the past is never far away.
By drawing on machine learning to make this simulated interactivity possible, Dimensions in Testimony falls prey to the enduring belief in the emancipatory power of technology. In altering the possibilities of remembrance, the project offers a solution that competes with history, blurring the lines between simulation and reality. These alterations not only disrupt our understanding of the past but also undermine how we perceive the survivors themselves.
For decades, the artificial-intelligence field has obsessed over how to have machines convincingly participate in conversations. It is the basis of Alan Turing’s famous “imitation game,” which has an AI try to pass as human in a blind conversation. From chatbots for talk therapy to virtual assistants that we could supposedly fall in love with, contemporary media and speculative fiction alike fetishize the blurring of this conversational boundary. The success of conversational AI has always been measured according to the degree to which it passes as real, not the degree to which it offers anything meaningful.
This longstanding fixation brings the limitations of Dimensions in Testimony into focus. The project orients audiences away from a qualitative experience of listening and attending to the experience of another, toward a parlor trick preoccupied with fidelity and authenticity. Dimensions in Testimony invites audiences to measure its success not in terms of what they learn but rather by the quality of their interaction. Does the system retrieve proper answers to difficult questions? Is it fast enough to properly simulate conversation? Do the holograms seem real? In doing so, the project re-enacts a process of dehumanization, positioning the audience in a role reminiscent of the perpetrator’s: to judge whether someone is “authentically” human.
Even if we overlook how Dimensions in Testimony points audiences toward its technical accomplishments, it has further fraught implications. In falling prey to technological fascination with computer-generated conversation, it has forgotten to engage with what it means to have an algorithm interact on behalf of a survivor. This is no different from any other situation when someone speaks on behalf of another: It is an act of pre-emptive interpretation that denies agency to the spoken for, contradicting the very foundation of testimony itself. Machine learning obfuscates this with the allure of AI. And yet, no matter how competently the algorithm performs, Dimensions in Testimony will fail to deliver on its reparative promises.
What, precisely, is to be learned from testimony? Is it to provide us with facts for the historical record? Is it to have our questions answered? Above all else, testimony affords us the qualitative experience of listening, through which we witness the emotions of an individual attempting to convey the ineffable.
In altering the possibilities of remembrance, the project offers a solution that competes with history
Listening to my grandmother’s 1977 tapes, I find myself privy to my grandmother as a middle-aged woman, recounting her life story to her teenage daughter. The tapes form a markedly different testimony than the video of my grandmother in 1998, recounting for the Shoah Foundation, or the transcript of my grandmother in 1965, recounting as a witness in a German state trial against an SS officer, or the video of my grandmother in 2013, recounting to me as her teenage grandson. Each testimony is a snapshot of my grandmother at a different age, in a different context, with a different audience, offering a unique perspective on a portion of her life that continues to evolve in meaning as she ages. None of these recountings could be replicated by me speaking with my grandmother today or a hundred years in the future.
Where testimony is grounded in specificity, machine learning aspires for generality. A successful algorithm is one that can produce plausible responses to any interlocutor at any time in any place. In offering a machine-learning solution, Dimension in Testimony’s speculative and flexible algorithms subsume the singular and definitive nature of testimony.
That we cannot conjure the dead is a reality that we must accept. That we cannot summon people from different points in their lives, in different contexts, so too follows. When the last of the survivors are gone, we must therefore turn to what we already rely on: the testimonies they have recorded throughout their lives, and we must listen.