On January 20, 2017, the day of Donald Trump’s inauguration, white supremacist dandy Richard Spencer got punched in the face. This occurred during an impromptu interview, just as Spencer was about to explain the symbolic significance of the totem known as Pepe the Frog. At that moment, an anti-fascist protester in a black bandana appeared from nowhere and assailed him with a swift right hook before disappearing into the chaos of the D.C. streets.
The incident was captured on video and widely shared on social media. The pleasure it offered for many viewers was cinematic in scope; all this scene of smartphone cinéma vérité needed was a soundtrack.
“The ear goes more toward the within, the eye toward the outer,” French director Robert Bresson wrote in “Notes on Sound.” Thus the seductive power of musical accompaniment, and the Spencer punch’s lack of it raised an irresistible question: If this was a movie scene, what song would fit it best? Its audience produced a range of answers in the form of video clips on Twitter and YouTube, and the same punch footage was scored to everything from New Order’s “Blue Monday” to Thin Lizzy’s “The Boys Are Back in Town” to Kanye West’s “Father Stretch My Hands, Pt. 1.” It produced so many iterations that it became recognizable as a meme, falling into the broader online-video genre of audiovisual remixes.
The mixed-media dub forces a lack of alignment between eye and ear, even if both aural and visual aspects appear unified
If one can create a new text by juxtaposing the same image with different sounds, it’s also possible to do the reverse: use the same music to bring a specific idea to a range of footage. For example the Curb Your Enthusiasm theme can be applied to any video to imply (or underscore) its depiction of an awkward social situation. In its setting to music, documentary becomes comedy, as poetry can become song.
These videos are sometimes called “mashups” in tribute to a Napster-era fad that superimposed one song’s vocal on another’s backing track. But the rescored videos are not simply layered sounds; they are mixed media. In replacing one soundtrack for another, they evoke not a mashing together but a dubbing over, as when a movie is dubbed into another language.
While dubbed movies usually try to invisibly make the same narrative comprehensible to viewers in other national markets, a dub can also be a creative act. In Jamaican reggae, dub refers to a production approach that spawned its own genre. Dick Hebdige describes its origin in Cut ‘n’ Mix:
One day, King Tubby, a record engineer, was working in his studio mixing a few ska “specials” (i.e., exclusive recordings) for Six Coxsone’s Downbeat system. He began fading out the instrumental track, to make sure that the vocals sounded right. And he was excited by the effect produced when he brought the music back in. So instead of mixing the specials in the usual way, he cut back and forth between the vocal and instrumental tracks and played with the bass and treble knobs until he changed the original tapes into something else entirely.
The Spencer punched to music meme and the overdubbed Curb Your Enthusiasm theme are iterations of both forms of the dub. In some of the punch videos, like in dub reggae, a hierarchy is deliberately inverted: The image is made to adapt to the music as the vocalist’s melody is made to accompany the bass line. In the “Blue Monday” dub, jump cuts and replays make the punch fit the thump of the Oberheim DMX drum machine in the song’s beat. But even without such revisions, the effect of the mixed-media dub severs the neat link that composer and film theorist Michel Chion calls the audiovisual contract, the agreement that sound and image create “a single entity.” The dub version forces a lack of alignment between eye and ear, even if both aural and visual aspects appear unified.
In the Richard Spencer meme, the ordinary sounds of the street obscured the latent affective qualities of the scene: humor, triumph, aggression. They emerged sonically, released by song
Scattered precedents for this method exist. Kenneth Anger’s Scorpio Rising matched stock footage with pop songs: Jesus wanders Jerusalem to the Shangri-La’s “He’s a Rebel.” In Can Dialectics Break Bricks, situationist filmmaker René Viénet dubbed quotations from the works of Mikhail Bakunin and Wilhelm Reich over Tu Guangqi’s 1972 Hong Kong martial arts film Crush. Chion coined the term synchresis, a neologism uniting synchronism and synthesis, to describe “the spontaneous and irresistible weld” between simultaneous audio and video in cinema. “This join,” he argued, “results independently of any rational logic.” Synchresis, operating beyond rationality, makes meaning instead through a suffusion of affective charge that can be in either harmony or counterpoint. While Hollywood relies on rational associations to manipulate viewers’ emotions, synchresis forces active engagement.
The Spencer and Curb Your Enthusiasm clips juxtapose pre-existing video and audio, but the dub version in online video can also take the form of adding original music, dialogue, or noise. The “Bad Lip Reading” series is closest to the cinematic practice — adding a synchronized voice-over in which the speaker is made to say something different from the original. “Literal Music Videos” dubs a sung narration of the video’s imagery in place of the original lyrics.
The greatest artist of the video dub version, though, is Santeri Ojala, a Finnish media artist known by the username StSanders. His best-known work is the “Shreds” series, satirizing videos of virtuoso guitarists: Eric Clapton, Carlos Santana, and other rock dinosaurs. Like a foley artist enhancing the sound of action in cinema, Ojala dubs not dialogue but noise, turning these “shredding” guitar heroes into buffoons putting on amateurish, cacophonic performances. Like King Tubby, StSanders discovered his art accidentally, opening a video of guitarist Steve Vai with the sound muted. Bresson famously pointed out that “the sound film made silence possible.” Silence, then, has made the dub version possible.
The Shreds videos disrupt each of the three categories of sound in film — the onscreen (made by a visible source), the offsceen (coming from an unseen source within the film’s world), and the nondiegetic (outside the world of the film entirely). Though none of the sound in Ojala’s work originates from the actual videos, they are rigorously diegetic, more so than the original — no sounds are heard that are not represented by instruments visible onscreen. The audiovisual contract is adhered to without deviation, to the point that its treacherousness is revealed.
The dub version in online video has liberated sound in a way that cinema usually avoids. As Chion points out, the power of sound tends to affect viewers in secret. This is most clearly expressed in the archetypal scene of the punch, which he describes as “the most immediate audiovisual relationship”: “We do not really see the punch; you can confirm this by cutting the sound out of a scene. What we hear is what we haven’t had time to see.” For this reason, a foley artist must provide a sound if none was obtained by the microphone. “Most falls, blows, and explosions on the screen, simulated to some extent or created from the impact of nonresistant materials, only take on consistency and materiality through sound,” Chion writes.
That violent thud expresses only one aspect of the scene. An original soundtrack is set within limits that a dub version can transgress. In the Shreds videos, the absurdity of the musician’s countenance and movement is revealed once the veil of virtuosity is pulled aside. In the Richard Spencer meme, the ordinary sounds of the street and the dull impact of flesh against flesh obscured the latent affective qualities of the scene: humor, triumph, aggression. They emerged sonically, released by song. A punch, Chion points out, may not make a sound in reality. But in cinema, it must.