Making a convincing deepfake

October 12, 2020

Topic

Statistics  /  , ,

For MIT Technology Review, Karen Hao looks into the process of artists Francesca Panetta and Halsey Burgund to produce a deepfake of Richard Nixon reading an alternate history of the moon landing:

This is how Lewis D. Wheeler, a Boston-based white male actor, found himself holed up in a studio for days listening to and repeating snippets of Nixon’s audio. There were hundreds of snippets, each only a few seconds long, “some of which weren’t even complete words,” he says.

The snippets had been taken from various Nixon speeches, much of it from his resignation. Given the grave nature of the moon disaster speech, Respeecher needed training materials that captured the same somber tone.

Wheeler’s job was to re-record each snippet in his own voice, matching the exact rhythm and intonation. These little bits were then fed into Respeecher’s algorithm to map his voice to Nixon’s. “It was pretty exhausting and pretty painstaking,” he says, “but really interesting, too, building it brick by brick.”

Sounds like a lot of work, luckily.