OpenAI launched its video-generating tool, Sora, on Monday. But the company’s opting not to release a key feature for most users pending further testing.
The feature in question generates a video using an uploaded photo or footage of a real person as a reference. OpenAI says that it’ll give a “subset” of Sora users access to it, but that it won’t roll out the capability broadly until it has a chance to fine-tune its “approach to safety.”
“The ability to generate a video using an uploaded photo or video of a real person as the ‘seed’ is a vector of potential misuse that we are taking a particularly incremental approach toward to learn from early patterns of use,” OpenAI wrote in a blog post. “Early feedback from artists indicate that this is a powerful creative tool they value, but given the potential for abuse, we are not initially making it available to all users.”
Generative video is a powerful tool — and a controversial one. Deepfakes are a concern, of course, as is misinformation. According to data from ID verification service Sumsub, deepfake fraud worldwide increased by more than 10 times from 2022 to 2023.
Among other steps OpenAI says it’s taking to prevent misuse, Sora has a filter to detect whether a generated video depicts someone under the age of 18. If it does, OpenAI applies a “stricter threshold” for moderation related to sexual, violent, or self-harm content, the company claims.
All Sora-generated videos contain metadata to show their provenance — specifically metadata that abides by the C2PA technical standard. The metadata can be removed, granted. But OpenAI’s pitching it as a way for platforms that support C2PA to quickly detect whether a video originated from Sora.
In a bid to fend off copyright complaints, OpenAI also says that it’s using “prompt re-writing” to prevent Sora from generating videos in the style of a living creator.
“We have added prompt re-writes that are designed to trigger when a user attempts to generate a video in the style of a living artist,” the company wrote. “We opted to take a conservative approach with this version of Sora as we learn more about how Sora is used by the creative community … There is a very long tradition in creativity of building off of other artists’ styles, but we appreciate that some creators may have concerns.”
A number of artists have sued AI companies, including OpenAI, over allegedly training on their works without permission to create AI tools that regurgitate content in their unique styles. The companies, for their parts, have claimed that fair use doctrine protects them from copyright infringement claims, and that AI models don’t, in fact, regurgitate.
According to video blogger Marcus Brownlee, who got an early preview of the features, Sora can create videos from a text prompt or image and edit existing videos via a Re-mix tool. A Storyboard interface lets users create sequences of videos, while a Blend tool takes two videos and creates a new one that preserves elements of both.
Subscribers to OpenAI’s ChatGPT Pro and Plus plans get access to Sora, OpenAI’s video generator, today — but only if they live in certain countries.
