Edgar Cervantes / Android Authority
TL;DR
- Google has filed a trademark for RealFill technology.
- The technology will allow users to expand images based on up to five reference images.
- This should result in more accurate image expansion compared to other solutions.
Google has been aggressively promoting generative AI over the last 18 months or so, with the Magic Editor feature being one of the most prominent demonstrations of the tech. Now, it looks like the company’s next big AI-enabled photo feature could be RealFill.
Google quietly filed a trademark for so-called RealFill technology last week. The trademark was filed via the European Union Intellectual Property Office (EUIPO) and the US Patent and Trade Office (USPTO).
“Providing non-downloadable software using artificial intelligence (AI) for inpainting images; Providing online non-downloadable software for creating generative models,” reads a brief description of the trademark.
RealFill explained
It turns out that RealFill tech actually broke cover late last year in a paper and website by a team of researchers from Google and Cornell University. The paper, titled “Reference-Driven Generation for Authentic Image Completion,” describes a way to more accurately expand and inpaint images.
More specifically, RealFill is able to more accurately expand and inpaint an existing image by using up to five images as a reference:
These reference images do not have to be aligned with the target image, and can be taken with drastically varying viewpoints, lighting conditions, camera apertures, or image styles.
The team first fine-tunes a personalized generative AI model on the reference and target images. This process allows the model to learn the lighting, style, and contents of the scene in the images.
The results nevertheless speak for themselves, as seen above and below. The images below also show how RealFill pictures compare to other solutions, such as Stable Diffusion.
In saying so, the team did note a few limitations with RealFill. One notable downside is that it needs to undergo a “gradient-based fine-tuning process” on input images, which makes the process slow. It can also be tough to recover the scene in the final image if there’s a huge difference between the reference images and the target image. Furthermore, the researchers found that text could be an issue when using this technique.
Will we see this on the Pixel 9 or Google Photos?
Filed patents or trademarks aren’t a guarantee that RealFill will be a commercial reality. Nevertheless, it stands to reason that this could come to a future Pixel series phone and/or Google Photos if it is indeed planned for a commercial release.
We’re guessing this would likely be a cloud-based feature rather than an on-device photo editing option, especially since the team noted that the fine-tuning process is slow.
Current photo expansion and inpainting solutions are far from perfect, though, so a solution that uses reference images could still make for much better results. It also means users could theoretically go back to old snaps in their Google Photos library and generate better images.
Either way, this feature will likely raise more questions about the definition of a photo, much like Google’s Magic Editor has sparked a debate about the topic.
