Rebs becomes cat Booth, Pikachu becomes dog Kachu： Nvidia’s latest AI open source, all things change faces with only one picture ｜ Demo
Qubit, report | official account QbitAI
Nvidia’s new AI can quickly turn a golden retriever in a video into a meerkat.
Just show the AI two static pictures of meerkats:
Before seeing these two images, the AI had never seen an animal like a meerkat, not in the training dataset.
There were many other animals that the AI had never seen before. Just give it a photo or two to familiarize it with, and you can replace the golden hair with their faces.
For example, ferocious big cats:
In this way, the Golden Retriever’s relatives, such as huskies, are even more concerned:
Although the same translation is from picture to picture, and the same is unsupervised, this AI is very different from the predecessors of the horse-turned-zebra. The predecessors can only translate between animals they have seen. During training, I have seen many horses and many zebras;
And NVIDIA’s new contestant is equivalent to never seeing a zebra in training. Animals that are not in the training dataset, as long as they can see one or two pictures during the test, it is enough.
The Few-Shot Learning algorithm greatly reduces the requirements for training data.
The team has open-sourced the algorithm and launched a one-click face-changing demo. So, let’s play with it today and then talk about the principle.
The wrong and correct way to open
I tried the demo on the chicken freezer.
The first question is to pass a cold meow up and select the head with the box.
?? Thank you to my old Si Xia
Therefore, all kinds of animals have gained a cold side.
The second question, passed a Rebs up, which was also half-sided:
Unexpectedly, the orcs who generated the aura of nobility:
AI’s artistic talent is upon us, but I still want to solemnly remind you:
The demo application is called PetSwap. Please select the correct opening method. Here is the correct demonstration.
The third question is to break through the dimensional wall. Go, Detective Pikachu:
Everyone donned Sherlock Holmes hats, but only the hound in the upper right corner unlocked the sexy blush.
What an elegant and fulfilling morning.
Play enough, it’s time to see the principle.
Never seen animals, just take a look
As mentioned at the beginning, this is a small sample translator. It needs to replace a Content Image, such as a golden retriever, with a Class Image, such as a husky or a cougar.
The model is divided into three parts:
One is the Content Encoder, the pink part. It maps the input content map (Golden Retriever) to a Content Latent Code.
The second is the class encoder (Class Encoder), the green part. First map each class map (Husky) to a code. Then take an average to get the entire class code.
The third is the decoder (Decoder), the blue part. First map the category lattice code to the adaIN parameter, and then decode the content lattice code to generate the translation result map.
When training, there are golden retrievers and huskies in the dataset. The AI is between these categories and cultivates the face-changing skill:
However, there are no cougars in the training dataset. What should I do to generate cougars during testing?
Here’s the secret: When the class diagram and the content diagram look the same, let the model generate a reconstruction.
In this way, as long as you temporarily look at a cougar (or a few) during the test, you can turn the unseen moment into a seen one. It is not difficult for the AI to translate the dog in the training dataset into a cougar:
You’re done.
Go play, too!
Such a (excellent) show of AI, you must also want to train it.
The code, demo, and paper are all here:
Demo Portal:
https://nvlabs.github.io/FUNIT/petswap.html
Paper Portal:
https://arxiv.org/abs/1905.01723
Code Portal:
https://github.com/nvlabs/FUNIT/
Home Portal:
https://nvlabs.github.io/FUNIT/
This article was first published on WeChat official account: Qubit. The content of the article belongs to the author’s personal opinion and does not represent the position of Hexun.com. Investors operate accordingly, please bear the risk.