Nvidia has been researching the possible uses of neural networks intensively for years. One of the results is the GauGAN software, which was shown for the first time in 2019 and is intended to generate realistic landscapes from simple color sketches. However, the corresponding technology was only released this June: Since then, owners of an RTX graphics card have been able to try out GauGAN as part of the “Nvidia Canvas” beta.
GauGAN 2 with text as an addition
The results of GauGAN are impressive but mostly easy to recognize as computer-generated content. With the successor GauGAN 2 just presented, this is supposed to be a little better and the possible use more flexible. Because according to Nvidia, GauGAN 2 can no longer only be fed with rough sketches of the desired landscape, but also with a description text.
A demo video published by Nvidia shows how this ideally works. The user makes a (very) rough sketch of the desired landscape, enters a descriptive text such as “foggy island” and the result is a relatively realistic picture. There are also other options, for example, to edit existing images. Even in the demo video, it can often be seen that it is not a photo or a good 3D simulation: GauGAN 2 also only has limited options. The web version provided by Nvidia is definitely exciting.
Nvidia also has a few details ready for creating the new software. Accordingly, GauGAN 2 was trained with more than ten million landscape images including descriptions. The Nvidia supercomputer Selene, which is currently number 6 on the list of the fastest mainframes, was used.