Skybox generation

Skybox generation

This tasks uses a simple combination of models: image generation and inpainting. The idea is to use inpainting for image-to-panorama generation by rewriting the image borders. Then, using two projections, a panorama-to-skybox is acheived.

Text-to-image

The next image is an AI-generated landscape of a sunny mountain, using StableDiffusion XL 1.0 (base only).

.

Image-to-panorama

We want to create an horizontal panorama from the image, so that no seam line appears at the image borders. This is relatively easy when we formulate the issue as the creation of an asymetric tiling tile. The idea is to simply inpaint the borders of the image to make them match.

With the image and the mask, the inpainted result is:

Projected in the initial view:

Panorama-to-skybox

For this step, we use polar projections first, but the idea is the same.

For the ground, we extend the lower part of the image as a base for the inpainting.

We apply a classical inpainting.

Now we apply a reverse polar projection.

The same procedure is applied for the sky. The main difference is that we select pixels from the sky before the inpainting, other tall elements would be dupplicated.