Using overlays

[avidseeker]

Section 2.3 of avif spec says that it supports multi-layered images or overlays.

Is this featured supported by libavif?

E.g usage: how can I list all layers within an avif file? How do I merge separate images 1.avif, 2.avif, ... into layer1 layer2, ... of out.avif

Stack exchange question: https://graphicdesign.stackexchange.com/questions/166214/

[maryla-uc]

libavif does have experimental support for layers.

AVIF layers

First, let me clarify what AVIF layers are so that we're on the same page.

Layers are a feature from the AV1 video codec that AVIF is based on, that was designed as a way to provide multiple qualities and/or sizes for a given frame. For example, you could have a first layer that is a smaller size, and a second layer that is full size. This way a device with low capabilities, or with a small screen, could decide to only show the first layer.

For images, the main use case is for "progressive decoding", so that the software can display the first low quality/small layer early, then when it's done receving all the bytes for the second layer (full image) it can decode and display it.

Layers in AVIF are not overlays. They are not at all like Photoshop layers.

For support in libavif, you can search avif.h for the words layer and progressive.

Encoding

To encode an image with N layers using the API, set extraLayerCount to N-1, then call avifEncoderAddImage() N times. Between each call, you can change some encoder settings like quality or scalingMode.

You can do the same thing with the avifenc command line, using the --layered flag, e.g:

avifenc --layered -q 10 --scaling-mode 1/2 myimage.jpg -q:u 80 --scaling-mode:u 1/1 myimage.jpg output.avif -a end-usage=cbr

The -a end-usage=cbr flag is to make sure the size of the image doesn't change too much, see #1667
Note that in this case the input image for both layers is the same, only the quality/size changes, but you could also provide different file inputs. For example, the first layer could be a blurred version of the image.

Alternatively, you can simply use the --progressive flag which automatically creates two layers where the first one is very low quality, and therefore very small and can be displayed quickly.

Decoding

At decoding time, using the API you can set allowProgressive to decode all layers one by one. If not set, only the last layer (usually highest quality) is decoded. Software that supports progressive decoding would typically display each layer as it becomes available, replacing completely the previous layer. This is again not the same as overlays/Photoshop layers where all layers are blended into a single image before displaying.

With avifdec, you can similarly set the --progressive flag and --index to specify the index of the layer you want to decode.

[avidseeker]

For images, the main use case is for "progressive decoding",

Layers in AVIF are not overlays. They are not at all like Photoshop layers.

But according to this comment: AOMediaCodec/av1-avif#155 (comment)

Overlays are a container level feature... allows you to specify a canvas of a
specific size and then allows compositing images into this canvas

The AV1 codec supports multi-layer streams. This can be used to do progressive decoding.

So it seems that avif supports both layer overlays and progressive decoding. But
does libavif support both as well?

[maryla-uc]

Thanks for the link. Indeed, the feature I was talking about was the second one in the message you linked, "Multi-layer streams".

The first feature listed, "overlays", is indeed part of the HEIF/MIAF specification, but it's not supported at all by libavif, either for encoding or decoding. The AVIF specification does not mention this feature but does not forbid it either.

It looks like libheif supports overlays, although I haven't tried it.