Varlen attention vs multipack masking

[euclaise]

At the current ollama event, multipack was mentioned as using masking to prevent cross-sequence attention.

However, it seems to me like this needlessly wastes memory - since attention memory/compute is quadratic with seq len, it should be more efficient to run N attentions batched in parallel.

It seems that flash-attn already supports variable-length batching, and its varlen functions are even used in the patch scripts.

Is there a reason to prefer masking that I'm missing?