AutoModel class for image-text-to-text models

[merveenoyan]

Feature request

It would be nice to get a standard AutoModel class for image-text-to-text models (since @molbap is standardizing the processor)

Motivation

@NielsRogge noticed that in model repositories the automatic snippets fallback to AutoModelForPreTraining because these models don't exist in PIPELINE_TAGS_AND_AUTO_MODELS (due to lack of AutoClass) More importantly it would be nice to load it to a single class.

Your contribution

I haven't checked what it takes to implement an AutoClass when model classes exist in different names for the same task but if decided I don't mind looking into it and taking a stab.

[NielsRogge]

First attempt was at #29572, but is awaiting standardization of processors which is tracked at #31911

[amyeroberts]

cc @zucchini-nlp re VLMs

[yonigozlan]

Another blocker was that some models need custom processing code to be moved into their processors. I started #32059 and will get to work on checking which models need additional processing to standardize the inputs and outputs :).

[zucchini-nlp]

The PR for auto mapping was merged recently, guess we an close this?

[NielsRogge]

Yes this was added in #32472