Add image-text-to-text and edit image-to-text task pages

[merveenoyan]

No description provided.

[osanseviero]

Looking nice!

[osanseviero]

I would avoid to put examples of things that are covered in other task pages to avoid confusion. The 3 examples now are already covered. E.g.

• VQA https://huggingface.co/tasks/visual-question-answering
• Doc QA https://huggingface.co/tasks/document-question-answering
• Captioning https://huggingface.co/tasks/image-to-text

[merveenoyan]

I think I disagree here. one could segment humans with both image segmentation and mask detection for instance (except for zero shot part). some hugging face tasks are similar. some models are very generalistic that they can handle these tasks altogether and the paradigm is shifting more towards there anyway.

these models are similar either way, if I were an MLE who had to do captioning I'd like to try both VLMs (BTW which perform better IMO) and direct captioning models (image-to-text) which has sole purpose of captioning. I wouldn't like to keep this information from the user.

[pcuenca]

Maybe we can briefly explain this just after the Use Cases heading. Also, I think we haven't defined VLMs yet. For example:

These models are commonly called vision-language models, or VLMs. They can typically generalize to various types of tasks for which specialist models may also exist. For example, you can use a VLM to caption an image, or you can use specific captioning models as described in the image to text task page.

Edit: VLMs are indeed defined in data.ts, but I think it doesn't hurt to also mention it here.

[merveenoyan]

I think it would be too repetitive. See here for instance the one in data.ts will be at the top, and then right after use case one repeats it. I wouldn't like to add it there

[osanseviero]

My main concern is that now we have two tasks that cover the same thing, so it could end up confusing users. As an example, imagine if https://huggingface.co/tasks/text-generation use cases were Translation and Summarization, which are also their own separate tasks.

I wonder if we can use here specific applications rather than sub-tasks covered here (thinking of https://huggingface.co/tasks/text-generation#use-cases as a nice example)

[merveenoyan]

@osanseviero I think as a specific/solid use case there's not a lot (they'd be way too specific) or they'd eventually fall under visual question answering/retrieval

[merveenoyan]

an organic way for me to infer use cases for tasks were checking out example inputs to many Spaces based on a specific task, and for VLMs it's mostly visual question answering for instance.

[merveenoyan]

I added bunch of other things

[pcuenca]

Fixed a couple of formatting issues. Agree with Omar's comments :)

[pcuenca]

TODO

[merveenoyan]

fyi @pcuenca this PR doesn't have to be merged until Niels' pipeline PR is merged imo