Conversational dataset support for Online DPO

[qgallouedec]

What does this PR do?

Item of #2071

Changes:

Major

• Added conversational dataset support to Online DPO (and consequently to Nash-MD and XPO)
• Removed applying the chat template in the example script (XPO, Nash-MD and Online DPO)
• Added tests for conversational dataset support (XPO, Nash-MD and Online DPO)
• Added a better default learning rate for OnlineDPOConfig (consequently changes the default for XPOConfig and NashMDConfig) (similar to [KTO] learning rate recomentations for kto #2070)
• Changed the structure of the doc of XPO, Nash-MD and Online DPO to have
  1. Overview
  2. Quick start
  3. Usage tips
  4. Example script
  5. Logged metrics
  6. Autofocus of config and trainer

Minor

• Changed Nash MD to Nash-MD
• Added autodoc of LogCompletionsCallback
• Added the generation script for https://huggingface.co/datasets/trl-lib/ultrafeedback-prompt
• Other minor fixes

Before submitting

• This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
• Did you read the contributor guideline,
  Pull Request section?
• Was this discussed/approved via a GitHub issue? Please add a link
  to it if that's the case.
• Did you make sure to update the documentation with your changes? Here are the
  documentation guidelines.
• Did you write any new necessary tests?

Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors who may be interested in your PR.

[HuggingFaceDocBuilderDev]

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.