Code

Dist ref kl#529

Code

Open

Dahoas wants to merge 27 commits intomainCarperAI/trlx:mainfrom dist-ref-klCarperAI/trlx:dist-ref-klCopy head branch name to clipboard

+484-134

Commits

Commits on Jul 18, 2023

• Implementing support for dense rewards

  Dahoas

  committed
  f04446d

• added "num_return_sequences" param which corresponds to n in Best-of-N sampling

  

  SharathRaparthy

  authored and

  Dahoas

  committed
  13a01fc

• updates to "num_return_sequences" param

  

  SharathRaparthy

  authored and

  Dahoas

  committed
  5421a73

• BoN implementation

  

  SharathRaparthy

  authored and

  Dahoas

  committed
  2f3ac28

• Changed back to default.

  

  SharathRaparthy

  authored and

  Dahoas

  committed
  2f1dace

• TopK sampling instead of Top1

  

  SharathRaparthy

  authored and

  Dahoas

  committed
  f58170d

• summed along dim=1

  

  SharathRaparthy

  authored and

  Dahoas

  committed
  be8bc1a

• Generating samples in chunks

  

  SharathRaparthy

  authored and

  Dahoas

  committed
  608d812

• added gen_chunk_size parameter

  

  SharathRaparthy

  authored and

  Dahoas

  committed
  d8557e7

• chunking in forward prop

  

  SharathRaparthy

  authored and

  Dahoas

  committed
  8ef9c36

• chunking generations in train and eval

  

  SharathRaparthy

  authored and

  Dahoas

  committed
  4c1d82d

• Implementing support for dense rewards

  Dahoas

  committed
  ecd5107

• Fix distributed ref_mean, ref_var bug for dense rewards

  Dahoas

  committed
  4071604

• Make generation respect max seq length

  Dahoas

  committed
  5f41413

• Make experience before first round of training

  Dahoas

  committed
  22ae83f

• Refactoring .generate/.generate_eval

  Dahoas

  committed
  7d0a4be

• Fix BoN metric support

  Dahoas

  committed
  b79dd19

• Enforce chunk_size param for eval generation when present

  Dahoas

  committed
  cb49dc5

• Fix: Don't shuffle prompt dataset

  Dahoas

  committed
  e290412

• Move inputs to device

  Dahoas

  committed
  391d04c

• Fix style

  Dahoas

  committed
  8de84e4

• Fix: Do not shuffle empty experience dataloader

  Dahoas

  committed
  404ef14

• Make experience before first round of training

  Dahoas

  committed
  67b711a

• Refactoring .generate/.generate_eval

  Dahoas

  committed
  34e185a

• Refactored decode, make_experience and added support for external reference models

  Dahoas

  committed
  11e1e95

• Fix BoN sampling after big refactor

  Dahoas

  committed
  527ba23

• Fixing style

  Dahoas

  committed
  676a1cd