Code

Add DPO support for DeepSpeed-Chat#828

Code

Merged

tjruwase merged 11 commits intodeepspeedai:masterdeepspeedai/DeepSpeedExamples:masterfrom stceum:dpo_supportstceum/DeepSpeedExamples:dpo_supportCopy head branch name to clipboardJan 6, 2025

+7,216

Commits

Commits on Jan 27, 2024

• Add label_smoothing while calculating step2 DPO loss in DeepSpeed-Chat.

  stceum

  committed
  ae1c11c

  Copy full SHA for ae1c11c

• Add training scripts for step2 DPO in DeepSpeed-Chat.

  stceum

  committed
  7efa35a

  Copy full SHA for 7efa35a

• Remove unused packages and format the code of step2 DPO in DeepSpeed-Chat.

  stceum

  committed
  27a8782

  Copy full SHA for 27a8782

• Update training scripts of step2 DPO in DeepSpeed-Chat.

  stceum

  committed
  b563ff9

  Copy full SHA for b563ff9

• Follow upstream fixes.

  stceum

  committed
  b5f0068

  Copy full SHA for b5f0068

Commits on Oct 30, 2024

• Merge branch 'master' into dpo_support

  loadams

  authored
  f146c26

  Copy full SHA for f146c26

Commits on Dec 18, 2024

• Merge branch 'master' into dpo_support

  tjruwase

  authored
  bfd895d

  Copy full SHA for bfd895d

Commits on Dec 27, 2024

• Merge branch 'master' into dpo_support

  stceum

  authored
  40c6e83

  Copy full SHA for 40c6e83

Commits on Jan 1, 2025

• Update README.md for Step2 DPO finetuning.

  stceum

  committed
  ec55e20

  Copy full SHA for ec55e20

Commits on Jan 2, 2025

• Add opt 350M training log demo for step 2 dpo finetuning in DeepSpeed-Chat.

  stceum

  committed
  42f4b6e

  Copy full SHA for 42f4b6e

Commits on Jan 3, 2025

• Address the formatting issue in step2 dpo finetuning in DeepSpeed-Chat.

  stceum

  committed
  5216b3d

  Copy full SHA for 5216b3d