Code

Features integration without fp8#7

Code

Merged

gshtras merged 39 commits intomainROCm/vllm:mainfrom integration_no_fp8ROCm/vllm:integration_no_fp8Copy head branch name to clipboardMar 22, 2024

+1,515-88

Commits

Commits on Jan 27, 2024

• [ROCm] Fixup arch checks for ROCM

  Show description for 9c2367c
  dllehr-amd

  committed
  9c2367c

  Copy full SHA for 9c2367c

• yapf cleanup

  dllehr-amd

  committed
  a9d752c

  Copy full SHA for a9d752c

Commits on Feb 10, 2024

• Initial port of gradlib gemm tuner

  dllehr-amd

  committed
  20b5f10

  Copy full SHA for 20b5f10

Commits on Feb 11, 2024

• Enable torchrun vs Ray

  Show description for 6f28107
  dllehr-amd

  committed
  6f28107

  Copy full SHA for 6f28107

• Add custom matvec kernels and sampler matmul call tuned_gemm

  dllehr-amd

  committed
  184806e

  Copy full SHA for 184806e

Commits on Feb 14, 2024

• Add silu gemm fusion when batch and seq_len = 1

  dllehr-amd

  committed
  af9e9d1

  Copy full SHA for af9e9d1

• Add tunable flags to VLLM

  dllehr-amd

  committed
  5f8eac3

  Copy full SHA for 5f8eac3

• Allow benchmark_latency to take a list of input/output/batches for faster execution

  Show description for 22766b4
  dllehr-amd

  committed
  22766b4

  Copy full SHA for 22766b4

Commits on Feb 15, 2024

• Add dynamic tuning feature to vllm

  dllehr-amd

  committed
  87b4c1b

  Copy full SHA for 87b4c1b

• Add rpd tracer controls to benchmark_latency.py

  dllehr-amd

  committed
  694ae1d

  Copy full SHA for 694ae1d

Commits on Feb 22, 2024

• Fix Dockerfile errors

  dllehr-amd

  committed
  0e73aed

  Copy full SHA for 0e73aed

• Add llama2 run script

  dllehr-amd

  committed
  90df0c9

  Copy full SHA for 90df0c9

• Increase Partition and Num threads for attention blocks

  dllehr-amd

  committed
  ab67280

  Copy full SHA for ab67280

• Fix WORKDIR

  dllehr-amd

  committed
  1d53722

  Copy full SHA for 1d53722

• Add accuracy flag to benchmark_latency.py

  dllehr-amd

  committed
  5148aa5

  Copy full SHA for 5148aa5

Commits on Feb 26, 2024

• Don't broadcast when using torchrun

  dllehr-amd

  committed
  534dcff

  Copy full SHA for 534dcff

Commits on Mar 15, 2024

• Adding new rocm triton flash attention kernel

  Show description for e569133
  

  jpvillam-amd

  and

  vgokhale

  committed
  e569133

  Copy full SHA for e569133

Commits on Mar 18, 2024

• Merge remote-tracking branch 'origin/main' into v0.3.3_greg

  gshtras

  committed
  8cb05bc

  Copy full SHA for 8cb05bc

• Removed gradlib and its tuned gemm in favor of tunable ops

  gshtras

  committed
  be708d0

  Copy full SHA for be708d0

Commits on Mar 19, 2024

• Small fix on dockerfile

  jpvillam-amd

  committed
  0e63661

  Copy full SHA for 0e63661

• Merge remote-tracking branch 'origin/main' into jpvillam/v0.3.3_triton

  jpvillam-amd

  committed
  c89c0e3

  Copy full SHA for c89c0e3

• Rebase updates and PR review changes

  Show description for d4cb905
  jpvillam-amd

  committed
  d4cb905

  Copy full SHA for d4cb905

Commits on Mar 21, 2024

• Introducing torchrun multi GPU support

  gshtras

  committed
  bc750fa

  Copy full SHA for bc750fa

• Update dockerfile

  gshtras

  committed
  a83b7ea

  Copy full SHA for a83b7ea

• Merge branch 'greg/torchrun' into integration_no_fp8

  gshtras

  committed
  2ff59d7

  Copy full SHA for 2ff59d7

• Merge remote-tracking branch 'origin/jpvillam/v0.3.3_triton' into integration_no_fp8

  gshtras

  committed
  04fd3fb

  Copy full SHA for 04fd3fb

• add use case for custom kernel for matvec operation

  

  charlifu

  authored and

  gshtras

  committed
  e01b8cd

  Copy full SHA for e01b8cd

• limit the custom kernel under is_hip

  

  charlifu

  authored and

  gshtras

  committed
  eb21ad7

  Copy full SHA for eb21ad7

• fix custom kernel

  

  charlifu

  authored and

  gshtras

  committed
  1bf736f

  Copy full SHA for 1bf736f

• Rocm defaults and cleanup

  gshtras

  committed
  7ab4a24

  Copy full SHA for 7ab4a24

• Remove ignored file

  gshtras

  committed
  fbea667

  Copy full SHA for fbea667

• Refactor torchrun executor to reuse single gpu executor code

  gshtras

  committed
  c8fce27

  Copy full SHA for c8fce27

Commits on Mar 22, 2024

• Added interleaving for MQA for triton kernel

  

  jpvillam

  committed
  1fff99a

  Copy full SHA for 1fff99a

• linter

  gshtras

  committed
  0a2309a

  Copy full SHA for 0a2309a

• Merge remote-tracking branch 'origin/jpvillam/v0.3.3_triton' into integration_no_fp8

  gshtras

  committed
  1ec6554

  Copy full SHA for 1ec6554

• Making torchrun the default multi GPU executor on ROCm unless overridden by --worker-use-ray

  gshtras

  committed
  44c2cee

  Copy full SHA for 44c2cee

• Make triton the default FA

  jpvillam-amd

  committed
  1256bee

  Copy full SHA for 1256bee

• Make workaround only applicable to triton path

  jpvillam-amd

  committed
  b687795

  Copy full SHA for b687795

• Pin ray version to 2.9.3

  gshtras

  committed
  5e3ec52

  Copy full SHA for 5e3ec52