Code

Add LLM support to Brute Search#769

Code

Merged

nv-braf merged 23 commits intoadd-llm-modetriton-inference-server/model_analyzer:add-llm-modefrom brute-search-llm-changestriton-inference-server/model_analyzer:brute-search-llm-changesCopy head branch name to clipboardOct 12, 2023

+620-214

Commits

Commits on Oct 5, 2023

• Initial coding complete

  nv-braf

  committed
  49b4e15

  Copy full SHA for 49b4e15

• First unit test passing

  nv-braf

  committed
  43d5c1d

  Copy full SHA for 43d5c1d

• Adding test for prompt length

  nv-braf

  committed
  d765027

  Copy full SHA for d765027

• Refactor PACG methods

  nv-braf

  committed
  c198e5a

  Copy full SHA for c198e5a

• Further refactoring

  nv-braf

  committed
  79aa02a

  Copy full SHA for 79aa02a

• Ensure early exit isn't enabled for LLM models

  nv-braf

  committed
  ac81a6b

  Copy full SHA for ac81a6b

Commits on Oct 6, 2023

• Fix type checking errors

  nv-braf

  committed
  015a2c2

  Copy full SHA for 015a2c2

Commits on Oct 7, 2023

• Attempt at fixing codeql issue

  nv-braf

  committed
  2619b83

  Copy full SHA for 2619b83

Commits on Oct 10, 2023

• Revert "Attempt at fixing codeql issue"

  Show description for 9f2a065
  nv-braf

  committed
  9f2a065

  Copy full SHA for 9f2a065

• Attempt at codeQL fix

  nv-braf

  committed
  c5b702e

  Copy full SHA for c5b702e

• Adding deepcopy back in

  nv-braf

  committed
  cbdc746

  Copy full SHA for cbdc746

• Removing deepcopy in an attempt to fix codeQL errors

  nv-braf

  committed
  0c909ea

  Copy full SHA for 0c909ea

Commits on Oct 11, 2023

• Update model_analyzer/config/input/config_command_profile.py

  Show description for 3f4450a
  

  nv-braf

  and

  nv-hwoo

  authored
  3f4450a

  Copy full SHA for 3f4450a

• Update model_analyzer/config/generate/perf_analyzer_config_generator.py

  Show description for c69b577
  

  nv-braf

  and

  nv-hwoo

  authored
  c69b577

  Copy full SHA for c69b577

• Update model_analyzer/config/generate/perf_analyzer_config_generator.py

  Show description for b1eed54
  

  nv-braf

  and

  nv-hwoo

  authored
  b1eed54

  Copy full SHA for b1eed54

• Update model_analyzer/config/generate/perf_analyzer_config_generator.py

  Show description for a2fa148
  

  nv-braf

  and

  nv-hwoo

  authored
  a2fa148

  Copy full SHA for a2fa148

• Moving location of method

  nv-braf

  committed
  c96d897

  Copy full SHA for c96d897

• Changing parameter to inference load

  nv-braf

  committed
  daee4cc

  Copy full SHA for daee4cc

• Changing parameter to inference load

  nv-braf

  committed
  3966c6c

  Copy full SHA for 3966c6c

• Changing prompt length to text input length

  nv-braf

  committed
  fbe1abf

  Copy full SHA for fbe1abf

• Changing max_tokens to use request-parameter

  nv-braf

  committed
  abec25d

  Copy full SHA for abec25d

Commits on Oct 12, 2023

• Fix input-data typo

  nv-braf

  committed
  f8729db

  Copy full SHA for f8729db

• Changing non-parameter to parameter

  nv-braf

  committed
  2cda3df

  Copy full SHA for 2cda3df