Skip to content

tensorrt-llm 0.12.0.dev2024073000, triton 2.46.0 #52

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Open
yorickvP wants to merge 35 commits into
base: main
Choose a base branch
from
Open

tensorrt-llm 0.12.0.dev2024073000, triton 2.46.0 #52

yorickvP wants to merge 35 commits into from

Conversation

@yorickvP
Copy link
Contributor

@yorickvP yorickvP commented Aug 8, 2024

Breaking changes:

  • enable_trt_overlap now useless
  • max_queue_size is now required, 0 is a good default
  • max_seq_len required until next week, but will default to the model max input size after

Other changes:

  • removed bls

yorickvP and others added 30 commits July 19, 2024 13:54
@yorickvP
tensorrt-llm: 0.9.0 -> 0.10.0, triton: 2.42.0 -> 2.44.0
d12ae7b
@yorickvP
update cognix
c3eb6f3
@yorickvP
update triton_templates to trtllm-0.10
cec6d28
@yorickvP
don't tokenize in postprocessing (see #27)
0626e44
@yorickvP
instantiate triton_model_repo with the default config
4cd5533
@yorickvP
Adjust decoding_mode after testing, omit missing optional params
25cb314 
Remove the tensorrt_llm python script, since it confuses
`maybe_download_tarball_with_pget`
@yorickvP
ignore empty SSE's, remove decoding_mode, just omit topk instead
a19b28b
@yorickvP
bump pget==0.8.2, cog==0.10.0-alpha16
d86927d
@yorickvP
tensorrt-llm: decrease closure size by cleaning up kernels
098f7e7
@yorickvP
bump cognix to exclude train: null
cd3ce0e
@yorickvP
tensorrt-llm: 0.10.0 -> 0.11.0
e5ac906
@yorickvP
update triton_templates
57e97ee
@yorickvP
update triton_model_repo
7129024
@yorickvP
Add a $TRTLLM_PYTHON environment with pydantic 2
8a2170f
@technillogue @yorickvP
use pytriton 0.5.8
494d476
@yorickvP
add nvidia-modelopt setuptools dependency
5c3eded
@yorickvP
datasets: 2.14.4 -> 2.20.0
dfd19b4 
hopefully fixes
`ValueError: Invalid pattern: '**' can only be an entire path component`
@yorickvP
update cognix
d32b60c
@yorickvP
tensorrt-llm: 0.11.0 -> 0.12.0.dev2024072300
a2fc518
@yorickvP
update triton_model_repo, triton_templates
8cc2174
@yorickvP
don't add 2 copies of tensorrt-llm in builder
92000d5
@yorickvP
tensorrt-lllm: 0.12.0.dev2024072300 -> 0.12.0.dev2024072301
dc78a43
@yorickvP
github action: limit to 12 cores
5fb8fb7
@yorickvP
remove rogue .gitkeep files
6888d8f
@yorickvP
tensorrt-cu12: 10.2.0 -> 10.2.0.post1
49ccca6 
Should fix #51
@yorickvP
Push builder to triton-builder-h100
73567de
@yorickvP
fix: work around replicate unique model check
d2adf71 
adds TRTLLM_BUILDER_VARIANT=h100 to work around
"It looks like a copy of this model version already exists on Replicate"
@yorickvP
build tensorrt-llm from source
2805da2
@yorickvP
trtllm: fix patch location to work in nix develop
d10ce53
@yorickvP
Add support for building torch & trtllm(python) from source
95e44d8
@yorickvP yorickvP requested a review from joehoover August 8, 2024 14:04
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@joehoover joehoover Awaiting requested review from joehoover

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@yorickvP @technillogue