Skip to content

Commit c07100b

Browse files
authored
Add encoder model support in trt-llm (#1294)
* add encoder * update: port * update: serving image builder * fmt * fmt * fmt * add variables for encoder * update default config settings * update: constants, serving image builder * update: serving image builder * update start command * update truss serving builder * add clearer separation between encoder and decoder paths * remove concurrenct overwrite * serving image builder: update predict concurrency * add: comments for max batch size * make serving image builder asserion * add predict endpopint as /v1/embeddings for faster testing
1 parent 9a029be commit c07100b

File tree

5 files changed

+97
-262
lines changed

5 files changed

+97
-262
lines changed

0 commit comments

Comments
 (0)