{"payload":{"feedbackUrl":"https://github.com/orgs/community/discussions/53140","repo":{"id":654122609,"defaultBranch":"main","name":"lmdeploy","ownerLogin":"InternLM","currentUserCanPush":false,"isFork":false,"isEmpty":false,"createdAt":"2023-06-15T12:38:06.000Z","ownerAvatar":"https://avatars.githubusercontent.com/u/135356492?v=4","public":true,"private":false,"isOrgOwned":true},"refInfo":{"name":"","listCacheKey":"v0:1726197152.0","currentOid":""},"activityList":{"items":[{"before":"82d0c00b7120afba2943cebea168de1bc6ba0174","after":"8b9f6abcaaa85e1f128e683079a39e7665d5940d","ref":"refs/heads/main","pushedAt":"2024-09-20T03:50:58.000Z","pushType":"pr_merge","commitsCount":1,"pusher":{"login":"lvhan028","name":"Lyu Han","path":"/lvhan028","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/4560679?s=80&v=4"},"commit":{"message":"Add max_log_len option to control length of printed log (#2478)\n\n* check max_batch_size\r\n\r\n* add max_log_len option to control length of printed log\r\n\r\n* minor fix\r\n\r\n* remove autotest/README.md\r\n\r\n* missed docstring\r\n\r\n* change log format","shortMessageHtmlLink":"Add max_log_len option to control length of printed log (<a class=\"issue-link js-issue-link\" data-error-text=\"Failed to load title\" data-id=\"2533595859\" data-permission-text=\"Title is private\" data-url=\"https://github.com/InternLM/lmdeploy/issues/2478\" data-hovercard-type=\"pull_request\" data-hovercard-url=\"/InternLM/lmdeploy/pull/2478/hovercard\" href=\"https://github.com/InternLM/lmdeploy/pull/2478\">#2478</a>)"}},{"before":"97449e3c5d8c5290c751271696c66349908d6695","after":"82d0c00b7120afba2943cebea168de1bc6ba0174","ref":"refs/heads/main","pushedAt":"2024-09-20T03:50:36.000Z","pushType":"pr_merge","commitsCount":1,"pusher":{"login":"lvhan028","name":"Lyu Han","path":"/lvhan028","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/4560679?s=80&v=4"},"commit":{"message":"adjust schedule to improve TTFT in pytorch engine (#2477)","shortMessageHtmlLink":"adjust schedule to improve TTFT in pytorch engine (<a class=\"issue-link js-issue-link\" data-error-text=\"Failed to load title\" data-id=\"2533321171\" data-permission-text=\"Title is private\" data-url=\"https://github.com/InternLM/lmdeploy/issues/2477\" data-hovercard-type=\"pull_request\" data-hovercard-url=\"/InternLM/lmdeploy/pull/2477/hovercard\" href=\"https://github.com/InternLM/lmdeploy/pull/2477\">#2477</a>)"}},{"before":"a513526d9b96b83693609274dbbdc4dbb1ae385b","after":"97449e3c5d8c5290c751271696c66349908d6695","ref":"refs/heads/main","pushedAt":"2024-09-19T08:53:58.000Z","pushType":"pr_merge","commitsCount":1,"pusher":{"login":"lvhan028","name":"Lyu Han","path":"/lvhan028","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/4560679?s=80&v=4"},"commit":{"message":"Add silu mul kernel (#2469)\n\n* add silu kernel, optimize apply rotary kernek\r\n\r\n* add ut\r\n\r\n* lint\r\n\r\n* fix fused moe","shortMessageHtmlLink":"Add silu mul kernel (<a class=\"issue-link js-issue-link\" data-error-text=\"Failed to load title\" data-id=\"2526309889\" data-permission-text=\"Title is private\" data-url=\"https://github.com/InternLM/lmdeploy/issues/2469\" data-hovercard-type=\"pull_request\" data-hovercard-url=\"/InternLM/lmdeploy/pull/2469/hovercard\" href=\"https://github.com/InternLM/lmdeploy/pull/2469\">#2469</a>)"}},{"before":"4305ddfa6d97ccaabf3b6e2bc3c34a214cde8f67","after":"a513526d9b96b83693609274dbbdc4dbb1ae385b","ref":"refs/heads/main","pushedAt":"2024-09-19T08:52:44.000Z","pushType":"pr_merge","commitsCount":1,"pusher":{"login":"lvhan028","name":"Lyu Han","path":"/lvhan028","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/4560679?s=80&v=4"},"commit":{"message":"fix ascend atten_mask (#2483)","shortMessageHtmlLink":"fix ascend atten_mask (<a class=\"issue-link js-issue-link\" data-error-text=\"Failed to load title\" data-id=\"2535177878\" data-permission-text=\"Title is private\" data-url=\"https://github.com/InternLM/lmdeploy/issues/2483\" data-hovercard-type=\"pull_request\" data-hovercard-url=\"/InternLM/lmdeploy/pull/2483/hovercard\" href=\"https://github.com/InternLM/lmdeploy/pull/2483\">#2483</a>)"}},{"before":"dd4987717d10588c1f8845a9137e0ecded1b59ec","after":"4305ddfa6d97ccaabf3b6e2bc3c34a214cde8f67","ref":"refs/heads/main","pushedAt":"2024-09-18T13:14:28.000Z","pushType":"pr_merge","commitsCount":1,"pusher":{"login":"lvhan028","name":"Lyu Han","path":"/lvhan028","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/4560679?s=80&v=4"},"commit":{"message":"Fix \"TypeError: Got unsupported ScalarType BFloat16\" (#2472)","shortMessageHtmlLink":"Fix \"TypeError: Got unsupported ScalarType BFloat16\" (<a class=\"issue-link js-issue-link\" data-error-text=\"Failed to load title\" data-id=\"2527375930\" data-permission-text=\"Title is private\" data-url=\"https://github.com/InternLM/lmdeploy/issues/2472\" data-hovercard-type=\"pull_request\" data-hovercard-url=\"/InternLM/lmdeploy/pull/2472/hovercard\" href=\"https://github.com/InternLM/lmdeploy/pull/2472\">#2472</a>)"}},{"before":"1678dc5e399f523f3479adcf7ce62266e9f39c83","after":"dd4987717d10588c1f8845a9137e0ecded1b59ec","ref":"refs/heads/main","pushedAt":"2024-09-18T11:42:59.000Z","pushType":"pr_merge","commitsCount":1,"pusher":{"login":"lvhan028","name":"Lyu Han","path":"/lvhan028","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/4560679?s=80&v=4"},"commit":{"message":"[ci] regular update (#2431)\n\n* update\r\n\r\n* update\r\n\r\n* update\r\n\r\n* update\r\n\r\n* update\r\n\r\n* update\r\n\r\n* update\r\n\r\n* update\r\n\r\n* update\r\n\r\n* updaet\r\n\r\n* update\r\n\r\n* update\r\n\r\n* update\r\n\r\n* update\r\n\r\n* update\r\n\r\n* update\r\n\r\n* update\r\n\r\n* update\r\n\r\n* update\r\n\r\n* update\r\n\r\n* update\r\n\r\n* update\r\n\r\n* update\r\n\r\n* update\r\n\r\n* update\r\n\r\n* update\r\n\r\n* update\r\n\r\n* update\r\n\r\n* update\r\n\r\n* update\r\n\r\n* updaste\r\n\r\n* update","shortMessageHtmlLink":"[ci] regular update (<a class=\"issue-link js-issue-link\" data-error-text=\"Failed to load title\" data-id=\"2509627453\" data-permission-text=\"Title is private\" data-url=\"https://github.com/InternLM/lmdeploy/issues/2431\" data-hovercard-type=\"pull_request\" data-hovercard-url=\"/InternLM/lmdeploy/pull/2431/hovercard\" href=\"https://github.com/InternLM/lmdeploy/pull/2431\">#2431</a>)"}},{"before":"e2aa4bd7a434944f2f95f7f55a8272ebc786bf2f","after":"1678dc5e399f523f3479adcf7ce62266e9f39c83","ref":"refs/heads/main","pushedAt":"2024-09-18T06:33:33.000Z","pushType":"pr_merge","commitsCount":1,"pusher":{"login":"lvhan028","name":"Lyu Han","path":"/lvhan028","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/4560679?s=80&v=4"},"commit":{"message":"Support user-sepcified data type (#2473)\n\n* support specifying data type\r\n\r\n* update\r\n\r\n* update\r\n\r\n* add dtype in CLI\r\n\r\n* add user specified dtype for pytorch engine\r\n\r\n* update log\r\n\r\n* fix linting & ut\r\n\r\n* fix vl\r\n\r\n* minor","shortMessageHtmlLink":"Support user-sepcified data type (<a class=\"issue-link js-issue-link\" data-error-text=\"Failed to load title\" data-id=\"2528108477\" data-permission-text=\"Title is private\" data-url=\"https://github.com/InternLM/lmdeploy/issues/2473\" data-hovercard-type=\"pull_request\" data-hovercard-url=\"/InternLM/lmdeploy/pull/2473/hovercard\" href=\"https://github.com/InternLM/lmdeploy/pull/2473\">#2473</a>)"}},{"before":"64fe4c53b51d07c20deb686b829479730abf14a4","after":"e2aa4bd7a434944f2f95f7f55a8272ebc786bf2f","ref":"refs/heads/main","pushedAt":"2024-09-13T03:11:07.000Z","pushType":"pr_merge","commitsCount":1,"pusher":{"login":"lvhan028","name":"Lyu Han","path":"/lvhan028","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/4560679?s=80&v=4"},"commit":{"message":"bump version to v0.6.0 (#2445)\n\n* bump version to 0.6.0\r\n\r\n* update readme\r\n\r\n* update supported models\r\n\r\n* update get_started on ascend platform","shortMessageHtmlLink":"bump version to v0.6.0 (<a class=\"issue-link js-issue-link\" data-error-text=\"Failed to load title\" data-id=\"2518920888\" data-permission-text=\"Title is private\" data-url=\"https://github.com/InternLM/lmdeploy/issues/2445\" data-hovercard-type=\"pull_request\" data-hovercard-url=\"/InternLM/lmdeploy/pull/2445/hovercard\" href=\"https://github.com/InternLM/lmdeploy/pull/2445\">#2445</a>)"}},{"before":"7e20324d5a6c6da4ae300da495c456f8f291782e","after":"dc891efd5745d5c83f3b541b68dfd80bfd6da08a","ref":"refs/heads/bump-version","pushedAt":"2024-09-12T13:38:58.000Z","pushType":"push","commitsCount":4,"pusher":{"login":"lvhan028","name":"Lyu Han","path":"/lvhan028","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/4560679?s=80&v=4"},"commit":{"message":"Merge branch 'main' into bump-version","shortMessageHtmlLink":"Merge branch 'main' into bump-version"}},{"before":"f98d15235d06ce4d7f3541afe438ec3e93dc81df","after":"64fe4c53b51d07c20deb686b829479730abf14a4","ref":"refs/heads/main","pushedAt":"2024-09-12T13:36:21.000Z","pushType":"pr_merge","commitsCount":1,"pusher":{"login":"lvhan028","name":"Lyu Han","path":"/lvhan028","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/4560679?s=80&v=4"},"commit":{"message":"Fix initialization of runtime_min_p (#2461)\n\n* fix minp\r\n\r\n* better test params","shortMessageHtmlLink":"Fix initialization of runtime_min_p (<a class=\"issue-link js-issue-link\" data-error-text=\"Failed to load title\" data-id=\"2522296693\" data-permission-text=\"Title is private\" data-url=\"https://github.com/InternLM/lmdeploy/issues/2461\" data-hovercard-type=\"pull_request\" data-hovercard-url=\"/InternLM/lmdeploy/pull/2461/hovercard\" href=\"https://github.com/InternLM/lmdeploy/pull/2461\">#2461</a>)"}},{"before":"5d09bfd402bc808d81071052bed7dc11155000c5","after":"f98d15235d06ce4d7f3541afe438ec3e93dc81df","ref":"refs/heads/main","pushedAt":"2024-09-12T13:35:42.000Z","pushType":"pr_merge","commitsCount":1,"pusher":{"login":"lvhan028","name":"Lyu Han","path":"/lvhan028","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/4560679?s=80&v=4"},"commit":{"message":"fix MultinomialSampling operator builder (#2460)","shortMessageHtmlLink":"fix MultinomialSampling operator builder (<a class=\"issue-link js-issue-link\" data-error-text=\"Failed to load title\" data-id=\"2522256604\" data-permission-text=\"Title is private\" data-url=\"https://github.com/InternLM/lmdeploy/issues/2460\" data-hovercard-type=\"pull_request\" data-hovercard-url=\"/InternLM/lmdeploy/pull/2460/hovercard\" href=\"https://github.com/InternLM/lmdeploy/pull/2460\">#2460</a>)"}},{"before":"edcdd8e36520b8bf7dbc99feecd2d2822c4cb5ba","after":"5d09bfd402bc808d81071052bed7dc11155000c5","ref":"refs/heads/main","pushedAt":"2024-09-12T09:28:39.000Z","pushType":"pr_merge","commitsCount":1,"pusher":{"login":"lvhan028","name":"Lyu Han","path":"/lvhan028","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/4560679?s=80&v=4"},"commit":{"message":"fix tensors on different devices when deploying MiniCPM-V-2_6 with tensor parallelism (#2454)\n\n* fix pp\r\n\r\n* fix lint","shortMessageHtmlLink":"fix tensors on different devices when deploying MiniCPM-V-2_6 with te…"}},{"before":"1fc5dc5f303db934ba4ce41e99c7f6e7a9479a2c","after":"7e20324d5a6c6da4ae300da495c456f8f291782e","ref":"refs/heads/bump-version","pushedAt":"2024-09-11T16:46:18.000Z","pushType":"push","commitsCount":3,"pusher":{"login":"lvhan028","name":"Lyu Han","path":"/lvhan028","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/4560679?s=80&v=4"},"commit":{"message":"Merge branch 'main' into bump-version","shortMessageHtmlLink":"Merge branch 'main' into bump-version"}},{"before":"e681077b6ec11911d720ae58c18e24254341c3cd","after":"edcdd8e36520b8bf7dbc99feecd2d2822c4cb5ba","ref":"refs/heads/main","pushedAt":"2024-09-11T09:14:29.000Z","pushType":"pr_merge","commitsCount":1,"pusher":{"login":"lvhan028","name":"Lyu Han","path":"/lvhan028","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/4560679?s=80&v=4"},"commit":{"message":"fix llama3 rotary in pytorch engine (#2444)","shortMessageHtmlLink":"fix llama3 rotary in pytorch engine (<a class=\"issue-link js-issue-link\" data-error-text=\"Failed to load title\" data-id=\"2518534099\" data-permission-text=\"Title is private\" data-url=\"https://github.com/InternLM/lmdeploy/issues/2444\" data-hovercard-type=\"pull_request\" data-hovercard-url=\"/InternLM/lmdeploy/pull/2444/hovercard\" href=\"https://github.com/InternLM/lmdeploy/pull/2444\">#2444</a>)"}},{"before":"ca57a60062a5a5ee0ea549698c15b23b62bd17e3","after":"1fc5dc5f303db934ba4ce41e99c7f6e7a9479a2c","ref":"refs/heads/bump-version","pushedAt":"2024-09-11T08:04:46.000Z","pushType":"push","commitsCount":1,"pusher":{"login":"lvhan028","name":"Lyu Han","path":"/lvhan028","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/4560679?s=80&v=4"},"commit":{"message":"update get_started on ascend platform","shortMessageHtmlLink":"update get_started on ascend platform"}},{"before":"e8a1a33aa826646420ed2e67250d8eb97cf5cea9","after":"e681077b6ec11911d720ae58c18e24254341c3cd","ref":"refs/heads/main","pushedAt":"2024-09-11T06:41:03.000Z","pushType":"pr_merge","commitsCount":1,"pusher":{"login":"lvhan028","name":"Lyu Han","path":"/lvhan028","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/4560679?s=80&v=4"},"commit":{"message":"refactor pytorch engine(ascend) (#2440)\n\n* add support for llama ascend 910b using torch layers\r\n\r\n* feat(ascend): modify the usage of ascend kernels to fit torch.compile\r\n\r\n* refactor for llama\r\n\r\n* format code\r\n\r\n* add AscendSoftmaxTopKBuilder\r\n\r\n* feat: support ascend mixtral\r\n\r\n* refactor step_context for fusion attention\r\n\r\n* fix ascend op_backend after rebase\r\n\r\n* add comment\r\n\r\n* format code\r\n\r\n* remove unused param in rms_norm","shortMessageHtmlLink":"refactor pytorch engine(ascend) (<a class=\"issue-link js-issue-link\" data-error-text=\"Failed to load title\" data-id=\"2516016333\" data-permission-text=\"Title is private\" data-url=\"https://github.com/InternLM/lmdeploy/issues/2440\" data-hovercard-type=\"pull_request\" data-hovercard-url=\"/InternLM/lmdeploy/pull/2440/hovercard\" href=\"https://github.com/InternLM/lmdeploy/pull/2440\">#2440</a>)"}},{"before":"87e51def3e972b21113dea6e56289f4aeffc2278","after":"ca57a60062a5a5ee0ea549698c15b23b62bd17e3","ref":"refs/heads/bump-version","pushedAt":"2024-09-11T03:50:02.000Z","pushType":"force_push","commitsCount":0,"pusher":{"login":"lvhan028","name":"Lyu Han","path":"/lvhan028","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/4560679?s=80&v=4"},"commit":{"message":"update supported models","shortMessageHtmlLink":"update supported models"}},{"before":"e6d70a02241ff88de4cf791db8a46ec3a295c34b","after":"e8a1a33aa826646420ed2e67250d8eb97cf5cea9","ref":"refs/heads/main","pushedAt":"2024-09-10T08:27:43.000Z","pushType":"pr_merge","commitsCount":1,"pusher":{"login":"lvhan028","name":"Lyu Han","path":"/lvhan028","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/4560679?s=80&v=4"},"commit":{"message":"Refactor pytorch engine (#2104)\n\n* attn layer\r\n\r\n* move to backend\r\n\r\n* add base layer\r\n\r\n* finish llama base\r\n\r\n* add lora and w8a8\r\n\r\n* support awq\r\n\r\n* add add_rms_norm kernel\r\n\r\n* optimize step context\r\n\r\n* attn meta as input\r\n\r\n* add cuda graph support\r\n\r\n* disable one of mha kernel\r\n\r\n* share graph pool\r\n\r\n* del graph\r\n\r\n* update docstring\r\n\r\n* awq cudagraph\r\n\r\n* merge main\r\n\r\n* support llava for llama\r\n\r\n* fix support cudagraph flag\r\n\r\n* support lora cudagraph\r\n\r\n* support logit softcapping\r\n\r\n* support transformers 4.43\r\n\r\n* fix ut\r\n\r\n* fix dynamic ntk cudagraph\r\n\r\n* add moe support\r\n\r\n* add custom module support\r\n\r\n* optimize awq kernel\r\n\r\n* optimize attention\r\n\r\n* fix graph runner\r\n\r\n* optimize prefill\r\n\r\n* fix response\r\n\r\n* optmize prefill\r\n\r\n* adjust grid of paged attention\r\n\r\n* add attention stages\r\n\r\n* support llama3\r\n\r\n* optimize apply rotary\r\n\r\n* rename\r\n\r\n* fix sampling\r\n\r\n* remove print\r\n\r\n* prepare for new weight loader\r\n\r\n* refactor add model\r\n\r\n* optimize nn\r\n\r\n* fix linear device\r\n\r\n* support baichuan 7b 13b\r\n\r\n* support deepseekv2 no-tp\r\n\r\n* support deepseek v2 tp\r\n\r\n* add log\r\n\r\n* fix ut\r\n\r\n* support chatglm\r\n\r\n* support llava\r\n\r\n* add falcon\r\n\r\n* add internlm2 and mistral\r\n\r\n* add gemma/gemma2\r\n\r\n* add deepseek, qwen1\r\n\r\n* remove request timeout\r\n\r\n* add qwen2, qwen-moe\r\n\r\n* add starcoder2 phi-3 phi-3 vision\r\n\r\n* support phi3 moe\r\n\r\n* support dbrx\r\n\r\n* support internvl\r\n\r\n* support merged awq weight\r\n\r\n* add cogvlm\r\n\r\n* update docs\r\n\r\n* fused layernorm\r\n\r\n* add gelu and mul\r\n\r\n* support triton==3.0.0\r\n\r\n* update names\r\n\r\n* fix\r\n\r\n* cogvlm2\r\n\r\n* fix\r\n\r\n* fix\r\n\r\n* fix internlm2 awq\r\n\r\n* rename\r\n\r\n* fix a hanging problem when using cli serve mode and device ascend on exit\r\n\r\n* raise -> return\r\n\r\n* optimize moe\r\n\r\n* fix linear awq bias, default awq kernel\r\n\r\n* fix\r\n\r\n* optimize default awq\r\n\r\n* fix llama rope, add internlm\r\n\r\n* optimize decoding\r\n\r\n* recovery attention\r\n\r\n* fix fill kv cache\r\n\r\n* fix internlm oom\r\n\r\n* fix llama3 memory usage\r\n\r\n* remove float deepseekv2\r\n\r\n* fix llama3\r\n\r\n* update smooth quant flag\r\n\r\n* fix w8a8\r\n\r\n* fix w8a8 tp\r\n\r\n---------\r\n\r\nCo-authored-by: grimoire <yaoqian@pjlab.org.cn>\r\nCo-authored-by: chenchiyu <chenchiyu@pjlab.org.cn>","shortMessageHtmlLink":"Refactor pytorch engine (<a class=\"issue-link js-issue-link\" data-error-text=\"Failed to load title\" data-id=\"2422833620\" data-permission-text=\"Title is private\" data-url=\"https://github.com/InternLM/lmdeploy/issues/2104\" data-hovercard-type=\"pull_request\" data-hovercard-url=\"/InternLM/lmdeploy/pull/2104/hovercard\" href=\"https://github.com/InternLM/lmdeploy/pull/2104\">#2104</a>)"}},{"before":"8e478d4f67529c7b07fc66fc61c45780102a8286","after":"e6d70a02241ff88de4cf791db8a46ec3a295c34b","ref":"refs/heads/main","pushedAt":"2024-09-09T12:50:47.000Z","pushType":"pr_merge","commitsCount":1,"pusher":{"login":"lvhan028","name":"Lyu Han","path":"/lvhan028","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/4560679?s=80&v=4"},"commit":{"message":"automatically set max_batch_size according to the device when it is not specified (#2434)\n\n* automatically set max_batch_size\r\n\r\n* update\r\n\r\n* fix\r\n\r\n* update\r\n\r\n* update","shortMessageHtmlLink":"automatically set max_batch_size according to the device when it is n…"}},{"before":"659a6b03a5f97046922f7c6c148ab71ec52f24e7","after":"8e478d4f67529c7b07fc66fc61c45780102a8286","ref":"refs/heads/main","pushedAt":"2024-09-09T10:01:50.000Z","pushType":"pr_merge","commitsCount":1,"pusher":{"login":"lvhan028","name":"Lyu Han","path":"/lvhan028","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/4560679?s=80&v=4"},"commit":{"message":"support min_p sampling parameter (#2420)\n\n* support pytorch backend min_p\r\n\r\n* support turbomind backend min_p\r\n\r\n* fix comments\r\n\r\n* remove unused header\r\n\r\n* remove inplace\r\n\r\n* use _filter_minp_sorted_\r\n\r\n* remove end_ids from sampling\r\n\r\n* use larger grid size in invokeTopPSortInitialize\r\n\r\n* skip softmax for topk request\r\n\r\n* use const\r\n\r\n* use nullptr\r\n\r\n* use eps\r\n\r\n* fix pr test","shortMessageHtmlLink":"support min_p sampling parameter (<a class=\"issue-link js-issue-link\" data-error-text=\"Failed to load title\" data-id=\"2504214343\" data-permission-text=\"Title is private\" data-url=\"https://github.com/InternLM/lmdeploy/issues/2420\" data-hovercard-type=\"pull_request\" data-hovercard-url=\"/InternLM/lmdeploy/pull/2420/hovercard\" href=\"https://github.com/InternLM/lmdeploy/pull/2420\">#2420</a>)"}},{"before":"3df11e7e58135529d35335249166c2048723af4b","after":"659a6b03a5f97046922f7c6c148ab71ec52f24e7","ref":"refs/heads/main","pushedAt":"2024-09-07T13:55:56.000Z","pushType":"pr_merge","commitsCount":1,"pusher":{"login":"lvhan028","name":"Lyu Han","path":"/lvhan028","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/4560679?s=80&v=4"},"commit":{"message":"Fix some issues encountered by modelscope and community (#2428)\n\n* fix modelscope\r\n\r\n* fix llava model when input images have size (x, 1)\r\n\r\n* larger interval\r\n\r\n* skip get_font for xcomposer2d5\r\n\r\n* fix custom image token position\r\n\r\n* fix potential mismatching issues\r\n\r\n* update docs","shortMessageHtmlLink":"Fix some issues encountered by modelscope and community (<a class=\"issue-link js-issue-link\" data-error-text=\"Failed to load title\" data-id=\"2506959594\" data-permission-text=\"Title is private\" data-url=\"https://github.com/InternLM/lmdeploy/issues/2428\" data-hovercard-type=\"pull_request\" data-hovercard-url=\"/InternLM/lmdeploy/pull/2428/hovercard\" href=\"https://github.com/InternLM/lmdeploy/pull/2428\">#2428</a>)"}},{"before":"62ac4f17e80c8c6bd29db4a61cd88af93ab330e7","after":"3df11e7e58135529d35335249166c2048723af4b","ref":"refs/heads/main","pushedAt":"2024-09-07T12:18:48.000Z","pushType":"pr_merge","commitsCount":1,"pusher":{"login":"lvhan028","name":"Lyu Han","path":"/lvhan028","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/4560679?s=80&v=4"},"commit":{"message":"build nccl in dockerfile for cu118 (#2433)","shortMessageHtmlLink":"build nccl in dockerfile for cu118 (<a class=\"issue-link js-issue-link\" data-error-text=\"Failed to load title\" data-id=\"2510364291\" data-permission-text=\"Title is private\" data-url=\"https://github.com/InternLM/lmdeploy/issues/2433\" data-hovercard-type=\"pull_request\" data-hovercard-url=\"/InternLM/lmdeploy/pull/2433/hovercard\" href=\"https://github.com/InternLM/lmdeploy/pull/2433\">#2433</a>)"}},{"before":"eab8536304483b729193165b21d5f86497f6d2ab","after":"62ac4f17e80c8c6bd29db4a61cd88af93ab330e7","ref":"refs/heads/main","pushedAt":"2024-09-06T10:14:44.000Z","pushType":"pr_merge","commitsCount":1,"pusher":{"login":"lvhan028","name":"Lyu Han","path":"/lvhan028","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/4560679?s=80&v=4"},"commit":{"message":"build: update ascend dockerfile (#2421)\n\n* build: update ascend dockerfile\r\n\r\n* add description of run file in ascend dockerfile\r\n\r\n* use multi-stage build to copy lmdeploy without run file","shortMessageHtmlLink":"build: update ascend dockerfile (<a class=\"issue-link js-issue-link\" data-error-text=\"Failed to load title\" data-id=\"2504696054\" data-permission-text=\"Title is private\" data-url=\"https://github.com/InternLM/lmdeploy/issues/2421\" data-hovercard-type=\"pull_request\" data-hovercard-url=\"/InternLM/lmdeploy/pull/2421/hovercard\" href=\"https://github.com/InternLM/lmdeploy/pull/2421\">#2421</a>)"}},{"before":"f32fc53ec903ed87aafca951efc36a0e7ba17a0b","after":"eab8536304483b729193165b21d5f86497f6d2ab","ref":"refs/heads/main","pushedAt":"2024-09-05T11:45:51.000Z","pushType":"pr_merge","commitsCount":1,"pusher":{"login":"lvhan028","name":"Lyu Han","path":"/lvhan028","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/4560679?s=80&v=4"},"commit":{"message":"Split token_embs and lm_head weights (#2252)\n\n* split lm_head\r\n\r\n* split token_embed\r\n\r\n* fix memory malloc and load from converted model\r\n\r\n* support tie_word_embeddings\r\n\r\n* split tok_embeddings along hidden dim\r\n\r\n* Revert \"support tie_word_embeddings\"\r\n\r\nThis reverts commit b7dc61b3d7c8621d2ee668de15d19d6d7a5430f8.\r\n\r\n* remove unused\r\n\r\n* fix windows build\r\n\r\n* fix loading from workspace\r\n\r\n* remove the use of ConcateSlice\r\n\r\n* remove ConcateSlice class\r\n\r\n* use context_decoder_output_buf_ buffer\r\n\r\n* use tp in config.yaml\r\n\r\n* add sync_check_cuda_error\r\n\r\n* update\r\n\r\n* remove unused header\r\n\r\n* udpate check\r\n\r\n* remove unused","shortMessageHtmlLink":"Split token_embs and lm_head weights (<a class=\"issue-link js-issue-link\" data-error-text=\"Failed to load title\" data-id=\"2452800915\" data-permission-text=\"Title is private\" data-url=\"https://github.com/InternLM/lmdeploy/issues/2252\" data-hovercard-type=\"pull_request\" data-hovercard-url=\"/InternLM/lmdeploy/pull/2252/hovercard\" href=\"https://github.com/InternLM/lmdeploy/pull/2252\">#2252</a>)"}},{"before":"5e29b90cdf2fc0690273041c7515485004a63d1d","after":"f32fc53ec903ed87aafca951efc36a0e7ba17a0b","ref":"refs/heads/main","pushedAt":"2024-09-05T09:55:14.000Z","pushType":"pr_merge","commitsCount":1,"pusher":{"login":"lvhan028","name":"Lyu Han","path":"/lvhan028","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/4560679?s=80&v=4"},"commit":{"message":"handle invalid images (#2312)\n\n* handle invalid images\r\n\r\n* fix test\r\n\r\n* update encode_image_base64\r\n\r\n* use image instead\r\n\r\n* update tests","shortMessageHtmlLink":"handle invalid images (<a class=\"issue-link js-issue-link\" data-error-text=\"Failed to load title\" data-id=\"2467461373\" data-permission-text=\"Title is private\" data-url=\"https://github.com/InternLM/lmdeploy/issues/2312\" data-hovercard-type=\"pull_request\" data-hovercard-url=\"/InternLM/lmdeploy/pull/2312/hovercard\" href=\"https://github.com/InternLM/lmdeploy/pull/2312\">#2312</a>)"}},{"before":"48664104653e9db84157e2854e52b3432f2b474d","after":"5e29b90cdf2fc0690273041c7515485004a63d1d","ref":"refs/heads/main","pushedAt":"2024-09-05T07:19:36.000Z","pushType":"pr_merge","commitsCount":1,"pusher":{"login":"lvhan028","name":"Lyu Han","path":"/lvhan028","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/4560679?s=80&v=4"},"commit":{"message":"inplace logits process as default (#2427)","shortMessageHtmlLink":"inplace logits process as default (<a class=\"issue-link js-issue-link\" data-error-text=\"Failed to load title\" data-id=\"2506873790\" data-permission-text=\"Title is private\" data-url=\"https://github.com/InternLM/lmdeploy/issues/2427\" data-hovercard-type=\"pull_request\" data-hovercard-url=\"/InternLM/lmdeploy/pull/2427/hovercard\" href=\"https://github.com/InternLM/lmdeploy/pull/2427\">#2427</a>)"}},{"before":"aa44e3d2b2ef90e06689c8d8e6a4ca5381e1d0bb","after":"48664104653e9db84157e2854e52b3432f2b474d","ref":"refs/heads/main","pushedAt":"2024-09-05T06:34:06.000Z","pushType":"pr_merge","commitsCount":1,"pusher":{"login":"lvhan028","name":"Lyu Han","path":"/lvhan028","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/4560679?s=80&v=4"},"commit":{"message":"ignore *.pth when download model from model hub (#2426)\n\n* ignore *.pth when download model from model hub\r\n\r\n* update","shortMessageHtmlLink":"ignore *.pth when download model from model hub (<a class=\"issue-link js-issue-link\" data-error-text=\"Failed to load title\" data-id=\"2506704248\" data-permission-text=\"Title is private\" data-url=\"https://github.com/InternLM/lmdeploy/issues/2426\" data-hovercard-type=\"pull_request\" data-hovercard-url=\"/InternLM/lmdeploy/pull/2426/hovercard\" href=\"https://github.com/InternLM/lmdeploy/pull/2426\">#2426</a>)"}},{"before":"e0fa42f1903769b428a51f60de9b8b3ae293896d","after":"aa44e3d2b2ef90e06689c8d8e6a4ca5381e1d0bb","ref":"refs/heads/main","pushedAt":"2024-09-04T11:59:34.000Z","pushType":"pr_merge","commitsCount":1,"pusher":{"login":"lvhan028","name":"Lyu Han","path":"/lvhan028","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/4560679?s=80&v=4"},"commit":{"message":"update actions/download-artifact to v4 to fix security issue (#2419)\n\n* update actions/download-artifact to 4.1.7 to fix security issue\r\n\r\n* update\r\n\r\n* debug\r\n\r\n* upgrade upload-artifact v4\r\n\r\n* update","shortMessageHtmlLink":"update actions/download-artifact to v4 to fix security issue (<a class=\"issue-link js-issue-link\" data-error-text=\"Failed to load title\" data-id=\"2504184620\" data-permission-text=\"Title is private\" data-url=\"https://github.com/InternLM/lmdeploy/issues/2419\" data-hovercard-type=\"pull_request\" data-hovercard-url=\"/InternLM/lmdeploy/pull/2419/hovercard\" href=\"https://github.com/InternLM/lmdeploy/pull/2419\">#2419</a>)"}},{"before":"761ae0966d04c64e6292e27dfed938873d6f3737","after":"e0fa42f1903769b428a51f60de9b8b3ae293896d","ref":"refs/heads/main","pushedAt":"2024-09-04T06:02:55.000Z","pushType":"pr_merge","commitsCount":1,"pusher":{"login":"lvhan028","name":"Lyu Han","path":"/lvhan028","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/4560679?s=80&v=4"},"commit":{"message":"add Ascend get_started (#2417)\n\n* add ascend_readme_en\r\n\r\n* add ascend_readme_zh_cn\r\n\r\n* fix ascend_readme typo\r\n\r\n* fix typo\r\n\r\n* fix indent&typo\r\n\r\n* fix indent again\r\n\r\n* fix indent again","shortMessageHtmlLink":"add Ascend get_started (<a class=\"issue-link js-issue-link\" data-error-text=\"Failed to load title\" data-id=\"2504133271\" data-permission-text=\"Title is private\" data-url=\"https://github.com/InternLM/lmdeploy/issues/2417\" data-hovercard-type=\"pull_request\" data-hovercard-url=\"/InternLM/lmdeploy/pull/2417/hovercard\" href=\"https://github.com/InternLM/lmdeploy/pull/2417\">#2417</a>)"}},{"before":"0b05c7042e2ac147849406dac51a4dfa579c7e84","after":"761ae0966d04c64e6292e27dfed938873d6f3737","ref":"refs/heads/main","pushedAt":"2024-09-03T14:06:22.000Z","pushType":"pr_merge","commitsCount":1,"pusher":{"login":"lvhan028","name":"Lyu Han","path":"/lvhan028","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/4560679?s=80&v=4"},"commit":{"message":"import dlinfer before imageencoding (#2413)\n\n* import dlinfer before imageencoding\r\n\r\n* import dlinfer if device_type is set to ascend\r\n\r\n* fix isort error","shortMessageHtmlLink":"import dlinfer before imageencoding (<a class=\"issue-link js-issue-link\" data-error-text=\"Failed to load title\" data-id=\"2500627397\" data-permission-text=\"Title is private\" data-url=\"https://github.com/InternLM/lmdeploy/issues/2413\" data-hovercard-type=\"pull_request\" data-hovercard-url=\"/InternLM/lmdeploy/pull/2413/hovercard\" href=\"https://github.com/InternLM/lmdeploy/pull/2413\">#2413</a>)"}}],"hasNextPage":true,"hasPreviousPage":false,"activityType":"all","actor":null,"timePeriod":"all","sort":"DESC","perPage":30,"cursor":"djE6ks8AAAAEu4e0gwA","startCursor":null,"endCursor":null}},"title":"Activity · InternLM/lmdeploy"}