Skip to content

Commit

Permalink
Calling garbage collector before vLLM constructor to clear memory fro…
Browse files Browse the repository at this point in the history
…m previous run. vLLM object is being retained by PyTest module and deleting it doesn't clear the memory.
  • Loading branch information
alexeykondrat committed Jun 21, 2024
1 parent dcd4721 commit f06b796
Showing 1 changed file with 3 additions and 1 deletion.
4 changes: 3 additions & 1 deletion tests/lora/test_long_context.py
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
import ast
import gc
from typing import List, Optional, Tuple

import numpy as np
Expand Down Expand Up @@ -110,7 +111,8 @@ def lora_llm(long_context_infos):
context_len_to_scaling_factor[info["context_length"]]
for info in long_context_infos.values()
]

#clear memory from previous run
gc.collect()
llm = vllm.LLM(
"meta-llama/Llama-2-13b-chat-hf",
enable_lora=True,
Expand Down

0 comments on commit f06b796

Please sign in to comment.