Milvus collection scaling #23938
-
|
My organization has a particular use case where we have an arbitrary amount of collections in the database. I want to inquire about the memory behavior as more collections get loaded, say 10,000-100k. Is there any behavior where collections that are stale are automatically released? The discussion is definitely related to: #21341 Otherwise, does the underlying system just run out of memory and crash? |
Beta Was this translation helpful? Give feedback.
Replies: 2 comments 4 replies
-
|
Milvus doesn't automatically release collection. The LRU load enhancement is not ready, yet. The underlying system checks memory usage during loading and tries to avoid OOM as much as possible. If the memory is insufficient, the load() API will return an error to the client. |
Beta Was this translation helpful? Give feedback.
-
|
Really looking forward to the LRU load enhancement, I think we can manage scale until it is ready. |
Beta Was this translation helpful? Give feedback.
-
|
Do I understand correctly that the "LRU load enhancement" is meant to release() the oldest collection when you load() a new one once the memory requirement reaches a certain threshold? Is that kind of feature available? This seems like something the server should be tracking, and having to manually keep a reference count of loads and ages of queries across multiple services using a particular collection in a cluster so you can be reasonably sure it is safe to release() it is harsh... |
Beta Was this translation helpful? Give feedback.
-
|
This is on the roadmap of Milvus 2.6.X, which could be released end of 2025 Q1. |
Beta Was this translation helpful? Give feedback.
-
|
Great news! This would go a long way in ensuring the cluster remains healthy as multiple users use the same database at different times/rates. I really look forward to implementing it. |
Beta Was this translation helpful? Give feedback.
-
|
@liliu-z and his team is working hard on delivering it on march. Please stay tuned |
Beta Was this translation helpful? Give feedback.
Milvus doesn't automatically release collection. The LRU load enhancement is not ready, yet.
DISKANN can save more memory than other index types. Typically, DISKANN only requires 1/6 of the original data memory size.
The underlying system checks memory usage during loading and tries to avoid OOM as much as possible. If the memory is insufficient, the load() API will return an error to the client.