Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Switching to v2 API #11

Merged
merged 8 commits into from
Jul 4, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions .env.example
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
MILVUS_URL=http://localhost:19530
MILVUS_API_KEY=
4 changes: 3 additions & 1 deletion .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -10,4 +10,6 @@
# rspec failure tracking
.rspec_status

/volumes
/volumes

.env
10 changes: 9 additions & 1 deletion CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,13 @@
## [Unreleased]

## [0.1.0] - 2023-04-20
## [0.10.0] - 2024-07-04 🇺🇸
- BREAKING: Switched the gem to use newer V2 API with different endpoints and corresponding endpoints.

## [0.9.3] - 2023-07-01

## [0.9.2] - 2023-08-10

## [0.9.1] - 2023-05-10

## [0.9.0] - 2023-04-21
- Initial release
10 changes: 3 additions & 7 deletions Gemfile.lock
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
PATH
remote: .
specs:
milvus (0.9.3)
milvus (0.10.0)
faraday (>= 2.0.1, < 3)

GEM
Expand Down Expand Up @@ -68,11 +68,7 @@ GEM
unicode-display_width (2.4.2)

PLATFORMS
arm64-darwin-23
x86_64-darwin-19
x86_64-darwin-21
x86_64-darwin-22
x86_64-linux
ruby

DEPENDENCIES
milvus!
Expand All @@ -82,4 +78,4 @@ DEPENDENCIES
standard (~> 1.27.0)

BUNDLED WITH
2.4.0
2.5.14
193 changes: 123 additions & 70 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,9 @@ Available for paid consulting engagements! [Email me](mailto:andrei@sourcelabs.i
[![](https://dcbadge.vercel.app/api/server/WDARp7J2n8?compact=true&style=flat)](https://discord.gg/WDARp7J2n8)
[![X](https://img.shields.io/twitter/url/https/twitter.com/cloudposse.svg?style=social&label=Follow%20%40rushing_andrei)](https://twitter.com/rushing_andrei)

## API Docs
https://docs.zilliz.com/reference/restful/data-plane-v2

## Installation

Install the gem and add to the application's Gemfile by executing:
Expand All @@ -37,11 +40,25 @@ If bundler is not being used to manage dependencies, install the gem by executin
require 'milvus'

client = Milvus::Client.new(
url: 'http://localhost:9091'
url: 'http://localhost:19530'
)
```

### Using the Collections endpoints
```ruby
# Check if the collection exists.
client.collections.has(collection_name: "book")
```

```ruby
# Rename a collection.
client.collections.rename(collection_name: "book", new_collection_name: "note")
```

```ruby
# Get collection stats
client.collections.get_stats(collection_name: "book")
```

```ruby
# Data types: https://github.com/patterns-ai-core/milvus/blob/main/lib/milvus/constants.rb
Expand All @@ -53,46 +70,47 @@ client.collections.create(
auto_id: false,
fields: [
{
"name": "book_id",
"fieldName": "book_id",
"description": "book id",
"is_primary_key": true,
"isPrimary": true,
"autoID": false,
"data_type": Milvus::DATA_TYPES["int64"]
"dataType": "Int64"
},
{
"name": "word_count",
"fieldName": "word_count",
"description": "count of words",
"is_primary_key": false,
"data_type": Milvus::DATA_TYPES["int64"]
"isPrimary": false,
"dataType": "Int64"
},
{
"name": "book_intro",
"fieldName": "book_intro",
"description": "embedded vector of book introduction",
"data_type": Milvus::DATA_TYPES["binary_vector"],
"is_primary_key": false,
"type_params": [
{
"key": "dim",
"value": "2"
}
]
"dataType": "FloatVector",
"isPrimary": false,
"elementTypeParams": {
"dim": "2"
}
}
]
)
```
```ruby
# Get the collection info
client.collections.get(collection_name: "book")
# Descrbie the collection
client.collections.describe(collection_name: "book")
```
```ruby
# Delete the collection
client.collections.delete(collection_name: "book")
# Drop the collection
client.collections.drop(collection_name: "book")
```
```ruby
# Load the collection to memory before a search or a query
client.collections.load(collection_name: "book")
```
```ruby
# List all collections in the specified database.
client.collections.list
```
```ruby
# Release a collection from memory after a search or a query to reduce memory usage
client.collections.release(collection_name: "book")
```
Expand Down Expand Up @@ -143,91 +161,126 @@ client.entities.compact_status(
# => {"status"=>{}, "state"=>2}
```

### Indices
### Indexes
```ruby
client.indices.create(
collection_name: "book",
field_name: "book_intro",
extra_params: [
{ key: "metric_type", "value": "L2" },
{ key: "index_type", "value": "IVF_FLAT" },
{ key: "params", "value": "{\"nlist\":1024}" }
]
# Create an index
index_params = {
fieldName: "example_field",
indexType: "IVF_FLAT",
metricType: "L2",
params: { nlist: 100 }
}

client.indexes.create(
collection_name: "example_collection",
index_params: index_params
)
```
```ruby
collection.indices.create(
field_name: "book_name",
index_name: "scalar_index",
# Describe an index
client.indexes.describe(
collection_name: "example_collection",
index_name: "example_index"
)
```
```ruby
client.indices.delete(
collection_name: "book",
field_name: "book_intro"
# List indexes
client.indexes.list(
collection_name: "example_collection"
)
```
```ruby
# Drop an index
client.indexes.drop(
collection_name: "example_collection",
index_name: "example_index"
)
```

### Search & Querying
### Search, Querying & Hybrid Search
```ruby
client.search(
collection_name: "book",
output_fields: ["book_id"], # optional
anns_field: "book_intro",
top_k: "2",
params: "{\"nprobe\": 10}",
metric_type: "L2",
round_decimal: "-1",
vectors: [ [0.1,0.2] ],
dsl_type: 1
client.entities.search(
collection_name: "recipes",
anns_field: "vectors",
data: [embedding],
filter: "id in [450847466900987454]"
)
```
```ruby
client.query(
collection_name: "book",
output_fields: ["book_id", "book_intro"],
expr: "book_id in [2,4,6,8]"
client.entities.query(
collection_name: "recipes",
filter: "id in [450847466900987455, 450847466900987454]"
)
```
```ruby
client.entities.hybrid_search(
collection_name: "recipes",
search: [{
filter: "id in [450847466900987455]",
data: [embedding],
annsField: "vectors",
limit: 10,
outputFields: ["content", "id"]
}],
rerank: {
"strategy": "rrf",
"params": {
"k": 10
}
},
limit: 10,
output_fields: ["content", "id"]
)
```

### Partitions
```ruby
client.partitions.create(
"collection_name": "book",
"partition_name": "novel"
# List partitions
client.partitions.list(
collection_name: "example_collection"
)
```
```ruby
client.partitions.get(
"collection_name": "book",
"partition_name": "novel"
# Create a partition
client.partitions.create(
collection_name: "example_collection",
partition_name: "example_partition"
)
```
```ruby
client.partitions.delete(
"collection_name": "book",
"partition_name": "novel"
# Check if a partition exists
client.partitions.has(
collection_name: "example_collection",
partition_name: "example_partition"
)
```
```ruby
# Load partition data into memory
client.partitions.load(
"collection_name": "book",
"partition_names": ["novel"],
"replica_number": 1
collection_name: "example_collection",
partition_names: ["example_partition"]
)
```
```ruby
# Release partition data from memory
client.partitions.release(
"collection_name": "book",
"partition_names": ["novel"],
"replica_number": 1
collection_name: "example_collection",
partition_names: ["example_partition"]
)
```

### Health
```ruby
# Live determines whether the application is alive. It can be used for Kubernetes liveness probe.
client.health
# Get statistics of a partition
client.partitions.get_stats(
collection_name: "example_collection",
partition_name: "example_partition"
)
```
```ruby
# Drop a partition
client.partitions.drop(
collection_name: "example_collection",
partition_name: "example_partition"
)
```

## Development
Expand All @@ -238,7 +291,7 @@ To install this gem onto your local machine, run `bundle exec rake install`. To

## Contributing

Bug reports and pull requests are welcome on GitHub at https://github.com/andreibondarev/milvus.
Bug reports and pull requests are welcome on GitHub at https://github.com/patterns-ai-core/milvus.

## License

Expand Down
2 changes: 1 addition & 1 deletion bin/console
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@ require "milvus"
# Pry.start

client = Milvus::Client.new(
url: "http://localhost:9091", #ENV["MILVUS_URL"],
url: ENV["MILVUS_URL"],
api_key: ENV["MILVUS_API_KEY"]
)

Expand Down
5 changes: 1 addition & 4 deletions lib/milvus.rb
Original file line number Diff line number Diff line change
Expand Up @@ -9,9 +9,6 @@ module Milvus
autoload :Client, "milvus/client"
autoload :Error, "milvus/error"
autoload :Entities, "milvus/entities"
autoload :Health, "milvus/health"
autoload :Indices, "milvus/indices"
autoload :Indexes, "milvus/indexes"
autoload :Partitions, "milvus/partitions"
autoload :Search, "milvus/search"
autoload :Query, "milvus/query"
end
Loading