Support Milvus v2.4

.github/workflows/check.yml

@@ -4,10 +4,12 @@ on:
   pull_request:
     branches:
       - main
+      - 2.4
     types: [opened, synchronize]
   push:
     branches:
       - main
+      - 2.4
 jobs:
   publish:
     runs-on: ubuntu-latest

.gitmodules

@@ -1,4 +1,4 @@
 [submodule "proto"]
 	path = proto
 	url = https://github.com/milvus-io/milvus-proto.git
-	branch = master
+	branch = 2.4

README.md

@@ -1,8 +1,8 @@
 # Milvus2-sdk-node
 
 [![typescript](https://badges.aleen42.com/src/typescript.svg)](https://badges.aleen42.com/src/typescript.svg)
-[![version](https://img.shields.io/npm/v/@zilliz/milvus2-sdk-node?color=bright-green)](https://img.shields.io/npm/v/@zilliz/milvus2-sdk-node)
-[![downloads](https://img.shields.io/npm/dw/@zilliz/milvus2-sdk-node?color=bright-green)](https://img.shields.io/npm/dw/@zilliz/milvus2-sdk-node)
+[![version](https://img.shields.io/npm/v/@zilliz/milvus2-sdk-node?color=bright-green)](https://github.com/zilliztech/attu/releases)
+[![downloads](https://img.shields.io/npm/dw/@zilliz/milvus2-sdk-node?color=bright-green)](https://www.npmjs.com/package/@zilliz/milvus2-sdk-node)
 [![codecov](https://codecov.io/gh/milvus-io/milvus-sdk-node/branch/main/graph/badge.svg?token=Zu5FwWstwI)](https://codecov.io/gh/milvus-io/milvus-sdk-node)
 
 The official [Milvus](https://github.com/milvus-io/milvus) client for Node.js.
@@ -11,9 +11,11 @@ The official [Milvus](https://github.com/milvus-io/milvus) client for Node.js.
 
 The following table shows the recommended `@zilliz/milvus2-sdk-node` versions for different Milvus versions:
 
-| Milvus version | Node sdk version | Installation                        |
-| :------------: | :--------------: | :---------------------------------- |
-|    v2.2.0+     |    **latest**    | `yarn add @zilliz/milvus2-sdk-node` |
+| Milvus version | Node sdk version | Installation                               |
+| :------------: | :--------------: | :----------------------------------------- |
+|    v2.4.0+     |    **latest**    | `yarn add @zilliz/milvus2-sdk-node@latest` |
+|    v2.3.0+     |      v2.3.5      | `yarn add @zilliz/milvus2-sdk-node@2.3.5`  |
+|    v2.2.0+     |      v2.3.5      | `yarn add @zilliz/milvus2-sdk-node@2.3.5`  |
 
 ## Dependencies
 
@@ -31,21 +33,137 @@ npm install @zilliz/milvus2-sdk-node
 yarn add @zilliz/milvus2-sdk-node
 ```
 
-This will download the Milvus Node.js client and add a dependency entry in your package.json file.
+## What's new in v2.4.0
+
+### New vector data types: float16 and bfloat16
+
+Machine learning and neural networks often use half-precision data types, such as Float16 and BFloat16, [Milvus 2.4](https://milvus.io/docs/release_notes.md#Float16-and-BFloat16-Vector-DataType) supports inserting vectors in the BF16 and FP16 formats as bytes.
+
+> However, these data types are not natively available in the Node.js environment, To enable users to utilize these formats, the Node SDK provides support for transformers during insert, query, and search operations.
+>
+> There are four default transformers for performing a float32 to bytes transformation for BF16 and Float16 types: f32ArrayToF16Bytes, f16BytesToF32Array, f32ArrayToBf16Bytes, and bf16BytesToF32Array. If you wish to use your own transformers for Float16 and BFloat16, you can specify them.
+>
+> ```javascript
+> import {
+>   f32ArrayToF16Bytes,
+>   f16BytesToF32Array,
+>   f32ArrayToBf16Bytes,
+>   bf16BytesToF32Array,
+> } from '@zilliz/milvus2-sdk-node';
+>
+> //Insert float32 array for the float16 field. Node SDK will transform it to bytes using `f32ArrayToF16Bytes`. You can use your own transformer.
+> const insert = await milvusClient.insert({
+>   collection_name: COLLECTION_NAME,
+>   data: data,
+>   // transformers: {
+>   //  [DataType.BFloat16Vector]: f32ArrayToF16Bytes, // use your own transformer
+>   // },
+> });
+> // query: output float32 array other than bytes,
+> const query = await milvusClient.query({
+>   collection_name: COLLECTION_NAME,
+>   filter: 'id > 0',
+>   output_fields: ['vector', 'id'],
+>   // transformers: {
+>   // [DataType.BFloat16Vector]: bf16BytesToF32Array, // use your own transformer
+>   // },
+> });
+> // search: use bytes to search, output float32 array
+> const search = await milvusClient.search({
+>   vector: data[0].vector, // if you pass bytes, no transform will performed
+>   collection_name: COLLECTION_NAME,
+>   output_fields: ['id', 'vector'],
+>   limit: 5,
+>   // transformers: {
+>   //   [DataType.BFloat16Vector]: bf16BytesToF32Array, // use your own transformer
+>   // },
+> });
+> ```
+
+### New vector data types: sparse vector(beta)
+
+Sparse vectors in the Node SDK support four formats: `dict`, `coo`, `csr`, and `array`, However, query and search operations currently only output in the dict format.
 
-## Code Examples
+```javascript
+// dict
+const sparseObject = {
+  3: 1.5,
+  6: 2.0,
+  9: -3.5,
+};
+// coo
+const sparseCOO = [
+  { index: 2, value: 5 },
+  { index: 5, value: 3 },
+  { index: 8, value: 7 },
+];
+// csr
+const sparseCSR = {
+  indices: [2, 5, 8],
+  values: [5, 3, 7],
+};
+// array
+const sparseArray = [undefined, 0.0, 0.5, 0.3, undefined, 0.2];
+```
 
-### Milvus examples
+### Multi-vector and Hybrid Search
 
-You can find code examples in the [examples/milvus](./examples/milvus) directory. These examples cover various aspects of working with Milvus, such as connecting to Milvus, vector search, data query, dynamic schema, partition key, and database operations.
+Starting from Milvus 2.4, it supports [Multi-Vector Search](https://milvus.io/docs/multi-vector-search.md#API-overview), you can continue to utilize the search API with similar parameters to perform multi-vector searches, and the format of the results remains unchanged.
 
-### Langchain.js example
+```javascript
+// single-vector search on a collection with multiple vector fields
+const search = await milvusClient.search({
+  collection_name: collection_name,
+  data: [1, 2, 3, 4, 5, 6, 7, 8],
+  anns_field: 'vector', // required if you have multiple vector fields in the collection
+  params: { nprobe: 2 },
+  filter: 'id > 100',
+  limit: 5,
+});
 
-You can find a basic langchain.js example in the [examples/langchain](./examples/LangChain) directory.
+// multi-vector search on a collection with multiple vector fields
+const search = await milvusClient.search({
+  collection_name: collection_name,
+  data: [
+    {
+      data: [1, 2, 3, 4, 5, 6, 7, 8],
+      anns_field: 'vector',
+      params: { nprobe: 2 },
+    },
+    {
+      data: [1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16],
+      anns_field: 'vector1',
+    },
+  ],
+  limit: 5,
+  filter: 'id > 100',
+});
+```
+
+### New Typescript client
+
+Starting from v2.4.0, we introduced a TypeScript client to provide better support for the [Milvus RESTful API V2](https://milvus.io/api-reference/restful/v2.3.x/About.md), take a look at our [test file](https://github.com/milvus-io/milvus-sdk-node/blob/main/test/http/test.ts).
 
-### next.js example
+```javascript
+import { HttpClient } from '@zilliz/milvus2-sdk-node';
+const client = new HttpClient(config);
+await client.createCollection(params);
+await client.describeCollection(params);
+await client.listCollections(params);
+await client.insert(params);
+await client.upsert(params);
+await client.query(params);
+await client.search(params);
+```
 
-You can find nextjs app example in the [examples/nextjs](./examples/nextjs) directory.
+## Code Examples
+
+This table organizes the examples by technology, providing a brief description and the directory where each example can be found.
+| Technology | Example | Directory |
+|------------------|--------------------------------------------|-----------------------------------|
+| Next.js | Next.js app example | [examples/nextjs](./examples/nextjs) |
+| Node.js | Basic Node.js examples for Milvus | [examples/milvus](./examples/milvus) |
+| Langchain.js | Basic Langchain.js example | [examples/langchain](./examples/LangChain) |
 
 ## Basic usages
 
@@ -54,16 +172,14 @@ This guide will show you how to set up a simple application using Node.js and Mi
 ### Start a Milvus server
 
 ```shell
-# Download the milvus standalone yaml file
-$ wget https://github.com/milvus-io/milvus/releases/latest/download/milvus-standalone-docker-compose.yml -O docker-compose.yml
-
-# start the milvus server
-sudo docker-compose up -d
+# Start Milvus with script
+wget https://raw.githubusercontent.com/milvus-io/milvus/master/scripts/standalone_embed.sh
+bash standalone_embed.sh start
 ```
 
 ### Connect to Milvus
 
-Create a new app.js file and add the following code to try out some basic vector operations using the Milvus node.js client. More details on the [API reference](https://milvus.io/api-reference/node/v2.2.x/Client/MilvusClient.md).
+Create a new app.js file and add the following code to try out some basic vector operations using the Milvus node.js client. More details on the [API reference](https://milvus.io/api-reference/node/v2.3.x/Client/MilvusClient.md).
 
 ```javascript
 import { MilvusClient, DataType } from '@zilliz/milvus2-sdk-node';
@@ -82,7 +198,7 @@ In Milvus, the concept of the collection is like the table in traditional RDBMS,
 
 #### Define schema for collection
 
-A schema defines the fields of a collection, such as the names and data types of the fields that make up the vectors. More details of how to define schema and advanced usage can be found in [API reference](https://milvus.io/api-reference/node/v2.2.x/Collection/createCollection.md).
+A schema defines the fields of a collection, such as the names and data types of the fields that make up the vectors. More details of how to define schema and advanced usage can be found in [API reference](https://milvus.io/api-reference/node/v2.3.x/Collection/createCollection.md).
 
 ```javascript
 // define schema
@@ -128,31 +244,31 @@ The data format utilized by the Milvus Node SDK comprises an array of objects. I
 ```javascript
 const fields_data = [
   {
+    name: 'zlnmh',
     vector: [
       0.11878310581111173, 0.9694947902934701, 0.16443679307243175,
       0.5484226189097237, 0.9839246709011924, 0.5178387104937776,
       0.8716926129208069, 0.5616972243831446,
     ],
     height: 20405,
-    name: 'zlnmh',
   },
   {
+    name: '5lr9y',
     vector: [
       0.9992090731236536, 0.8248790611809487, 0.8660083940881405,
       0.09946359318481224, 0.6790698063908669, 0.5013786801063624,
       0.795311915725105, 0.9183033261617566,
     ],
     height: 93773,
-    name: '5lr9y',
   },
   {
+    name: 'nes0j',
     vector: [
       0.8761291569818763, 0.07127366044153227, 0.775648976160332,
       0.5619757601304878, 0.6076543120476996, 0.8373907516027586,
       0.8556140171597648, 0.4043893119391049,
     ],
     height: 85122,
-    name: 'nes0j',
   },
 ];
 ```
@@ -164,7 +280,7 @@ Once we have the data, you can insert data into the collection by calling the `i
 ```javascript
 await client.insert({
   collection_name,
-  fields_data,
+  data,
 });
 ```
 
@@ -175,8 +291,7 @@ By creating an index and loading the collection into memory, you can improve the
 ```javascript
 // create index
 await client.createIndex({
-  // required
-  collection_name,
+  collection_name, // required
   field_name: 'vector', // optional if you are using milvus v2.2.9+
   index_name: 'myindex', // optional
   index_type: 'HNSW', // optional if you are using milvus v2.2.9+
@@ -185,7 +300,7 @@ await client.createIndex({
 });
 ```
 
-Milvus supports [several different types of indexes](https://milvus.io/docs/index.md), each of which is optimized for different use cases and data distributions. Some of the most commonly used index types in Milvus include IVF_FLAT, IVF_SQ8, IVF_PQ, and HNSW. When creating an index in Milvus, you must choose an appropriate index type based on your specific use case and data distribution.
+Milvus supports [several different types of indexes](https://milvus.io/docs/index.md), each of which is optimized for different use cases and data distributions. Some of the most commonly used index types in Milvus include HNSW, IVF_FLAT, IVF_SQ8, IVF_PQ. When creating an index in Milvus, you must choose an appropriate index type based on your specific use case and data distribution.
 
 ### Load collection
 
@@ -210,22 +325,26 @@ const searchVector = fields_data[0].vector;
 const res = await client.search({
   // required
   collection_name, // required, the collection name
-  vector: searchVector, // required, vector used to compare other vectors in milvus
+  data: searchVector, // required, vector used to compare other vectors in milvus
   // optionals
-  filter: 'height > 0', // optional, filter
+  filter: 'height > 0', // optional, filter expression
   params: { nprobe: 64 }, // optional, specify the search parameters
   limit: 10, // optional, specify the number of nearest neighbors to return
-  metric_type: 'L2', // optional, metric to calculate similarity of two vectors
-  output_fields: ['height', 'name'], // optional, specify the fields to return in the search results
+  output_fields: ['height', 'name'], // optional, specify the fields to return in the search results,
 });
 ```
 
 ## Next Steps
 
+- [Attu, Using GUI to manage Milvus](https://github.com/zilliztech/attu)
+  ![<img src="[./.github/images/screenshot.png](https://github.com/zilliztech/attu/raw/main/.github/images/screenshot.png)" width="800" alt="attu home view" />
+](https://github.com/zilliztech/attu/raw/main/.github/images/screenshot.png)
+
+## other useful links
+
 - [What is Milvus](https://milvus.io/)
-- [Milvus Node SDK API reference](https://milvus.io/api-reference/node/v2.2.x/About.md)
-- [Attu, Milvus GUI tool, based on Milvus node.js SDK](https://github.com/zilliztech/attu)
-- [Feder, anns index visuliazation tool](https://github.com/zilliztech/feder)
+- [Milvus Node SDK API reference](https://milvus.io/api-reference/node/v2.3.x/About.md)
+- [Feder, anns index visualization tool](https://github.com/zilliztech/feder)
 
 ## How to contribute
 

milvus/HttpClient.ts

@@ -1,5 +1,14 @@
 import { HttpClientConfig, FetchOptions } from './types';
-import { Collection, Vector } from './http';
+import {
+  Collection,
+  Vector,
+  User,
+  Role,
+  Partition,
+  MilvusIndex,
+  Alias,
+  Import,
+} from './http';
 import {
   DEFAULT_DB,
   DEFAULT_HTTP_TIMEOUT,
@@ -48,9 +57,7 @@ export class HttpBaseClient {
   get baseURL() {
     return (
       this.config.baseURL ||
-      `${this.config.endpoint}/${
-        this.config.version || DEFAULT_HTTP_ENDPOINT_VERSION
-      }`
+      `${this.config.endpoint}/${DEFAULT_HTTP_ENDPOINT_VERSION}`
     );
   }
 
@@ -81,6 +88,10 @@ export class HttpBaseClient {
       Authorization: this.authorization,
       Accept: 'application/json',
       ContentType: 'application/json',
+      'Accept-Type-Allow-Int64':
+        typeof this.config.acceptInt64 !== 'undefined'
+          ? this.config.acceptInt64.toString()
+          : 'false',
     };
   }
 
@@ -163,5 +174,20 @@ export class HttpBaseClient {
   }
 }
 
-// The HttpClient class extends the functionality of the HttpBaseClient class by mixing in the Collection and Vector APIs.
-export class HttpClient extends Collection(Vector(HttpBaseClient)) {}
+/**
+ * The HttpClient class extends the functionality
+ * of the HttpBaseClient class by mixing in the
+ * - Collection
+ * - Vector
+ * - Alias
+ * - Partition
+ * - MilvusIndex
+ * - Import
+ * - Role
+ * - User APIs.
+ */
+export class HttpClient extends User(
+  Role(
+    MilvusIndex(Import(Alias(Partition(Collection(Vector(HttpBaseClient))))))
+  )
+) {}

milvus/const/defaults.ts

@@ -13,7 +13,7 @@ export const DEFAULT_DB = 'default'; // default database name
 export const DEFAULT_DYNAMIC_FIELD = '$meta'; // default dynamic field name
 export const DEFAULT_COUNT_QUERY_STRING = 'count(*)'; // default count query string
 export const DEFAULT_HTTP_TIMEOUT = 60000; // default http timeout, 60s
-export const DEFAULT_HTTP_ENDPOINT_VERSION = 'v1'; // api version, default v1
+export const DEFAULT_HTTP_ENDPOINT_VERSION = 'v2'; // api version, default v1
 
 export const DEFAULT_POOL_MAX = 10; // default max pool client number
 export const DEFAULT_POOL_MIN = 2; // default min pool client number

milvus/const/error.ts

@@ -24,7 +24,7 @@ export const ERROR_REASONS = {
   COLLECTION_PARTITION_NAME_ARE_REQUIRED:
     'The `collection_name` or the `partition_name` property is missing.',
   INSERT_CHECK_FIELD_DATA_IS_REQUIRED:
-    'The type of the `fields_data` should be an array and length > 0.',
+    'The type of the `data or field_data` should be an array and length > 0.',
   INSERT_CHECK_WRONG_FIELD:
     'Insert fail: some field does not exist for this collection in line.',
   INSERT_CHECK_WRONG_DIM: