[Bug]: Not able to list the collection through REST #39587

nileshsironja · 2025-01-24T16:35:11Z

Is there an existing issue for this?

I have searched the existing issues

Environment

- Milvus version: v2.5.0
- Deployment mode(standalone or cluster): cluster
- MQ type(rocksmq, pulsar or kafka):    kafka
- SDK version(e.g. pymilvus v2.0.0rc2): Using REST/CURL
- OS(Ubuntu or CentOS):  Ubuntu
- CPU/Memory: 8vCPU/16834Mib

Current Behavior

Rest/curl command to fetch/flush the list of collection is not working.
I've created the collection and inserted some data. Can retrieve the data as well.

Expected Behavior

List the collections.

Steps To Reproduce

Port forwarding : kubectl port-forward --address 0.0.0.0 service/services-milvus 12345:19530 -n mntaps3

Commands: (no one is working)
1. curl --request POST --url "http://localhost:12345/v2/vectordb/collections/list" --header "Content-Type: application/json" -d '{ "dbName": "default"}'
2. curl -v --request POST --url "http://localhost:12345/v2/vectordb/collections/list" --header "Content-Type: application/json" -d '{ "dbName": "_default"}'
3. export CLUSTER_ENDPOINT="http://localhost:12345"
   curl --request GET     --url "${CLUSTER_ENDPOINT}/v2/vectordb/collections/list"  --header 'accept: application/json'     --header 'content-type: application/json'
4. curl -request POST  -url "${CLUSTER_ENDPOINT}/v2/vectordb/collections/list" --header 'accept: application/json'     --header 'content-type: application/json'

Milvus Log

milvus-log.tar.gz

Anything else?

No response

nileshsironja · 2025-01-24T19:10:36Z

For more information: command output.

curl -v -X POST "http://localhost:12345/v2/vectordb/collections/list" -d '{ "dbName": "_default" }'
Note: Unnecessary use of -X or --request, POST is already inferred.
Trying 127.0.0.1:12345...
Connected to localhost (127.0.0.1) port 12345 (#0)
POST /v2/vectordb/collections/list HTTP/1.1
Host: localhost:12345
User-Agent: curl/7.81.0
Accept: /
Content-Length: 24
Content-Type: application/x-www-form-urlencoded

Received HTTP/0.9 when not allowed
Closing connection 0
curl: (1) Received HTTP/0.9 when not allowed

nileshsironja · 2025-01-24T19:14:48Z

root@scs000514932:/.kube# curl -X POST "http://localhost:12345/v2/vectordb/collections/list" -H "Content-Type: application/json" -d '{ "dbName": "default" }'
curl: (1) Received HTTP/0.9 when not allowed
root@scs000514932:/.kube# curl -X POST "http://localhost:12345/v2/vectordb/collections/list" -H "Content-Type: application/json" -d '{ "dbName": "_default" }'
curl: (1) Received HTTP/0.9 when not allowed

nileshsironja · 2025-01-24T19:44:12Z

Thinking, might be ubuntu is not supporting HTTP/0.9.

yanliang567 · 2025-01-26T02:05:34Z

/assign @smellthemoon
please help to double check if it is an OS related issue.
/unassign

nileshsironja · 2025-01-27T10:53:29Z

Tried in linux redhat system as well. In the same issue.

curl --request POST --url "http://localhost:12345/v2/vectordb/collections/list" --header "Content-Type: application/json" -d '{ "dbName": "_default"}'
curl: (1) Received HTTP/0.9 when not allowed
root@scs000514932:/u/nsironja# curl -v -XPAQ --request POST --url "http://localhost:12345/v2/vectordb/collections/list" --header "Content-Type: application/json" -d '{ "dbName": "_default"}'
Note: Unnecessary use of -X or --request, POST is already inferred.

Host localhost:12345 was resolved.
IPv6: ::1
IPv4: 127.0.0.1
Trying [::1]:12345...
connect to ::1 port 12345 from ::1 port 47122 failed: Connection refused
Trying 127.0.0.1:12345...
Connected to localhost (127.0.0.1) port 12345

POST /v2/vectordb/collections/list HTTP/1.1
Host: localhost:12345
User-Agent: curl/8.5.0
Accept: /
Content-Type: application/json
Content-Length: 23

Received HTTP/0.9 when not allowed

nileshsironja · 2025-01-27T15:44:13Z

Here is some more trial but still failing..

root@scs000514932:~/.kube# curl -v --http0.9 --request POST --url "${CLUSTER_ENDPOINT}/v2/vector/collections/list" --header "Authorization: Bearer ${TOKEN}" --header "Content-Type: application/json"
Host localhost:12345 was resolved.
IPv6: ::1
IPv4: 127.0.0.1
Trying [::1]:12345...
connect to ::1 port 12345 from ::1 port 39840 failed: Connection refused
Trying 127.0.0.1:12345...
Connected to localhost (127.0.0.1) port 12345
POST /v2/vector/collections/list HTTP/1.1
Host: localhost:12345
User-Agent: curl/8.5.0
Accept: /
Authorization: Bearer root:Milvus
Content-Type: application/json

Warning: Binary output can mess up your terminal. Use "--output -" to tell
Warning: curl to output it to your terminal anyway, or consider "--output
Warning: " to save to a file.
Failure writing output to destination <<<<<<<<<<<< here issue is different.
Closing connection

xiaofan-luan · 2025-01-28T02:21:45Z

try see if this could work
curl -v --http1.1 --request POST
--url "http://localhost:12345/v2/vectordb/collections/list"
--header "Content-Type: application/json"
--data '{ "dbName": "_default" }'

nileshsironja · 2025-01-28T02:46:19Z

This is also not working out.

`
curl -v --http1.1 --request POST --url "http://localhost:12345/v2/vectordb/collections/list" --header "Content-Type: application/json" --data '{ "dbName": "_default" }'
Note: Unnecessary use of -X or --request, POST is already inferred.

Host localhost:12345 was resolved.
IPv6: ::1
IPv4: 127.0.0.1
Trying [::1]:12345...
connect to ::1 port 12345 from ::1 port 49432 failed: Connection refused
Trying 127.0.0.1:12345...
Connected to localhost (127.0.0.1) port 12345

POST /v2/vectordb/collections/list HTTP/1.1
Host: localhost:12345
User-Agent: curl/8.5.0
Accept: /
Content-Type: application/json
Content-Length: 24

Received HTTP/0.9 when not allowed
Closing connection
curl: (1) Received HTTP/0.9 when not allowed

curl -v --http0.9 --request POST --url "http://localhost:12345/v2/vectordb/collections/list" --header "Content-Type: application/json" --data '{ "dbName": "_default" }'
Note: Unnecessary use of -X or --request, POST is already inferred.

Host localhost:12345 was resolved.
IPv6: ::1
IPv4: 127.0.0.1
Trying [::1]:12345...
connect to ::1 port 12345 from ::1 port 37796 failed: Connection refused
Trying 127.0.0.1:12345...
Connected to localhost (127.0.0.1) port 12345

POST /v2/vectordb/collections/list HTTP/1.1
Host: localhost:12345
User-Agent: curl/8.5.0
Accept: /
Content-Type: application/json
Content-Length: 24

Warning: Binary output can mess up your terminal. Use "--output -" to tell
Warning: curl to output it to your terminal anyway, or consider "--output
Warning: " to save to a file.

Failure writing output to destination
Closing connection
`

xiaofan-luan · 2025-01-28T04:17:40Z

It seems to be Connection refused, which means it has nothing to do with milvus itself.

Could you do telnet 12345 port and see if it's connectable?

ghost · 2025-01-28T06:36:14Z

@xiaofan-luan I am facing the same issue with milvus v2.5.0

Trying 127.0.0.1...
Connected to localhost.

The verbose for the curl also shows that the connection to localhost:12345 was successful.

Note: Unnecessary use of -X or --request, POST is already inferred.
* Host localhost:12345 was resolved.
* IPv6: ::1
* IPv4: 127.0.0.1
*   Trying [::1]:12345...
* connect to ::1 port 12345 from ::1 port 35260 failed: Connection refused
*   Trying 127.0.0.1:12345...
* Connected to localhost (127.0.0.1) port 12345
> POST /v2/vectordb/collections/list HTTP/1.1
> Host: localhost:12345
> User-Agent: curl/8.5.0
> Accept: */*
> Content-Type: application/json
> Content-Length: 24
> 
* Received HTTP/0.9 when not allowed
* Closing connection
curl: (1) Received HTTP/0.9 when not allowed

xiaofan-luan · 2025-01-28T07:33:33Z

@nileshsironja
from the log, it seems you cluster is listen on 8080

[please check your proxy.http.port config](services-milvus-proxy-67855ffb47-cfc96.log:[2025/01/24 08:19:09.453 +00:00] [INFO] [proxy/listener_manager.go:156] ["Proxy server(http) listen on"] [port=8080])

please check the config of your restful port. 19530 is by default for grpc

xiaofan-luan · 2025-01-28T07:36:31Z

I guess you followed this document, and didn't under the meaning.

@sarthakdua-27 same here

ghost · 2025-01-28T09:11:34Z

@xiaofan-luan What I could see with my milvus service setup is that only these 2 ports are exposed

  ports:
  - name: milvus
    nodePort: 31252
    port: 19530
    protocol: TCP
    targetPort: milvus
  - name: metrics
    nodePort: 30240
    port: 9091
    protocol: TCP
    targetPort: metrics

Even on trying to do port forwarding to 8080 port I see this

r~# kubectl port-forward --address 0.0.0.0 service/services-milvus 8080:8080 -n <namespace>
error: Service services-milvus does not have a service port 8080

Also, these endpoints used to work fine for me with older rest sdk and have started seeing this issue recently on 2.5.x.

Also the web-content repo refers to using 19530 - https://github.com/milvus-io/web-content/blob/dependabot/npm_and_yarn/katex-0.16.21/API_Reference/milvus-restful/v2.4.x/About.md

nileshsironja · 2025-01-28T10:31:38Z

@xiaofan-luan I also checked the proxy ports are 19530, 9091 only.
I would have tried with 8080 port too, so the logs shows that.

kubectl describe pods services-milvus-proxy-67855ffb47-cfc96 -n namespace
Name: services-milvus-proxy-67855ffb47-cfc96
Namespace: namespace
Priority: 0
Service Account: default

proxy:
Container ID: containerd://89def8d978b5c94a7450e69bdb90de1cd201011a5d3e8665c041ba8d695649c3
Image: @.com/milvusdb/milvus:v2.5.0
Image ID: docker/milvusdb/milvus@sha256:129ad13dd76e7ae8eada4019396fa5d21550a4ed8c19addf080c6c10567b2259
Ports: 19530/TCP, 9091/TCP

xiaofan-luan · 2025-01-29T02:40:45Z

@xiaofan-luan I also checked the proxy ports are 19530, 9091 only. I would have tried with 8080 port too, so the logs shows that.

kubectl describe pods services-milvus-proxy-67855ffb47-cfc96 -n namespace
Name: services-milvus-proxy-67855ffb47-cfc96
Namespace: namespace
Priority: 0
Service Account: default
proxy:
Container ID: containerd://89def8d978b5c94a7450e69bdb90de1cd201011a5d3e8665c041ba8d695649c3
Image: @.com/milvusdb/milvus:v2.5.0
Image ID: docker/milvusdb/milvus@sha256:129ad13dd76e7ae8eada4019396fa5d21550a4ed8c19addf080c6c10567b2259
Ports: 19530/TCP, 9091/TCP

As I said, you set the config proxy.http.port to 8080.
and you don't expose 8080 port to outside docker.

xiaofan-luan · 2025-01-29T05:33:16Z

This is what I do:

deploy using https://milvus.io/docs/install_standalone-docker.md
curl --request POST --url "http://localhost:19530/v2/vectordb/collections/list" --header "Content-Type: application/json" --data '{ "dbName": "_default" }'
response i get {"code":800,"message":"database not found[database=_default]"}

everything seems to be perfect. please offer how you deploy this cluster to help on reproducing

nileshsironja · 2025-01-29T07:12:08Z

@xiaofan-luan
My deployment is cluster one.
And interesting thing is when I did port forwarding for proxy on 8080, listing collections are working.

Now facing issue with flushing the collection - 404, page not found

curl -v --http0.9 url "http://localhost:12345/v2/vectordb/collections/flush" --header "Content---header "Content-Type: application/json" -d '{"collectionName": "hello_milvus_shard" }'
> * Could not resolve host: url
> * Closing connection 0
> curl: (6) Could not resolve host: url
> *   Trying 127.0.0.1:12345...
> * Connected to localhost (127.0.0.1) port 12345 (#1)
> > POST0.9 /v2/vectordb/collections/flush HTTP/1.1
> > Host: localhost:12345
> > User-Agent: curl/7.81.0
> > Accept: */*
> > Content-Type: application/json
> > Content-Length: 41
> >
> * Mark bundle as not supporting multiuse
> < HTTP/1.1 404 Not Found
> < Access-Control-Allow-Credentials: true
> < Access-Control-Allow-Headers: Content-Type, Content-Length, Accept-Encoding, X-CSRF-Token, Authorization, accept, origin, Cache-Control, X-Requested-With
> < Access-Control-Allow-Methods: GET, HEAD, POST, PUT, DELETE, OPTIONS, PATCH
> < Access-Control-Allow-Origin: *
> < Content-Type: text/plain
> < Date: Wed, 29 Jan 2025 05:27:23 GMT
> < Content-Length: 18
> <
> * Connection #1 to host localhost left intact
> 404 page not found

nileshsironja · 2025-01-29T08:07:26Z

Adding to my previous comment for flushing milvus collections.

Here are the port-forwarding..
kubectl port-forward pod/services-milvus-proxy-66cd6f9457-ds54z 12345: 8080 -n ns
kubectl port-forward --address 0.0.0.0 service/services-milvus 56789:19530 -n ns

These are the command I've tried. Ideally "56789" port should be listening.
curl -v --http0.9 --request POST --url "http://localhost:56789/v2/vectordb/collections/flush" --header "Content-Type: application/json" --header 'accept: application/json' --data '{ "collectionName": "hello_milvus_shard" }'
curl -v --http1.1 --request POST --url "http://localhost:56789/v2/vectordb/collections/flush" --header "Content-Type: application/json" --header 'accept: application/json' --data '{ "collectionName": "hello_milvus_shard" }'

Please suggest.

xiaofan-luan · 2025-01-29T23:43:15Z

--http0.9

remove --http0.9.

I don't know how you changed your config but your restful port is changed to 8080, based on the log.
and again you should use 12345 port. All the restful API should forward to 8080 port
I test it through my environemnt and everything worked fine.

nileshsironja · 2025-01-30T13:41:53Z

I've tried with and without --http0.9 and port 12345 and 56789, no one worked for me.

Next what I tried is entered in proxy pod bash and tried listing the collection and it worked:
curl -v --request POST --url "http://localhost:8080/v2/vectordb/collections/list" --header "Content-Type: application/json" --data '{ "dbName": "default" }'

But flush is still throwing 404 page not found.
curl -v --request POST --url "http://localhost:8080/v2/vectordb/collections/flush" --header "Content-Type: application/json" --data '{"collectionName": "hello_milvus_shard" }'

Is there a way to mention the db name also in flush command? just to be more specific.
Or what else can be tried here?

xiaofan-luan · 2025-01-31T00:58:32Z

2.5.0 don't have flush API support for restful.
2.5.4 introduced it in #39425

xiaofan-luan · 2025-01-31T00:58:51Z

also, I don't think you need to call flush API directly for most of your use cases.

nileshsironja · 2025-01-31T04:08:40Z

@xiaofan-luan Thanks, I'll setup for v2.5.4 and validate the functionality. My intention was it got added by v2.5.0.

nileshsironja · 2025-02-03T04:51:21Z

@xiaofan-luan Updated milvus version v2.5.4 make flushing work for me. Thank you!

nileshsironja added kind/bug Issues or changes related a bug needs-triage Indicates an issue or PR lacks a `triage/foo` label and requires one. labels Jan 24, 2025

nileshsironja assigned yanliang567 Jan 24, 2025

sre-ci-robot assigned smellthemoon and unassigned yanliang567 Jan 26, 2025

yanliang567 added triage/accepted Indicates an issue or PR is ready to be actively worked on. and removed needs-triage Indicates an issue or PR lacks a `triage/foo` label and requires one. labels Jan 26, 2025

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[Bug]: Not able to list the collection through REST #39587

[Bug]: Not able to list the collection through REST #39587

nileshsironja commented Jan 24, 2025

nileshsironja commented Jan 24, 2025 •

edited

Loading

nileshsironja commented Jan 24, 2025

nileshsironja commented Jan 24, 2025

yanliang567 commented Jan 26, 2025

nileshsironja commented Jan 27, 2025

nileshsironja commented Jan 27, 2025 •

edited

Loading

xiaofan-luan commented Jan 28, 2025

nileshsironja commented Jan 28, 2025

xiaofan-luan commented Jan 28, 2025

ghost commented Jan 28, 2025 •

edited by ghost

Loading

xiaofan-luan commented Jan 28, 2025

xiaofan-luan commented Jan 28, 2025

ghost commented Jan 28, 2025

nileshsironja commented Jan 28, 2025 •

edited

Loading

xiaofan-luan commented Jan 29, 2025

xiaofan-luan commented Jan 29, 2025

nileshsironja commented Jan 29, 2025 •

edited

Loading

nileshsironja commented Jan 29, 2025

xiaofan-luan commented Jan 29, 2025

nileshsironja commented Jan 30, 2025

xiaofan-luan commented Jan 31, 2025

xiaofan-luan commented Jan 31, 2025

nileshsironja commented Jan 31, 2025

nileshsironja commented Feb 3, 2025

[Bug]: Not able to list the collection through REST #39587

[Bug]: Not able to list the collection through REST #39587

Comments

nileshsironja commented Jan 24, 2025

Is there an existing issue for this?

Environment

Current Behavior

Expected Behavior

Steps To Reproduce

Milvus Log

Anything else?

nileshsironja commented Jan 24, 2025 • edited Loading

nileshsironja commented Jan 24, 2025

nileshsironja commented Jan 24, 2025

yanliang567 commented Jan 26, 2025

nileshsironja commented Jan 27, 2025

nileshsironja commented Jan 27, 2025 • edited Loading

xiaofan-luan commented Jan 28, 2025

nileshsironja commented Jan 28, 2025

xiaofan-luan commented Jan 28, 2025

ghost commented Jan 28, 2025 • edited by ghost Loading

xiaofan-luan commented Jan 28, 2025

xiaofan-luan commented Jan 28, 2025

ghost commented Jan 28, 2025

nileshsironja commented Jan 28, 2025 • edited Loading

xiaofan-luan commented Jan 29, 2025

xiaofan-luan commented Jan 29, 2025

nileshsironja commented Jan 29, 2025 • edited Loading

nileshsironja commented Jan 29, 2025

xiaofan-luan commented Jan 29, 2025

nileshsironja commented Jan 30, 2025

xiaofan-luan commented Jan 31, 2025

xiaofan-luan commented Jan 31, 2025

nileshsironja commented Jan 31, 2025

nileshsironja commented Feb 3, 2025

nileshsironja commented Jan 24, 2025 •

edited

Loading

nileshsironja commented Jan 27, 2025 •

edited

Loading

ghost commented Jan 28, 2025 •

edited by ghost

Loading

nileshsironja commented Jan 28, 2025 •

edited

Loading

nileshsironja commented Jan 29, 2025 •

edited

Loading