Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Connection hang up with gunicorn + uvicorn workers #1292

Open
JanHoellmer opened this issue Sep 13, 2024 · 7 comments
Open

Connection hang up with gunicorn + uvicorn workers #1292

JanHoellmer opened this issue Sep 13, 2024 · 7 comments
Labels
grpc A problem with the underlying gRPC protocol

Comments

@JanHoellmer
Copy link

When you connect to a Weaviate instance in the Gunicorn on_startup server hook and close the connection, afterwards I'm unable to connect again inside the uvicorn workers (deployment setup for FastAPI application).

gunicorn.conf.py

import weaviate
def on_starting(server):
	client = weaviate.connect_to_custom(
            http_host=localhost,
            http_port=8080,
            http_secure=False,
            grpc_host=localhost,
            grpc_port=50051,
            grpc_secure=False,
        )
	client.close()

app.py (FastAPI)

from fastapi import FastAPI
from contextlib import asynccontextmanager
import weaviate

@asynccontextmanager
async def setup_databases(app: FastAPI):
    client = weaviate.connect_to_custom(
            http_host=localhost,
            http_port=8080,
            http_secure=False,
            grpc_host=localhost,
            grpc_port=50051,
            grpc_secure=False,
        )

    yield

    weaviate_client.close()

app = FastAPI(lifespan=setup_databases)

Run with gunicorn -b 0.0.0.0:8000 -c gunicorn.conf.py app:app.

Inside the on_starting hook, i can close and connect as often as I want, same for inside the FastAPI application. As soon as i close the connection to Weaviate once in the on_starting hook, connect_to_custom hangs up inside the application. This first happened with Weaviate python client version 4.7.0 (4.7.1 & 4.8.0 also don't work), for versions <= 4.6.7 everything works fine.

Weaviate Server Version: 1.26.1

@dirkkul
Copy link
Collaborator

dirkkul commented Sep 13, 2024

that is probably because we migrated the client to use async internally - could you try using our async client?

Here are our docs and a fastAPI example
https://weaviate.io/developers/weaviate/client-libraries/python/async
https://github.com/weaviate/weaviate-python-client/blob/main/journey_tests/test_fastapi.py

@tsmith023
Copy link
Contributor

Hi @JanHoellmer, I'll look into exactly what is causing the hang with gunicorn but be advised that this deployment method for FastAPI is deprecated and the current recommendation is to deploy using just uvicorn

@JanHoellmer
Copy link
Author

Hi @JanHoellmer, I'll look into exactly what is causing the hang with gunicorn but be advised that this deployment method for FastAPI is deprecated and the current recommendation is to deploy using just uvicorn

Thanks for the hint, have not seen that yet.

@tsmith023
Copy link
Contributor

If you choose to move away from gunicorn, it would be interesting to know whether this fixes your problem here. Likewise, if you refactor to use the WeaviateAsyncClient object it would be great to know if that resolves the issue!

@JanHoellmer
Copy link
Author

JanHoellmer commented Sep 13, 2024

I tested both:

  • Moving away from gunicorn to just uvicorn helps, but probably because there is no direct substitute for the on_startup server hook from gunicorn.
  • Using WeaviateAsyncClient, while still using gunicorn, also fixes the hang up. In on_startup you can still use the sync WeaviateClient, but as soon as you only use WeaviateAsyncClient on the FastAPI side, the connection can be astablished normally. (I tested the other way around, using WeaviateAsyncClient in gunicorn, and WeaviateClient in FastAPI -> also works!)
    So the problem only persists when using WeaviateClient on both ends.

But thanks again for the hint with uvicorn, I'm gonna migrate, but maybe the bug is still relevant for other setups.

@tsmith023
Copy link
Contributor

@JanHoellmer, when you say

on_startup server hook from gunicorn

do you mean this one?

If so, this is entirely inline with the client implementation as it is not multi-processing safe so if it is initialised in one process and then used in another then it won't function correctly as you observed!

If not, which other hook were you using specifically that was causing the hanging behaviour?

@JanHoellmer
Copy link
Author

Yes, it is the one you linked.
But the async client is multi-processing safe? Or why does it not happen there, that when i initialize and close the connection in one proces (in the on_startup hook), i can't initialize it again in another process?

@tsmith023 tsmith023 added the grpc A problem with the underlying gRPC protocol label Oct 14, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
grpc A problem with the underlying gRPC protocol
Projects
None yet
Development

No branches or pull requests

3 participants