max_requests and max_requests_jitter

[jisngprk]

I am using bentoml with gunicorn

when I use bentoml serve-gunicorn command, I want to use feature of max_requests, max_requests_jitter feature.

Because these config is not supported by bentoml, I patched gunicorn_config.py directly into bentoml site-packages path.

It looks like these config has applied. However, when the number of requests exceed the limit, 503, and Service unavilable message is returned.

It looks like the worker process does not restart properly. Is there any way to use these config ?

[withsmilo]

Hi, @jisngprk . 😃
I guess you're using BentoML-0.13.1 for your service now. If yes, you could apply your own patch to the following parts by yourself.
I think you can refer this commit also for your purpose. (1db97c3) I hope this helps you in some way.

https://github.com/bentoml/BentoML/blob/0.13-LTS/bentoml/server/gunicorn_model_server.py#L62-L65

[jisngprk]

Yes, you are right. I am using BentoML-0.13.1.

Already I patched gunicorn_config.py to load the config in 'load_config' function. However, It didn't work.

BentoML/bentoml/server/gunicorn_model_server.py

Line 72 in cc36fb0

def load_config(self):

self.load_config_from_file("python:bentoml.server.gunicorn_config")

I will try the solution that you suggested. Thank you.

[jisngprk]

It works! Thank you!

However, I can't understand the difference between applying 'max_requests' option into 'self.options' and 'python:bentoml.server.gunicorn_config' file.

why appending the option into the end of the file 'python:bentoml.server.gunicorn_config' didn't work?