feat: add extra args of feast serve cli in gunicorn #4693
Conversation
Signed-off-by: young-hun-jo <[email protected]>
| @@ -1894,6 +1894,9 @@ def serve( | |||
| type_: str = "http", | |||
| no_access_log: bool = True, | |||
| workers: int = 1, | |||
| threads: int = 1, | |||
| max_requests: int = 0, | |||
There was a problem hiding this comment.
how the server behaves if someone won't set this field and defaults to 0.
There was a problem hiding this comment.
if threads is set to zero(0), the only one thread is run in server.(In addtion, workers can be set to zero, too. But this is only running a master process not worker process. so can't process client's request. I test it in FastAPI in localhost). I rather worry about this worker argument set to zero.
And if max_requests is set to zero(0), the automatic restart process is disabled(ref: gunicorn docs)
sdk/python/feast/feature_server.py
Outdated
| @@ -369,6 +372,9 @@ def start_server( | |||
| bind=f"{host}:{port}", | |||
| accesslog=None if no_access_log else "-", | |||
| workers=workers, | |||
| threads=threads, | |||
| max_requests=max_requests, | |||
| max_requests_jitter=max_requests_jitter, | |||
There was a problem hiding this comment.
is this configuration only relevant in non windows platform?
There was a problem hiding this comment.
@lokeshrangineni yes. I know it. There is a condition
sdk/python/feast/cli.py
Outdated
| type=click.INT, | ||
| default=1, | ||
| show_default=True, | ||
| help="Number of thread" |
There was a problem hiding this comment.
could we add a bit verbose help especially for the beginners. when to increase threads or how it impacts? or may be reference to unicorn documentation.
There was a problem hiding this comment.
@lokeshrangineni
yes. I commit it more. A documentation link is so long, it may worsen visibility of source code, so I wrote the summary
There was a problem hiding this comment.
@lokeshrangineni
And short-cut of thread options doesn't exist in official gunicorn documentation. So, I remove this it, too
What this PR does / why we need it:
Python feast feature server is using FastAPI and gunicorn and it is run by command feast serve. When I search extra optons in gunicorn of feast serve, only few options can be used(for example,
keepalive, workers). So I add extra arguments of other gunicorn options. These arethreads, max_requests, max_requests_jitter.I would run Python feast feature server on EKS environments on production. In this production environments, so many requests is come to feast feature server but only workers options is not enough to run feature server on production.
Especially, I add these arguments for python memory leaking because of automated python's own garbage collector. Combination of
max_requests and max_requests_jitteris appropriate for preventing circumstances that multi-worker process is simultaneously restarted overmax_requests.If these options is added, python feast feature server on production is running when so many requests is come to the server. I looking forward to reviewing yours. thanks.
Which issue(s) this PR fixes:
Misc