Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Build Log classifier] Long log files give error when sent for classification to the Seldon service #461

Open
Shreyanand opened this issue Dec 15, 2021 · 0 comments

Comments

@Shreyanand
Copy link
Member

Shreyanand commented Dec 15, 2021

Describe the bug
When we send a request to the service at this point http://build-log-clustering-ds-ml-workflows-ws.apps.smaug.na.operate-first.cloud/predict, we see a 502 error when the log file is huge (>100,000 chars).

To Reproduce
Steps to reproduce the behavior:

  1. Make sure you have the local cache of data by running the build_log_EDA notebook.
  2. Go to model_seldon.ipynb
  3. Select a log file with length greater than 100,000
  4. Send it to the service endpoint using the notebook
  5. See Remote disconnected or Error 502 from server

Expected behavior
The log file predictions should be available for all log files with server status 200.

Additional context
On inspecting the pod logs, it seems the service can't find json in these cases. I think it is either because the length of the log files makes them difficult to send through the post request or because these files are big, they have a higher chance of having json parse errors.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant