Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

WARNING : tensorflow:Callback method on_train_batch_end is slow compared to the batch time #131

Open
essanhaji opened this issue Jan 5, 2021 · 2 comments

Comments

@essanhaji
Copy link

tensorflow:Callback

Hi,
Thank you very much for this amazing tool it helps a lot.
Just a have a simple warning that will make this tool great if it is fixed.
This warning occurs when I use PlotLossesKeras callback.

WARNING:tensorflow:Callback method `on_train_batch_end` is slow compared to the batch time (batch time: 0.1298s vs `on_train_batch_end` time: 0.2397s). Check your callbacks.

Environment

  • livelossplot==0.5.3
  • tensorflow==2.4.0
  • Keras==2.4.3
  • Windows 10
  • Jupyter Lab
  • Python version 3.6.8

Thank you

@Bartolo1024
Copy link
Collaborator

Do you use Livelossplot without any other callbacks? We don't have on_train_batch_end method in our callback (only 'on_epoch_end').

@Noah-Junseo
Copy link

Noah-Junseo commented May 20, 2022

Multiprocessing supported by TensorFlow itself is not supported on Windows.
This is a suggestion to improve it by using the multiprocessing provided by Python directly.
https://docs.python.org/en/3.7/library/multiprocessing.html

And, to see if multiprocessing is running, you need to look at what each process is doing, not the overall cpu stats.

In Linux, you can check by simply creating and disappearing multiple python processes with the same command as top, but I don't know Windows well, so if you google it, I think it might not come up.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants