NPT-14150: Logging timestamps in milliseconds leads to no data getting logged for async mode and confusing HTTPServiceUnavailable error

[cemlyn007]

Describe the bug

Logging timestamp milliseconds a.k.a timestamp seconds multiplied by 1000 appears to cause at least the async logger to hang indefinitely and never upload the logs. Whilst it feels pretty obvious that you guys want timestamp seconds to be logged, I stumbled across this bug because I accidentally converted my nanoseconds to milliseconds when I meant to convert them to seconds

# Python 3.12
# Neptune 1.8.2

import time
import neptune

step_start = 0
step_end = 4

start_timestamp_ns = time.time_ns()
time.sleep(1.0)
end_timestamp_ns = time.time_ns()

# These timestamps are in milliseconds, and won't log to Neptune
timestamps = [
    (
        start_timestamp_ns
        + i * ((end_timestamp_ns - start_timestamp_ns) / (step_end - step_start))
    )
    / 1000000
    for i in range(1, step_end - step_start + 1)
]

# # Uncomment this and see if the logging now works
# # Now these timestamps are in seconds, and should log to Neptune
# timestamps = [t / 1000 for t in timestamps]


with neptune.init_run() as run:
    run['metric'].extend([0] * len(timestamps), steps=list(range(step_start, step_end)), timestamps=timestamps)

Expected behavior

Would recommend if possible that an error is returned regarding the timestamp instead of the Neptune client occasionally logging that it can not log the data and display an error in the log message HTTPServiceUnavailable and hanging indefinitely.

[SiddhantSadangi]

Thanks for sharing the details @cemlyn007 🙇

I was able to reproduce this, and have passed this to the engineering team.

I'll keep this thread updated on the progress.