Resolve the issue in how we create timestamp_difference_microseconds

[KolbyML]

This changes ethereum/utp to handle this case the same way fluffy/utp and bittorrent/libutp handle it.

The issue we faced before

Having a cap defeats the purpose of why we get this difference. The number it gives isn't inheriently valuable since system clocks or how implementations get the time won't be the same between systems/implementations. What this measurement is useful for is the relative time delays between each client. So if this number is high or low it doesn't matter. It just matters what the delay changes are relative to each other over multiple sent packets. Using this information is how ledbat works (uTP's congestion protocol) to yeild to TCP traffic.

Capping timestamp_difference_microseconds to 1 second defeats the purpose of this since then the other client thinks the delay on our side is constant which isn't what we want. We should always return the difference between the two times unless the other side packet time is 0

[carver]

Capping timestamp_difference_microseconds to 1 second defeats the purpose of this since then the other client thinks the delay on our side is constant

Good find!

unless the other side packet time is 0

What's this special case about? When would we expect it? (It seems like it would be a very unusual circumstance) Even in this unusual circumstance, why would it be a problem to just subtract it as normal?

[KolbyML]

What's this special case about? When would we expect it? (It seems like it would be a very unusual circumstance) Even in this unusual circumstance, why would it be a problem to just subtract it as normal?

https://github.com/bittorrent/libutp/blob/2b364cbb0650bdab64a5de2abb4518f9f228ec44/utp_internal.cpp#L2019
read this comment write here

on line 2001 that is the calculation we are doing

[jacobkaufmann]

Capping timestamp_difference_microseconds to 1 second defeats the purpose of this since then the other client thinks the delay on our side is constant which isn't what we want.

note that we do not cap the diff to 1s. the cap is the configured max idle timeout, and if packets are taking that long to transmit, then we should expect the connection to close due to an idle timeout. the existing logic is in place to account for situations of clock drift, which could lead to inaccurate observations about round trip time (RTT). if the remote's clock drifts ahead of the local clock, then we could be in a situation where we compute the duration between a timestamp of 100 microseconds (remote UNIX time) and a timestamp of 50 microseconds (local UNIX time) would compute the wrap around from 100 to the max value and back to 50.

[KolbyML]

note that we do not cap the diff to 1s. the cap is the configured max idle timeout, and if packets are taking that long to transmit, then we should expect the connection to close due to an idle timeout.

It sounds like you are assuming that all implementations use the same method of getting time. Which they don't. so if the other machines time is 500 and ours is 5000000. Which realistically happens we send back useless time differences.

the existing logic is in place to account for situations of clock drift, which could lead to inaccurate observations about round trip time (RTT). if the remote's clock drifts ahead of the local clock, then we could be in a situation where we compute the duration between a timestamp of 100 microseconds (remote UNIX time) and a timestamp of 50 microseconds (local UNIX time) would compute the wrap around from 100 to the max value and back to 50.

It sounds like you assume that implementations use the same gettime method which they don't and they aren't expected it.

The only thing that matters is one senders diffs compared against each other.

Your implementation always will send 1s back in 99% of cases. bittorrent/libutp and fluffly/utp use the same method of getting time. Even so on other machines it isn't like there times will match, but that doesn't matter. It only matters what the time diffs are relative to each other. if we only send 1s back the other client will think our delay is nonexistant.

What is the congestion algo you get the base_delay and average_delay both numbers will be the same, which is meaningless for the remote. The delay will always be seen as 0 or something constant I would have to look at the exact number again. But that tells the remote nothing and defeats the purpose of even using this congestion protocol

[jacobkaufmann]

Your implementation always will send 1s back in 99% of cases.

what is the basis of this claim? it will send 1s if the observed diff is greater than the idle timeout. why would that occur in 99% of cases? if you have references, then I would like to see, because I don't think I fully understand.

[KolbyML]

#74 git pull this

cd into the utp-client-interop folder

add #define UTP_DEBUG_LOGGING 1 to libutp/utp_internal.cpp for libutp to see the time diff easier

if you want you can make ethereum/utp send a megabyte

Test ethereum/utp send to bittorrent/libutp

Terminal 1

make rcrecv

Terminal 2

make rrsend

After this is running you can see the vast differences in timestamp yourself. And observe the values of the time_diff yourself. You will probably notice that libutp and eth/utp clocks are way off (this shouldn't matter for a working implementation). The probelm is adding a cap assumes all ways of getting the timestamp are the same between utp implementations. Which isn't true. Also it shouldn't even matter if they are the same as long is they are a monotonic clock. Since the congestion protocol only cares about the relative time_diffs. i.e. x amount of time diffs compared to each other

[KolbyML]

https://youtu.be/VfbRhSrJ4qA?t=1978 32:58 -> 35:34 I found a good clip from a talk the author of bittorrent Bram Cohen gave which supports my arguments for this change. Hopefully we can get this merged soon as a capped time doesn't make sense.

[carver]

I would love to have a failing test case that we can all agree should be passing. In some ways, that's even more valuable than the fix. That would probably be easier than hashing it out in conversation.

That might look like:

1. an integration test that builds and runs fluffy or libutp against utp in CI
2. make a variant of utp that can uses a different clock (monotonic that starts at 0), and connect it to a default utp instance

Connect the two instances over local UDP, and the delay should be well below 1s. It might be easier to write the test with something similar to #112 that exposes the delay as a statistic on the connection.

1 is probably the solution that will pay dividends, when we want to run other kinds of tests against a different implementation. 2 is probably less work to get going. Your call.