Proposal for maintaining relative time during deep sleep on ESP32

[cmidgley]

Moving from a gitter.im discussion...

When an ESP32 runs in light sleep, both Date() (time of day) and Time() (milliseconds since boot) are able track time. However, in deep sleep, only Date() has the correct time upon wakeup. You can not just use Date() for tracking milliseconds, as operations such as timesync change the time and it is no longer relative to system boot.

To address this, rather than a prior proposal to track time spent asleep, I propose we always use the system clock (gettimeofday) for time (time of day, and time since boot) but then track time change requests as an offset to be used for relative time.

We would introduce a static time offset variable, millisecondsOffset in this example, that would be saved in RTC slow memory (default at 0) so that it is retained across deep and light sleeps. The modMilliseconds macro, which currently uses esp_timer_get_time, would be changed to use a function that consumes gettimeofday to get milliseconds and add the offset, such as:

#define modMilliseconds() modMillisecondsRTCOffset()

uint32_t RTC_SLOW_ATTR millisecondsOffset = 0;

uint32_t modMillisecondsRTCOffset() {
   struct timeval tv_now;
   gettimeofday(&tv_now, NULL);
   return tv_now.tv_sec - millisecondsOffset;
}

A similar approach would be taken for a modification to 'modSetTime' , to compute the delta between the requested time and the current time, and store that in the offset (millisecondsOffset).

This approach eliminates the need to know if we go to sleep or not, eliminates induced drift, doesn't need to execute before the VM starts or worry about fastest-time-to-execution and is totally contained within the host implementation (I believe everything is located in xsHost) and exposes no new APIs for developers to consume. Thanks to ESP-IDF, the gettimeofday uses the high accuracy clock and only downshifts to the low accuracy during sleep, so there should be no loss of accuracy.

The only downside I've thought of so far is perhaps there is a small performance penalty, but that would really depend on the implementation details of esp_timer_get_time and gettimeofday, and likewise on modSetTime.

[cmidgley]

Implemented in pull request #840. Conceptually what is mentioned above, but implementation different as I misunderstood some aspects of how this worked in xsHost and had some minor mistakes above. Tested and working in light and deep sleep on ESP32.

[phoddie]

I'm just getting to this one. Thanks for your patience.

This is an interesting problem. Your solution makes sense. Still, I'd prefer to avoid making it more expensive to retrieve milliseconds (or depending on an emulated POSIX call to have a fast implementation). That's the kind of call that should be as fast as practical.

What we need is a way to restore milliseconds to the expected value - the milliseconds value at deep sleep plus the time spent In deep sleep (and booting) A modMilliseccondsSet(s) API would calculate and store an offset from the time since boot. That offset would be applied by modMilliseconds() with a 32-bit integer addition.

As an added bonus, having an API to set milliseconds would make it realistic to write tests for issues like #875.

[cmidgley]

Just be careful on time measurement as I found that trying to calculate time asleep and using that had lots of accuracy issues (losing microseconds). I have a test bed that will sleep (deep or light) and wake the process over and over, and monitor time throughout the process. This is what I used to test this code, where it remained stable on time across >25K sleep/wake cycles. I can't easily share the code (as it builds on my rather complex build/library setup), but it's easy for me to run the test.

[phoddie]

Good point. Incrementally saving and restoring the time on each boot cycle isn't correct. With millisecond accuracy, there is an inaccuracy on the order of microseconds. With enough cycles, those small errors will accumulate to a noticeable inaccuracy.

A better approach is to record both the current time of day and current ticks once, on cold boot. On wake from deep sleep, the current milliseconds is calculated from the current and recorded times. Using the same recorded time for all delta calculations, avoids accumulating errors that lead to drift.