Command queuing, mutex, and FT8 priority

[jeffkowalski]

I'm preserving here some prior comments from Brian concerning the matter of queuing commands from SOTAmat to SOTAcat, avoiding unwanted interruptions, and preserving the critical timing of FT8:

and the mechanism / queue for access to the radio has to have some concept of absolute priority so that I can "skip the line" or "freeze the queue" when doing an FT8 sequence. For example: a call to ask if the radio is still connected or to get the battery level should be at the back of the line compared to FT8 work.

There are a few places where FT8 timing matters:

When we get the command that we need to prepare for FT8. There are several settings we have to change on the radio, and some of those settings take a very long time: in the order of seconds. For example, we may need to change bands and all the relays may need to get to the right ATU state, filter state, etc. The timing of this matters is because:
The SOTAMAT app has the "clock" and knows when the FT8 "start" time boundary is. It has to coordinate with the ESP32 as to which FT8 window start time we are going to use. If we look at the clock and we have a start window that is 1 second in the future, but the KX relays take longer than 1 second to get into the "prepared" state, then we have a problem. So currently SOTAMAT and SOTACAT have some agreements that when we are close to an FT8 start window, it might be better to delay until the following start window. Any commands that come in (asking the radio if it is awake) can add 100ms to that timing and throw off the algorithm.
Once both SOTAMAT and SOTACAT agree on a time to start, SOTACAT has a special FT8 watchdog timer that covers the case when we lose connection to the mobile phone. If something goes wrong, the SOTACAT watchdog times out and it will restore all the KX settings back to their original non-FT8 state. We don't want that watchdog KX communication or timer to be interfered with by some low priority "are you awake" command.
Once SOTACAT starts transmitting FT8, we have a rigid 160ms window to pump UART commands with. This seems like a long time considering the device runs in the many MHz, however the ESP-IDF RTOS has a concept of "Ticks", and there are only so many ticks per second (it is programmable, but often 100Hz). Coding is simpler if we use "tick" based delays for the 160ms steps rather than lower level clocks that "fight" with the RTOS. The RTOS allows us to have a high-priority thread that will have absolute priority over lower priority threads, but we don't want other UART activity happening when in the middle of FT8 transmission as it will take a "tick" to task switch.
When done we have to restore the radio with the watchdog code. Restoring some settings requires we put the radio in special modes before we can access certain parameters, and then we switch back. We don't want some other thread interrupting that sequence.

[jeffkowalski]

Let's agree that:

1. The radio is a critical resource.
2. Actions requiring the radio can be initiated by the user. (Perhaps these are of higher priority, if priority comes into play).
3. Actions requiring the radio may also be triggered by the UI to inquire and supply status indicators, etc.
   It seems therefore that
4. the radio resource should be lockable for exclusive access
5. exclusive access should initiate at worst by receipt from outside caller, perhaps with finer scoping
6. cases for long-term transactions (setup..act..destruct) should be carefully examined to determine if locking should be scoped to outer or inner transactional scope. Obvious complications arise during multi-interaction transactions that may fail (i.e setup fails, and we never get to act, or act fails, etc. - who holds that state?)
7. fine grained locking may allow for invasion. If we setup, then have an interrupter that locks as we attempt "act", are we deficient?