Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add support for pipeline read-aheads and partial reads #212

Closed
gavv opened this issue Jun 6, 2019 · 5 comments
Closed

Add support for pipeline read-aheads and partial reads #212

gavv opened this issue Jun 6, 2019 · 5 comments
Labels
duplicate Already addressed by another issue or pull request

Comments

@gavv
Copy link
Member

gavv commented Jun 6, 2019

We need to rework packet reader interface:

@gavv
Copy link
Member Author

gavv commented Jun 6, 2019

Depends on #211.

@gavv
Copy link
Member Author

gavv commented Jun 11, 2019

It makes sense to do this together with #183 because both require changes in the same pipeline components.

@gavv
Copy link
Member Author

gavv commented Jun 11, 2019

After much thought, I came to the following interface:

packet::IReader {
    // Shift reading position.
    // After this call, it is guaranteed that packets
    // before @ts are never returned.
    bool shift(timestamp_t ts);

    // Read next packet.
    // Returns a packet that:
    //  - was not returned yet
    //  - is after the reading position set by shift()
    //  - has the minimum timestamp among all such packets
    PacketPtr read();
};

Here are the differences from the current interface:

  • the reading position is controlled manually by the caller
  • the packet stream produced by read() is no longer monotonic; if a delayed packet arrived that is before the previously returned packet bu still after the reading position, it is okay to return it

Every interface implementer will need to take care of never returning the same packet twice. It would be trivial for some readers (e.g. SortedQueue) but would require some special handling for another (e.g. fec::Reader). We need this because the callers want to handle only newly received packets.

@gavv gavv self-assigned this Jun 13, 2019
@gavv gavv added this to the 0.1.1 milestone Jun 13, 2019
@gavv gavv removed their assignment Jun 13, 2019
@gavv gavv removed this from the 0.1.1 milestone Jun 13, 2019
@gavv
Copy link
Member Author

gavv commented Jun 23, 2019

It seems that it will be useful to make a similar change in the audio::IReader interface: separate shift() and read() methods and thereby allow read-aheads without advancing read position. This will allow us to implement PLC as a separate pipeline element without additional latency cost.

PLC element will be placed right after Depacketizer. When Depacketizer returns complete frames, PLC just forwards them. When Depacketizer returns an incomplete frame, PLC may perform (depending on the PLC implementation) a read-ahead request to read the next frame and use it for extrapolation.

Without this change, PLC should be done either part of Depacketizer, or will increase latency, or will not be able to use future frames from extrapolation.

Read-aheads may be also used to pre-fill PulseAudio buffer above the target latency and then, when the read position will be actually shifted, rewrite it with newly rendered samples. Theoretically, this can increase robustness on low latencies.

@gavv gavv changed the title Rework packet reader interface Add support for pipeline read-aheads Jun 23, 2019
@gavv
Copy link
Member Author

gavv commented Jun 23, 2019

To extract PLC into a separate element, we would also need to allow partial reads, so that every pipeline element can return fewer samples than requested, and every caller is ready to repeat the call when necessary.

This rather simple change will allow us to ensure that frame flags are relevant to all samples in the frame, by "splitting" the frame if necessary. In particular, Depacketizer may split blank (i.e. set to zero because the corresponding packet was lost) and non-blank parts of the frame into separate frames, and the PLC element will be able to reliably distinguish which samples are blank and which are not.

Without this change, every frame may mix both blank and non-blank samples and it's not possible to say which samples are blank by looking at the frame.

@gavv gavv changed the title Add support for pipeline read-aheads Add support for pipeline read-aheads and partial reads Jun 23, 2019
@gavv gavv added duplicate Already addressed by another issue or pull request and removed enhancement labels Dec 19, 2019
@gavv gavv closed this as completed Dec 19, 2019
@gavv gavv added this to Roc Toolkit Jul 6, 2024
@gavv gavv moved this to Done in Roc Toolkit Jul 6, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
duplicate Already addressed by another issue or pull request
Projects
Status: Done
Development

No branches or pull requests

1 participant