Specify the maximum number of elements per page each endpoint can return. #359
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
General warning
Note: This PR is the first step in my desire to provide a way for the user to request a specific number of result on endpoint allowing pagination, as discussed with several developers on Discord and completing the work previosuly started in #276 (and thus use the internal pagination of the request function, which is currently not used by high-level functions!). This feature is not implemented in this PR.
However, the features implemented in this PR can be considered independent and already offer new cool stuff, so I propose it for review.
In this PR, I make a difference between the
first
parameter, which is the parameter sent to Twitch API when performing the HTTP request, and thefirst
argument exposed by some methods asUser.fetch_pools()
Changes
In the current version of TwitchIO, the number of elements requested to endpoints when pagination is enabled is 100 by default, this is set by the get_limit() subfunction:
This leads to two main issues:
first
is less than 100, we have to do a little trick by adding a "first" parameter when calling thefetch_
method, this is currently the case for 4 methods:get_reward_redemptions
,get_polls
,get_predictions
,get_channel_schedule
. These functions are the ONLY functions exposing afirst
parameter. All the other ones suppose by default thatfirst=100
due to the get_limit() function.This PR fix all of this by specifying the maximum value allowed by
first
when requesting the endpoint. This is done by adding a new parameter,max_elements_per_page
to the request() method, containing this information.Below, I listed all endpoint using the "first" parameter, with their min, default and max value. For each endpoint, the value of the
max_elements_per_page
argument is the maximum value allowed tofirst
by the documentation.(Note that for /videos, we can specify
first
parameter only if we specify the game_id or user_id query parameter. This condition has been implemented in this PR)In
fetch_
methods exposing afirst
argument, it has been rerouted to thelimit
attribute of request() to avoid any breaking change.Moreover, the following functions:
Can implement pagination, but were delivered with pagination=False.
Activate the pagination in these functions doesn't change or break anything and allow them to benefit from this PR. So I did it.
Finally, I disabled the assertion breaking the request when full_body was True as the same time as paginate, since the body is returned before the evaluation of paginate. This will prevent any futher bug in the future.
I tested all helix endpoints with both app token and user token to validate this PR.
Checklist