[SensorThings] Handling of server-side pagination for expanded entities

[lucagiovannini]

What is the bug or the crash?

SensorThings data provider doesn't handle server-side pagination of expanded entities, it stops at the first page.

Steps to reproduce the issue

Select Entity type = Things, limit it to 1
Expand to Datastreams, limit it to 1
Expand to Observations, set a limit greater than the server's pagination size.

In the example in the screenshot I am pointing to the following STA server (I'll provide read credentials privately to the debugger):
https://frost.labservice.it/FROST-Server/v1.1
The server has a pagination size = 100.

Things filtered with = id eq 'THI.FE.036'
Datastreams filtered with = substringof('CO2',name)
Observations filtered with = phenomenonTime ge 2024-11-12T23:00:00Z and phenomenonTime lt 2024-11-13T23:00:00Z
(well above 100 records)

The expected outcome corresponds to the following API query. The screenshot shows the nextPage pagination link.
https://frost.labservice.it/FROST-Server/v1.1/Locations?$expand=Things($filter=id%20eq%20%27THI.FE.036%27),Things/Datastreams($filter=substringof(%27CO2%27,name)),Things/Datastreams/Observations($filter=phenomenonTime%20ge%202024-11-12T23:00:00Z%20and%20phenomenonTime%20lt%202024-11-13T23:00:00Z)

The actual outcome has only 100 records.

Versions

QGIS version
3.38.2-Grenoble
QGIS code revision
130c432
Qt version
5.15.13
Python version
3.12.5
GDAL/OGR version
3.9.2
PROJ version
9.4.0
EPSG Registry database version
v11.004 (2024-02-24)
GEOS version
3.12.2-CAPI-1.18.2
SQLite version
3.45.1
PDAL version
2.6.3
PostgreSQL client version
16.2
SpatiaLite version
5.1.0
QWT version
6.2.0
QScintilla2 version
2.14.1
OS version
Windows 10 Version 2009

Active Python plugins
profiletool
4.1.8
QuickOSM
1.16.0
quick_map_services
0.19.26
SensorThingsAPI
1.1.12
SensorThingsAPI_2
1.0.1
db_manager
0.1.20
grassprovider
2.12.99
MetaSearch
0.3.6
processing
2.12.99

Supported QGIS version

• I'm running a supported QGIS version according to the roadmap.

New profile

• I tried with a new QGIS profile

Additional context

No response