Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

support multiple URLs/sources for vector tiles #58155

Merged
merged 8 commits into from
Sep 2, 2024

Conversation

3nids
Copy link
Member

@3nids 3nids commented Jul 17, 2024

This is a draft/POC to allow a discussion.

Goal: rendering Mapbox Vector Tile layer having multiple sources

for instance: https://vectortiles.geo.admin.ch/styles/ch.swisstopo.basemap.vt/style.json
(see online: https://cms.geo.admin.ch/fmc/bm.html)

{
  "version": 8,
  "id": "2fa0964e-8ce3-4fe9-91b3-2a0d18663c9d",
  "name": "basemap_v1.15.0",
  "sources": {
    "terrain_v1.0.0": {
      "url": "https://vectortiles.geo.admin.ch/tiles/ch.swisstopo.relief.vt/v1.0.0/tiles.json",
      "type": "vector"
    },
    "base_v1.0.0": {
      "url": "https://vectortiles.geo.admin.ch/tiles/ch.swisstopo.base.vt/v1.0.0/tiles.json",
      "type": "vector"
    }
  },
  "layers": [

This cannot be solved by using several QGIS layers since the symbols ordering doesn't necessarily match with the layers/sources.

The idea is to allow listing several URLs in the layer. The rendering of each tile will be done at once when every source are fetched.

I have adapted QgsVectorTileRawData so that it contains a list of QByteData instead of a single one. The MVT decoder has been adapted accordingly.

For the download algorithm I suppose we need to recombine the data into a single array.

The layer names are considered unique across sources. It might be necessary to map layer names to the proper source.

I have successfuly tested this for asynchronous tile fetching. Sync is implemented but not tested.

There is no UX yet to play with this, it will be done if the approach is accepted.
But here is a project to play with: mapbox-multi-sources.qgs.zip

Screenshot with the log of the 2 sources fetched:
image

Looking forward to read your thoughts :)

@github-actions github-actions bot added this to the 3.40.0 milestone Jul 17, 2024
@nirvn
Copy link
Contributor

nirvn commented Jul 17, 2024

@3nids , really happy about this one, it was literally less than two months ago that I had stumbled on this very issue.

Copy link

github-actions bot commented Jul 17, 2024

🪟 Windows builds ready!

Windows builds of this PR are available for testing here. Debug symbols for this build are available here.

(Built from commit 4f79b5b)

src/core/vectortile/qgsvectortiledataprovider.h Outdated Show resolved Hide resolved
src/core/vectortile/qgsvectortileloader.h Outdated Show resolved Hide resolved
src/core/vectortile/qgsvectortileloader.h Outdated Show resolved Hide resolved
src/core/vectortile/qgsxyzvectortiledataprovider.cpp Outdated Show resolved Hide resolved
Comment on lines 170 to 176
#if 0
// even if a feature has an internal ID, it's not guaranteed to be unique across different
// tiles. This may violate the specifications, but it's been seen on mbtiles files in the wild...
if ( feature.has_id() )
fid = static_cast<QgsFeatureId>( feature.id() );
else
#endif
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we just remove this block altogether? Or Re-enable it the tiles size == 1 (i.e. when we only have one source)?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@3nids any feedback on this one?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I did not added this ifdef. I dropped it.

@3nids
Copy link
Member Author

3nids commented Jul 23, 2024

@wonder-sk @troopa81 @nyalldawson @rouault do you have an opinion on this approach?

@github-actions github-actions bot added the stale Uh oh! Seems this work is abandoned, and the PR is about to close. label Aug 7, 2024
@qgis qgis deleted a comment from github-actions bot Aug 14, 2024
@github-actions github-actions bot removed the stale Uh oh! Seems this work is abandoned, and the PR is about to close. label Aug 14, 2024
@m-kuhn
Copy link
Member

m-kuhn commented Aug 26, 2024

Thank you for this, I am excited to see this land !

@nirvn are the latest updates ok for you?
If yes, should we merge it this week to have enough cooldown time before release?

@nirvn
Copy link
Contributor

nirvn commented Aug 28, 2024

@m-kuhn , yep, I'm good with the latest updates.

@3nids 3nids merged commit 4d150a8 into qgis:master Sep 2, 2024
29 checks passed
@3nids 3nids deleted the mvt-multiple-sources branch September 2, 2024 07:08
@zacharlie zacharlie added the Changelog Items that are queued to appear in the visual changelog - remove after harvesting label Sep 24, 2024
@qgis-bot
Copy link
Collaborator

@3nids

This pull request has been tagged for the changelog.

  • The description will be harvested so please provide a "nearly-ready" text for the final changelog
  • If possible, add a nice illustration of the feature. Only the first one in the description will be harvested (GIF accepted as well)
  • If you can, it's better to give credits to your sponsor, see below for different formats.

You can edit the description.

Format available for credits
  • Funded by NAME
  • Funded by URL
  • Funded by NAME URL
  • Sponsored by NAME
  • Sponsored by URL
  • Sponsored by NAME URL

Thank you!

@zacharlie zacharlie added ChangelogHarvested This PR description has been harvested in the Changelog already. and removed Changelog Items that are queued to appear in the visual changelog - remove after harvesting labels Sep 24, 2024
@DelazJ
Copy link
Contributor

DelazJ commented Oct 2, 2024

@3nids what is the status of this feature in regards of the docs? I've seen a series of mvt PRS but can't connect the points enough to tell if and what we could mention in docs...

@3nids
Copy link
Member Author

3nids commented Oct 2, 2024

This is the one #58663

@DelazJ
Copy link
Contributor

DelazJ commented Oct 4, 2024

Thanks @3nids

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
ChangelogHarvested This PR description has been harvested in the Changelog already. Feature Vector tiles
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants