Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Mapper frontend basemap management pt2 #1925

Open
wants to merge 5 commits into
base: development
Choose a base branch
from

Conversation

spwoodcock
Copy link
Member

@spwoodcock spwoodcock commented Nov 28, 2024

What type of PR is this? (check all applicable)

  • πŸ• Feature
  • πŸ› Bug Fix
  • πŸ“ Documentation
  • πŸ§‘β€πŸ’» Refactor
  • βœ… Test
  • πŸ€– Build or CI
  • ❓ Other (please specify)

Related Issue

Continuation from #1922
Fixes #1552

Describe this PR

  • Added a migration to upload all basemaps on the filesystem to the S3 bucket + replace url field in the db.
    • The migration is idempotent / skips files already done, or that don't exist.
    • It also cleans up FAILED basemap download entries + their associated BackgroundTask entries.
  • Now we access the url directly from the S3 bucket in the frontend, instead of calling the backend API (commented out the download endpoint until we need it for private bucket download / pre-signed).
  • The mapper frontend can load a pmtiles file from a URL, or download to OPFS and use offline.
  • If there is an OPFS pmtiles file already present when loading the mapper frontend, it is added to the basemap selection automatically (OPFS files persist across restarts etc).
    • Also renamed {projectId}/all.pmtiles to {projectId}/basemap.pmtiles in the OPFS storage.

Screenshots

image

image

image

image

image

Alternative Approaches Considered

  • The long term plan is to replace the basemap functionality in FMTM and outsource to a microservice hotosm/basemap-api.
  • This solution will do for now, until we migrate.

Review Guide

  • The migration script must be run manually on the server after deploy!
docker compose exec api bash
cd app
python upload-basemaps-s3.py                                           

Checklist before requesting a review

[optional] What gif best describes this PR or how it makes you feel?

@github-actions github-actions bot added enhancement New feature or request frontend Related to frontend code backend Related to backend code migration Contains a DB migration labels Nov 28, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
backend Related to backend code enhancement New feature or request frontend Related to frontend code migration Contains a DB migration
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Better handle PMTile offline map paths
3 participants