Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Test out OWS GetMap/GetTile caching in DEA Dev OWS #1138

Merged
merged 2 commits into from
Jan 31, 2023

Conversation

omad
Copy link
Contributor

@omad omad commented Jan 30, 2023

My attempt last week to enable caching failed, see opendatacube/datacube-ows#921

I think I know what went wrong, there was no max_datasets set for the layers, and the caching logic decided not to cache anything.

I'm not 100% sure what a reasonable max_datasets is, so I've copied the value of 24 from the WCS config.

This change is against DEA Dev OWS, if it's successful, we can make it for Prod too.

omad added 2 commits January 31, 2023 09:49
- Enable the same caching in DEA Dev OWS as in DEA Prod OWS
- Attempt to enable caching of GetMap/GetTile requests in DEA Dev OWS
@SpacemanPaul
Copy link
Collaborator

That should work - although it does mean that all non-empty tiles will be cached for at least one hour.

The risk with that is that the cache gets filled up with single-dataset tiles which can be served quickly on the fly, pushing out the more expensive multi-dataset tiles which take much longer to serve. But probably worth testing.

We don't usually apply max_dataset limits to WMS/WMTS layers. 24 should be fine though for most products though.

@omad
Copy link
Contributor Author

omad commented Jan 31, 2023

CloudFlare is mostly a black box, but the documentation indicates that tiles don't get "pushed out". It can respond much faster than OWS can.

We don't usually apply max_dataset limits to WMS/WMTS layers. 24 should be fine though for most products though.

See opendatacube/datacube-ows#921, if max_datasets isn't set, it's impossible to cache Map/Tile requests. I think it's a bug

@codecov-commenter
Copy link

Codecov Report

Merging #1138 (5687d58) into master (16ad4d1) will not change coverage.
The diff coverage is n/a.

Impacted file tree graph

@@           Coverage Diff           @@
##           master    #1138   +/-   ##
=======================================
  Coverage   99.29%   99.29%           
=======================================
  Files         120      120           
  Lines         993      993           
=======================================
  Hits          986      986           
  Misses          7        7           
Impacted Files Coverage Δ
dev/services/wms/ows_refactored/ows_reslim_cfg.py 100.00% <ø> (ø)
dev/services/wms/ows_refactored/ows_root_cfg.py 100.00% <ø> (ø)
prod/services/wms/ows_refactored/ows_reslim_cfg.py 100.00% <ø> (ø)

Copy link
Collaborator

@SpacemanPaul SpacemanPaul left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I agree there is an OWS bug to require max_datasets.

This should work though I think.

@@ -92,6 +92,8 @@
"s3_aws_zone": "ap-southeast-2",
"max_width": 512,
"max_height": 512,
# Allow the WMS/WMTS GetCapabilities responses to be cached for 1 hour
"caps_cache_maxage": 3600,
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In general I would prefer something like 60 * 60 - but 3600 should be familiar enough as the number of seconds in an hour.

@omad omad merged commit aa3c0b4 into master Jan 31, 2023
@omad omad deleted the fix/ows-map-tile-caching branch January 31, 2023 02:02
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants