Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Geoserver: protect web interface and ows routes behind magpie/twitcher #348

Merged
merged 20 commits into from
Nov 1, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
20 commits
Select commit Hold shift + click to select a range
fa4d9e5
make all geoserver requests go through twitcher
mishaschwartz Jun 15, 2023
e55f2bb
add public permissions for geoserver
mishaschwartz Jun 16, 2023
d352bbe
allow access to geoserver web interface
mishaschwartz Jun 16, 2023
71a25ad
update CHANGES.md
mishaschwartz Jun 16, 2023
25f82ac
revert change handled in a different PR
mishaschwartz Jun 16, 2023
8b1df81
Merge branch 'master' into geoserver-behind-twitcher
mishaschwartz Jul 25, 2023
9504d38
Merge branch 'master' into geoserver-behind-twitcher
mishaschwartz Sep 14, 2023
5439ded
update to use new magpie geoserver service
mishaschwartz Sep 14, 2023
c4f8d68
upate CHANGES.md
mishaschwartz Sep 14, 2023
72a9ab4
documentation updates
mishaschwartz Sep 15, 2023
22d006c
deprecate test-geoserver-secured-access
mishaschwartz Sep 15, 2023
cea52e6
Revert "deprecate test-geoserver-secured-access"
mishaschwartz Sep 25, 2023
67b8d46
Merge branch 'master' into geoserver-behind-twitcher
mishaschwartz Sep 27, 2023
a096910
move geoserver behind twitcher verify path
mishaschwartz Sep 27, 2023
9e676f0
add GEOSERVER_SKIP_AUTH environment variable
mishaschwartz Sep 27, 2023
f99bda1
Merge branch 'master' into geoserver-behind-twitcher
mishaschwartz Oct 2, 2023
1bc0bab
add comment explaining GEOSERVER_SKIP_AUTH
mishaschwartz Oct 2, 2023
65f4a2a
bump magpie version that contains service type update features
mishaschwartz Oct 10, 2023
80c2e8a
Merge branch 'master' into geoserver-behind-twitcher
mishaschwartz Nov 1, 2023
ee4087f
Bump version: 1.36.0 → 1.37.0
mishaschwartz Nov 1, 2023
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 3 additions & 3 deletions .bumpversion.cfg
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
[bumpversion]
current_version = 1.36.0
current_version = 1.37.0
commit = True
tag = False
tag_name = {new_version}
Expand Down Expand Up @@ -30,11 +30,11 @@ search = {current_version}
replace = {new_version}

[bumpversion:file:RELEASE.txt]
search = {current_version} 2023-10-31T17:20:38Z
search = {current_version} 2023-11-01T13:09:36Z
replace = {new_version} {utcnow:%Y-%m-%dT%H:%M:%SZ}

[bumpversion:part:releaseTime]
values = 2023-10-31T17:20:38Z
values = 2023-11-01T13:09:36Z

[bumpversion:file(version):birdhouse/config/canarie-api/docker_configuration.py.template]
search = 'version': '{current_version}'
Expand Down
31 changes: 31 additions & 0 deletions CHANGES.md
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,37 @@

[//]: # (list changes here, using '-' for each new entry, remove this when items are added)

[1.37.0](https://github.com/bird-house/birdhouse-deploy/tree/1.37.0) (2023-11-01)
------------------------------------------------------------------------------------------------------------------

## Changes
- Geoserver: protect web interface and ows routes behind magpie/twitcher

Updates Magpie version to [3.35.0](https://github.com/Ouranosinc/Magpie/tree/3.35.0) in order to take advantage of
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Woops, the actual Magpie in this PR is 3.36 and not 3.35 !

updated Geoserver Service.

The `geoserverwms` Magpie service is now deprecated. If a deployment is currently using this service, it is highly
recommended that the permissions are transferred from the deprecated `geoserverwms` service to the `geoserver`
service.

The `/geoserver` endpoint is now protected by default. If a deployment currently assumes open access to Geoserver and
would like to keep the same permissions after upgrading to this version, please update the permissions for the
`geoserver` service in Magpie to allow the `anonymous` group access.

A `Magpie` service named `geoserver` with type `wfs` exists already and must be manually deleted before the new
`Magpie` service created here can take effect.

The `optional-components/all-public-access` component provides full access to the `geoserver` service for the
`anonymous` group in Magpie. Please note that this includes some permissions that will allow anonymous users to
perform destructive operations. Because of this, please remember that enabling the
`optional-components/all-public-access` component is not recommended in a production environment.

Introduces the `GEOSERVER_SKIP_AUTH` environment variable. If set to `True`, then requests to the geoserver endpoint
will not be authorized through twitcher/magpie at all. This is not recommended at all. However, it will slightly
improve performance when accessing geoserver endpoints.

See https://github.com/bird-house/birdhouse-deploy/issues/333 for details.

[1.36.0](https://github.com/bird-house/birdhouse-deploy/tree/1.36.0) (2023-10-31)
------------------------------------------------------------------------------------------------------------------

Expand Down
2 changes: 1 addition & 1 deletion Makefile
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
# Generic variables
override SHELL := bash
override APP_NAME := birdhouse-deploy
override APP_VERSION := 1.36.0
override APP_VERSION := 1.37.0

# utility to remove comments after value of an option variable
override clean_opt = $(shell echo "$(1)" | $(_SED) -r -e "s/[ '$'\t'']+$$//g")
Expand Down
8 changes: 4 additions & 4 deletions README.rst
Original file line number Diff line number Diff line change
Expand Up @@ -14,13 +14,13 @@ for a full-fledged production platform.
* - releases
- | |latest-version| |commits-since|

.. |commits-since| image:: https://img.shields.io/github/commits-since/bird-house/birdhouse-deploy/1.36.0.svg
.. |commits-since| image:: https://img.shields.io/github/commits-since/bird-house/birdhouse-deploy/1.37.0.svg
:alt: Commits since latest release
:target: https://github.com/bird-house/birdhouse-deploy/compare/1.36.0...master
:target: https://github.com/bird-house/birdhouse-deploy/compare/1.37.0...master

.. |latest-version| image:: https://img.shields.io/badge/tag-1.36.0-blue.svg?style=flat
.. |latest-version| image:: https://img.shields.io/badge/tag-1.37.0-blue.svg?style=flat
:alt: Latest Tag
:target: https://github.com/bird-house/birdhouse-deploy/tree/1.36.0
:target: https://github.com/bird-house/birdhouse-deploy/tree/1.37.0

.. |readthedocs| image:: https://readthedocs.org/projects/birdhouse-deploy/badge/?version=latest
:alt: ReadTheDocs Build Status (latest version)
Expand Down
2 changes: 1 addition & 1 deletion RELEASE.txt
Original file line number Diff line number Diff line change
@@ -1 +1 @@
1.36.0 2023-10-31T17:20:38Z
1.37.0 2023-11-01T13:09:36Z
Original file line number Diff line number Diff line change
Expand Up @@ -109,8 +109,8 @@ SERVICES = {
# NOTE:
# Below version and release time auto-managed by 'make VERSION=x.y.z bump'.
# Do NOT modify it manually. See 'Tagging policy' in 'birdhouse/README.rst'.
'version': '1.36.0',
'releaseTime': '2023-10-31T17:20:38Z',
'version': '1.37.0',
'releaseTime': '2023-11-01T13:09:36Z',
'institution': 'Ouranos',
'researchSubject': 'Climatology',
'supportEmail': '${SUPPORT_EMAIL}',
Expand Down Expand Up @@ -142,8 +142,8 @@ PLATFORMS = {
# NOTE:
# Below version and release time auto-managed by 'make VERSION=x.y.z bump'.
# Do NOT modify it manually. See 'Tagging policy' in 'birdhouse/README.rst'.
'version': '1.36.0',
'releaseTime': '2023-10-31T17:20:38Z',
'version': '1.37.0',
'releaseTime': '2023-11-01T13:09:36Z',
'institution': 'Ouranos',
'researchSubject': 'Climatology',
'supportEmail': '${SUPPORT_EMAIL}',
Expand Down
2 changes: 1 addition & 1 deletion birdhouse/config/geoserver/.gitignore
Original file line number Diff line number Diff line change
@@ -1,9 +1,9 @@
config/proxy/conf.extra-service.d/geoserver.conf
config/canarie-api/canarie_api_monitoring.py
config/magpie/providers.cfg
service-config.json

# Old paths. Keep these so that old config files remain uncommittable after updates.
geoserver_canarie_api_monitoring.py
geoserver-magpie-provider.cfg
config/proxy/canarie_api_monitoring.py
config/magpie/providers.cfg
12 changes: 12 additions & 0 deletions birdhouse/config/geoserver/config/magpie/providers.cfg
Original file line number Diff line number Diff line change
@@ -0,0 +1,12 @@
providers:
geoserver:
# below URL is only used to fill in the required location in Magpie
# actual auth validation is performed with Twitcher 'verify' endpoint without accessing this proxied URL
url: http://proxy:80
title: geoserver
type: geoserver
configuration:
wfs: true
wms: true
wps: false
api: true
16 changes: 0 additions & 16 deletions birdhouse/config/geoserver/config/magpie/providers.cfg.template

This file was deleted.

Original file line number Diff line number Diff line change
@@ -1,4 +1,7 @@
location /geoserver/ {
auth_request /secure-geoserver-auth;
auth_request_set $auth_status $upstream_status;

proxy_pass http://geoserver:8080/geoserver/;
proxy_set_header Host $host;
proxy_set_header X-Forwarded-Proto $real_scheme;
Expand All @@ -8,3 +11,20 @@
gzip_comp_level 1;
gzip_types application/json text/plain application/xml text/html;
}

location = /secure-geoserver-auth {
internal;
${GEOSERVER_SKIP_AUTH_PROXY_INCLUDE}
fmigneault marked this conversation as resolved.
Show resolved Hide resolved

# If GEOSERVER_SKIP_AUTH is "True" then the following section is skipped and this
# location block will always return 200 (which means that the /geoserver/ location, above,
# will be publicly available.
proxy_pass https://${PAVICS_FQDN_PUBLIC}${TWITCHER_VERIFY_PATH}/geoserver$request_uri;
proxy_pass_request_body off;
proxy_set_header Host $host;
proxy_set_header Content-Length "";
proxy_set_header X-Original-URI $request_uri;
proxy_set_header X-Forwarded-Proto $real_scheme;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Host $host:$server_port;
}
6 changes: 6 additions & 0 deletions birdhouse/config/geoserver/default.env
Original file line number Diff line number Diff line change
Expand Up @@ -26,9 +26,14 @@ export GEOSERVER_COMMUNITY_EXTENSIONS="geopkg-plugin"
# Must use single-quote for delayed eval.
export GEOSERVER_DATA_DIR='${DATA_PERSIST_ROOT}/geoserver'

# If set, requests to the geoserver endpoint will not be authorized through twitcher/magpie
export GEOSERVER_SKIP_AUTH=False
export GEOSERVER_SKIP_AUTH_PROXY_INCLUDE='$([ x"${GEOSERVER_SKIP_AUTH}" = x"True" ] && echo "return 200;")'
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@mishaschwartz Thanks for adding GEOSERVER_SKIP_AUTH this PR looks good for me as well.

Can you eventually add a similar switch for THREDDS. Currently we are applying a custom hack on our fork of this repo for production to disable Twitcher in front of our Thredds. Performance increased between 3 and 4x.

@fmigneault is there a DB upgrade? If no upgrade I won't need to deploy this PR to test and can approuve straight.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No DB upgrade in Magpie. The existing geoserser type is extended.


export DELAYED_EVAL="
$DELAYED_EVAL
GEOSERVER_DATA_DIR
GEOSERVER_SKIP_AUTH_PROXY_INCLUDE
"

# add any new variables not already in 'VARS' or 'OPTIONAL_VARS' that must be replaced in templates here
Expand All @@ -44,4 +49,5 @@ OPTIONAL_VARS="
\$GEOSERVER_VERSION
\$GEOSERVER_TAGGED
\$GEOSERVER_IMAGE
\$GEOSERVER_SKIP_AUTH_PROXY_INCLUDE
"
2 changes: 1 addition & 1 deletion birdhouse/config/magpie/default.env
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@
# are applied and must be added to the list of DELAYED_EVAL.

# Tag version that will be used to update Magpie API, Magpie CLI, and matching Twitcher with Magpie Adapter
export MAGPIE_VERSION=3.34.0
export MAGPIE_VERSION=3.36.0

export MAGPIE_DB_NAME="magpiedb"

Expand Down
3 changes: 3 additions & 0 deletions birdhouse/env.local.example
Original file line number Diff line number Diff line change
Expand Up @@ -484,6 +484,9 @@ export GEOSERVER_ADMIN_PASSWORD=geoserverpass
# (note: if using 'DATA_PERSIST_ROOT', it must be defined earlier, either in this file or from 'default.env')
#export WEAVER_MONGODB_DATA_DIR='${DATA_PERSIST_ROOT}/mongodb_weaver_persist'

# If "True", requests to the geoserver endpoint will not be authorized through twitcher/magpie
# (note: this is NOT recommended but will slightly improve performance when accessing geoserver endpoints)
#export GEOSERVER_SKIP_AUTH=True

#############################################################################
# Monitoring components configs
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
version: "3.4"
services:
magpie:
volumes:
- ./optional-components/all-public-access/config/geoserver/permissions.cfg:${MAGPIE_PERMISSIONS_CONFIG_PATH}/all-public-access-geoserver-permissions.cfg:ro
Original file line number Diff line number Diff line change
@@ -0,0 +1,70 @@
permissions:
- service: geoserver
type: route
permission: read
group: anonymous
action: create
- service: geoserver
permission: describestoredqueries
group: anonymous
action: create
- service: geoserver
permission: describelayer
group: anonymous
action: create
- service: geoserver
permission: dropstoredquery
group: anonymous
action: create
- service: geoserver
permission: lockfeature
group: anonymous
action: create
- service: geoserver
permission: getmap
group: anonymous
action: create
- service: geoserver
permission: getfeature
group: anonymous
action: create
- service: geoserver
permission: getfeaturewithlock
group: anonymous
action: create
- service: geoserver
permission: getfeatureinfo
group: anonymous
action: create
- service: geoserver
permission: getgmlobject
group: anonymous
action: create
- service: geoserver
permission: getpropertyvalue
group: anonymous
action: create
- service: geoserver
permission: transaction
group: anonymous
action: create
- service: geoserver
permission: createstoredquery
group: anonymous
action: create
- service: geoserver
permission: getlegendgraphic
group: anonymous
action: create
- service: geoserver
permission: getcapabilities
group: anonymous
action: create
- service: geoserver
permission: describefeaturetype
group: anonymous
action: create
- service: geoserver
permission: liststoredqueries
group: anonymous
action: create
4 changes: 2 additions & 2 deletions docs/source/conf.py
Original file line number Diff line number Diff line change
Expand Up @@ -69,9 +69,9 @@
# built documents.
#
# The short X.Y version.
version = '1.36.0'
version = '1.37.0'
# The full version, including alpha/beta/rc tags.
release = '1.36.0'
release = '1.37.0'

# The language for content autogenerated by Sphinx. Refer to documentation
# for a list of supported languages.
Expand Down
Loading