Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

New OpenAir export format for aerial sensitive areas (#2372) #3349

Merged
merged 67 commits into from
Sep 28, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
67 commits
Select commit Hold shift + click to select a range
648e3d5
Add openair generator in SensitiveArea Model
lpofredc Oct 22, 2020
197e161
Add OpenAir SenitiveArea view (for aerial practices)
lpofredc Oct 22, 2020
85ae35f
Add OpenAir SenitiveArea url (by area)
lpofredc Oct 22, 2020
e3b3957
Add header comment on openair files
lpofredc Oct 26, 2020
2c42851
Merge remote-tracking branch 'upstream/master' into sensitivity_openair
lpofredc Oct 26, 2020
a8fbf30
Add openair key to api v2
lpofredc Oct 27, 2020
94e60e3
add SENSITIVITY_DEFAULT_ELEVATION config
lpofredc Nov 2, 2020
a3a54f3
Fix is_aerial check
lpofredc Nov 2, 2020
2efbb80
Improve OpenAir transformation
lpofredc Nov 2, 2020
0f5034a
Settings to filter OpenAir sport practices
lpofredc Nov 2, 2020
65d6eed
Clean code
lpofredc Nov 9, 2020
6b49d20
Add Openair descr
lpofredc Nov 9, 2020
a3bbad5
Fix new line missing
lpofredc Nov 10, 2020
8c291b1
Add pyopenair lib to requirements
lpofredc Nov 10, 2020
2f01e8a
feat: optionally off pagination
lpofredc May 25, 2021
03d7532
feat: add some info to OpenAir export
lpofredc May 25, 2021
d41bf98
fix: update pyopenair deps
lpofredc May 25, 2021
189b30e
Merge remote-tracking branch 'upstream/master' into sensitivity_openair
lpofredc May 25, 2021
3e0bb8b
fix: some OpenAir export fixes
lpofredc May 25, 2021
690b6c5
feat: optionnally disable pagination
lpofredc May 25, 2021
1994979
fix: revert to master
lpofredc May 25, 2021
1ba9cb4
fix: restore requirements versions
lpofredc May 25, 2021
49f71cf
fix: fix codestyle E231, E261
lpofredc Aug 26, 2021
dfb978f
Merge branch 'master' of github.com:GeotrekCE/Geotrek-admin into sens…
lpofredc Sep 29, 2021
2173691
fix merge conflicts
lpofredc Jun 9, 2022
07550c0
fix merge
lpofredc Jun 9, 2022
4680e0e
Merge remote-tracking branch 'upstream/master' into sensitivity_openair
lpofredc Jun 15, 2022
5cb5baf
apply .gitignore
lpofredc Jul 11, 2022
8d674fd
Merge tag '2.84.1' into biodivsports
lpofredc Jul 11, 2022
681d1f6
fix tests
lpofredc Dec 7, 2022
154a881
feat: new OpenAir format export for aerial in sensitivity module
lpofredc Dec 7, 2022
36309b3
Merge branch 'master' into biodivsports
submarcos Feb 1, 2023
cc6e79a
update repo and deps
lpofredc Feb 1, 2023
1b06cc9
update repo and deps
lpofredc Feb 1, 2023
de928aa
lint
lpofredc Feb 1, 2023
4382888
update requirements
lpofredc Feb 1, 2023
ffad600
reviews fixes #3349
lpofredc Feb 1, 2023
d9e714f
code cleanup
lpofredc Feb 2, 2023
d4b3c05
update Changelog
lpofredc Feb 2, 2023
6ced9d9
update changelog
lpofredc Feb 6, 2023
4cfa6ea
Merge remote-tracking branch 'upstream/master' into biodivsports
lpofredc Feb 8, 2023
7edaf39
improve sensitivity openair tests
lpofredc Feb 20, 2023
3b837fa
Merge remote-tracking branch 'upstream/master' into biodivsports
lpofredc Mar 6, 2023
b2d007b
Merge branch 'master' into biodivsports
submarcos Apr 5, 2023
5e32d8b
Merge remote-tracking branch 'upstream/master' into biodivsports
lpofredc Apr 5, 2023
451cc95
Merge remote-tracking branch 'upstream/master' into biodivsports
lpofredc May 10, 2023
9f0a251
Merge branch 'master' into biodivsports
submarcos Jun 5, 2023
e86ea7e
Merge tag '2.99.0' into biodivsports
lpofredc Aug 7, 2023
8bd0b48
Merge branch 'biodivsports' of github_lpo:lpofredc/Geotrek-admin into…
lpofredc Aug 7, 2023
36285ca
Merge remote-tracking branch 'upstream/master' into biodivsports
lpofredc Aug 8, 2023
ddaee97
senstivearea wgs84 transform test
lpofredc Aug 31, 2023
1d0b63e
Merge remote-tracking branch 'upstream/master' into biodivsports
lpofredc Aug 31, 2023
b808ff9
senstivearea wgs84 transform test
lpofredc Aug 31, 2023
4d28ec8
Merge remote-tracking branch 'upstream/master' into biodivsports
lpofredc Sep 19, 2023
1383434
test no pagination
lpofredc Sep 19, 2023
229438f
test sensitivity openair buffer on points
lpofredc Sep 19, 2023
104ee6d
WIP tests no pagination
lpofredc Sep 20, 2023
62529f0
improve openair list perfs
lpofredc Sep 20, 2023
70d15bb
test cleanup
lpofredc Sep 20, 2023
f1e07a0
WIP openair view tests
lpofredc Sep 20, 2023
0c4fd9f
test not an aerial area
lpofredc Sep 20, 2023
b53de4d
update changelog and locale files
lpofredc Sep 20, 2023
0dc2735
rm logger message
lpofredc Sep 21, 2023
5004c54
use list comprehension for openair list
lpofredc Sep 21, 2023
7ef11a0
fix missing OpenAir AL value
lpofredc Sep 21, 2023
d06a304
Merge branch 'master' into biodivsports
submarcos Sep 28, 2023
0b60184
Merge branch 'master' into biodivsports
submarcos Sep 28, 2023
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions docs/changelog.rst
Original file line number Diff line number Diff line change
Expand Up @@ -15,6 +15,9 @@ CHANGELOG
- Fix missing update rights for Infrastructure Condition and Infrastructure Type with no structure in Admin Site (#3747)
- Allow to load a signage with the year set to None, raise error if set to NaN (#3611)

**New features**

- Sensitivity: Add ``openair`` export format for aerial sensitive areas (#2372)

2.100.2 (2023-09-12)
------------------------
Expand Down
57 changes: 54 additions & 3 deletions geotrek/api/tests/test_v2.py
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@
Point, Polygon)
from django.contrib.gis.geos.collections import GeometryCollection
from django.db import connection
from django.test.testcases import TestCase
from django.test import TestCase, RequestFactory
from django.test.utils import override_settings
from django.urls import reverse
from django.utils import timezone
Expand All @@ -18,6 +18,7 @@
from rest_framework.test import APITestCase

from geotrek import __version__
from geotrek.api.v2.views.trekking import TrekViewSet
from geotrek.authent import models as authent_models
from geotrek.authent.tests import factories as authent_factory
from geotrek.common import models as common_models
Expand Down Expand Up @@ -55,6 +56,10 @@
'count', 'next', 'previous', 'features', 'type'
])

GEOJSON_COLLECTION_STRUCTURE = sorted([
'features', 'type',
])

GEOJSON_STRUCTURE = sorted([
'geometry',
'type',
Expand Down Expand Up @@ -167,8 +172,9 @@

SENSITIVE_AREA_PROPERTIES_JSON_STRUCTURE = sorted([
'id', 'contact', 'create_datetime', 'description', 'elevation', 'geometry',
'info_url', 'kml_url', 'name', 'period', 'practices', 'provider', 'published', 'species_id',
'structure', 'update_datetime', 'url', 'attachments', 'rules'
'info_url', 'kml_url', 'openair_url', 'name', 'period', 'practices', 'provider',
'published', 'species_id', 'structure', 'update_datetime', 'url', 'attachments',
'rules'
])

SENSITIVE_AREA_SPECIES_PROPERTIES_JSON_STRUCTURE = sorted([
Expand Down Expand Up @@ -818,6 +824,51 @@ def get_hdviewpoint_detail(self, id_hdviewpoint, params=None):
return self.client.get(reverse('apiv2:hdviewpoint-detail', args=(id_hdviewpoint,)), params)


class NoPaginationTestCase(BaseApiTest):
"""
Integration tests for disabled pagination.
"""

def setUp(self):
self.factory = RequestFactory()

def test_json_no_page(self):
request = self.factory.get(
reverse("apiv2:trek-list"),
{
"no_page": "true",
}
)
response = TrekViewSet.as_view({"get": "list"})(request)
self.assertEqual(response.status_code, 200)
self.assertIsInstance(response.data, list)
self.assertIsInstance(response.data[0], dict)
self.assertEqual(
len(response.data[0].get("geometry").get("coordinates")[0]),
3,
)

def test_geojson_no_page(self):
request = self.factory.get(
reverse("apiv2:trek-list"),
{
"no_page": "true",
"format": "geojson",
}
)
response = TrekViewSet.as_view({"get": "list"})(request)
self.assertEqual(response.status_code, 200)
self.assertIsInstance(response.data, dict)
self.assertEqual(sorted(response.data.keys()), GEOJSON_COLLECTION_STRUCTURE)
self.assertEqual(
len(response.data.get("features")), self.nb_treks, response.data
)
self.assertEqual(
sorted(response.data.get("features")[0].get("properties").keys()),
TREK_PROPERTIES_GEOJSON_STRUCTURE,
)


class APIAccessAnonymousTestCase(BaseApiTest):
""" TestCase for anonymous API profile """

Expand Down
5 changes: 5 additions & 0 deletions geotrek/api/v2/pagination.py
Original file line number Diff line number Diff line change
Expand Up @@ -31,3 +31,8 @@ def get_paginated_response(self, data):
]))
else:
return super().get_paginated_response(data)

def paginate_queryset(self, queryset, request, view=None):
if 'no_page' in request.query_params:
return None
return super().paginate_queryset(queryset, request, view)
11 changes: 10 additions & 1 deletion geotrek/api/v2/serializers.py
Original file line number Diff line number Diff line change
Expand Up @@ -954,6 +954,7 @@ class SensitiveAreaSerializer(DynamicFieldsMixin, TimeStampedSerializer):
geometry = geo_serializers.GeometryField(read_only=True, source="geom_transformed", precision=7)
species_id = serializers.SerializerMethodField()
kml_url = serializers.SerializerMethodField()
openair_url = serializers.SerializerMethodField(read_only=True)
rules = RuleSerializer(many=True)
attachments = AttachmentSerializer(many=True)

Expand All @@ -978,11 +979,19 @@ def get_kml_url(self, obj):
url = reverse('sensitivity:sensitivearea_kml_detail', kwargs={'lang': get_language(), 'pk': obj.pk})
return build_url(self, url)

def get_openair_url(self, obj):
is_aerial = obj.species.practices.filter(name__in=settings.SENSITIVITY_OPENAIR_SPORT_PRACTICES).exists()
if is_aerial:
url = reverse('sensitivity:sensitivearea_openair_detail', kwargs={'lang': get_language(), 'pk': obj.pk})
return self.context['request'].build_absolute_uri(url)
else:
return None

class Meta(TimeStampedSerializer.Meta):
model = sensitivity_models.SensitiveArea
fields = TimeStampedSerializer.Meta.fields + (
'id', 'contact', 'description', 'elevation',
'geometry', 'info_url', 'kml_url', 'name', 'period',
'geometry', 'info_url', 'kml_url', 'openair_url', 'name', 'period',
'practices', 'published', 'species_id', 'provider', 'structure',
'url', 'attachments', 'rules'
)
Expand Down
39 changes: 39 additions & 0 deletions geotrek/sensitivity/helpers.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,39 @@
from datetime import date
from calendar import monthrange


def openair_atimes(month: int) -> str:
"""Format month number to OpenAir ATime item

:param month: Months (1-12)
:type month: int
:return: ATimes item
:rtype: str
"""
year = date.today().year
str_lmonth = str(monthrange(year, month)[1]).zfill(2)
str_month = str(month).zfill(2)
tpl = '"{m}": ["UTC(01/{str_month}->{str_lmonth}/{str_month})", "ANY(00:00->23:59)"]'
return tpl.format(m=str(month), str_month=str_month, str_lmonth=str_lmonth)


def openair_atimes_concat(row) -> str:
months_fields = [
row.species.period01,
row.species.period02,
row.species.period03,
row.species.period04,
row.species.period05,
row.species.period06,
row.species.period07,
row.species.period08,
row.species.period09,
row.species.period10,
row.species.period11,
row.species.period12,
]
aatimes_list = []
for i in range(12):
if months_fields[i]:
aatimes_list.append(openair_atimes(i + 1))
return '{{{{{}}}}}'.format(','.join(aatimes_list))
5 changes: 4 additions & 1 deletion geotrek/sensitivity/locale/de/LC_MESSAGES/django.po
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@ msgid ""
msgstr ""
"Project-Id-Version: PACKAGE VERSION\n"
"Report-Msgid-Bugs-To: \n"
"POT-Creation-Date: 2023-02-14 14:36+0100\n"
"POT-Creation-Date: 2023-09-06 15:28+0200\n"
"PO-Revision-Date: YEAR-MO-DA HO:MI+ZONE\n"
"Last-Translator: FULL NAME <EMAIL@ADDRESS>\n"
"Language-Team: LANGUAGE <[email protected]>\n"
Expand Down Expand Up @@ -186,3 +186,6 @@ msgstr ""

msgid "Invalid geometry value."
msgstr ""

msgid "This is not an aerial area"
msgstr ""
5 changes: 4 additions & 1 deletion geotrek/sensitivity/locale/en/LC_MESSAGES/django.po
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@ msgid ""
msgstr ""
"Project-Id-Version: PACKAGE VERSION\n"
"Report-Msgid-Bugs-To: \n"
"POT-Creation-Date: 2023-02-14 14:36+0100\n"
"POT-Creation-Date: 2023-09-06 15:28+0200\n"
"PO-Revision-Date: YEAR-MO-DA HO:MI+ZONE\n"
"Last-Translator: FULL NAME <EMAIL@ADDRESS>\n"
"Language-Team: LANGUAGE <[email protected]>\n"
Expand Down Expand Up @@ -186,3 +186,6 @@ msgstr ""

msgid "Invalid geometry value."
msgstr ""

msgid "This is not an aerial area"
msgstr ""
5 changes: 4 additions & 1 deletion geotrek/sensitivity/locale/es/LC_MESSAGES/django.po
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@ msgid ""
msgstr ""
"Project-Id-Version: PACKAGE VERSION\n"
"Report-Msgid-Bugs-To: \n"
"POT-Creation-Date: 2023-02-14 14:36+0100\n"
"POT-Creation-Date: 2023-09-06 15:28+0200\n"
"PO-Revision-Date: YEAR-MO-DA HO:MI+ZONE\n"
"Last-Translator: FULL NAME <EMAIL@ADDRESS>\n"
"Language-Team: LANGUAGE <[email protected]>\n"
Expand Down Expand Up @@ -186,3 +186,6 @@ msgstr ""

msgid "Invalid geometry value."
msgstr ""

msgid "This is not an aerial area"
msgstr ""
5 changes: 4 additions & 1 deletion geotrek/sensitivity/locale/fr/LC_MESSAGES/django.po
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@ msgid ""
msgstr ""
"Project-Id-Version: PACKAGE VERSION\n"
"Report-Msgid-Bugs-To: \n"
"POT-Creation-Date: 2023-02-14 14:36+0100\n"
"POT-Creation-Date: 2023-09-06 15:28+0200\n"
"PO-Revision-Date: 2020-04-22 07:48+0000\n"
"Last-Translator: Emmanuelle Helly <[email protected]>\n"
"Language-Team: French <https://weblate.makina-corpus.net/projects/geotrek-"
Expand Down Expand Up @@ -187,3 +187,6 @@ msgstr "Ajouter une zone réglementaire"

msgid "Invalid geometry value."
msgstr "Valeur de géométrie invalide"

msgid "This is not an aerial area"
msgstr "Ce n'est pas une zone à enjeux aériens"
5 changes: 4 additions & 1 deletion geotrek/sensitivity/locale/it/LC_MESSAGES/django.po
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@ msgid ""
msgstr ""
"Project-Id-Version: PACKAGE VERSION\n"
"Report-Msgid-Bugs-To: \n"
"POT-Creation-Date: 2023-02-14 14:36+0100\n"
"POT-Creation-Date: 2023-09-06 15:28+0200\n"
"PO-Revision-Date: YEAR-MO-DA HO:MI+ZONE\n"
"Last-Translator: FULL NAME <EMAIL@ADDRESS>\n"
"Language-Team: LANGUAGE <[email protected]>\n"
Expand Down Expand Up @@ -186,3 +186,6 @@ msgstr ""

msgid "Invalid geometry value."
msgstr ""

msgid "This is not an aerial area"
msgstr ""
5 changes: 4 additions & 1 deletion geotrek/sensitivity/locale/nl/LC_MESSAGES/django.po
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@ msgid ""
msgstr ""
"Project-Id-Version: PACKAGE VERSION\n"
"Report-Msgid-Bugs-To: \n"
"POT-Creation-Date: 2023-02-14 14:36+0100\n"
"POT-Creation-Date: 2023-09-06 15:28+0200\n"
"PO-Revision-Date: YEAR-MO-DA HO:MI+ZONE\n"
"Last-Translator: FULL NAME <EMAIL@ADDRESS>\n"
"Language-Team: LANGUAGE <[email protected]>\n"
Expand Down Expand Up @@ -186,3 +186,6 @@ msgstr ""

msgid "Invalid geometry value."
msgstr ""

msgid "This is not an aerial area"
msgstr ""
35 changes: 34 additions & 1 deletion geotrek/sensitivity/models.py
Original file line number Diff line number Diff line change
Expand Up @@ -4,20 +4,25 @@

import datetime
import simplekml

import logging
from django.conf import settings
from django.contrib.gis.db import models
from django.contrib.gis.geos import GEOSGeometry, Polygon
from django.utils.translation import pgettext_lazy, gettext_lazy as _

from mapentity.serializers import plain_text
from pyopenair.factory import wkt2openair

from geotrek.authent.models import StructureRelated
from geotrek.common.mixins.models import (OptionalPictogramMixin, NoDeleteMixin, TimeStampedModelMixin,
AddPropertyMixin, GeotrekMapEntityMixin, get_uuid_duplication)
from geotrek.common.utils import intersecting, classproperty, queryset_or_model
from geotrek.sensitivity.managers import SensitiveAreaManager
from geotrek.sensitivity.helpers import openair_atimes_concat
from geotrek.core.models import simplify_coords

logger = logging.getLogger(__name__)


class Rule(TimeStampedModelMixin, OptionalPictogramMixin):
code = models.CharField(verbose_name=_("Code"), max_length=50, unique=True, blank=True, null=True)
Expand Down Expand Up @@ -218,6 +223,34 @@ def kml(self):
line.style.linestyle.width = 4 # pixels
return kml.kml()

def openair(self):
"""Exports sensitivearea into OpenAir format"""
geom = self.geom
if geom.geom_type == 'Point':
geom = geom.buffer(self.species.radius or settings.SENSITIVITY_DEFAULT_RADIUS, 4)
geom = geom.transform(4326, clone=True)
geom = geom.simplify(0.001, preserve_topology=True)
other = {}
other['*AUID'] = f"GUId=! UId=! Id=(Identifiant-GeoTrek-sentivity) {str(self.pk)}"
adescr = (self.species.name,)
if self.publication_date:
adescr += (f"(published on {self.publication_date.strftime('%d/%m/%Y')})",)
other['*ADescr'] = " ".join(adescr)
other['*ATimes'] = openair_atimes_concat(self)
wkt = geom.wkt
data = {
'wkt': wkt,
'an': self.species.name,
'ac': 'ZSM',
'ah_unit': 'm',
'ah_alti': self.species.radius or settings.SENSITIVITY_DEFAULT_RADIUS,
'ah_mode': 'AGL',
'al_mode': 'SFC',
# 'comment': self.species.name + ' (published on '+self.publication_date.strftime("%d/%m/%Y")+')',
'other': other
}
return wkt2openair(**data)

def is_public(self):
return self.published

Expand Down
9 changes: 8 additions & 1 deletion geotrek/sensitivity/serializers.py
Original file line number Diff line number Diff line change
Expand Up @@ -57,15 +57,22 @@ class Meta(MapentityGeojsonModelSerializer.Meta):
class SensitiveAreaAPISerializer(TranslatedModelSerializer):
species = SpeciesSerializer()
kml_url = rest_serializers.SerializerMethodField()
openair_url = rest_serializers.SerializerMethodField()
attachments = AttachmentSerializer(many=True)
rules = RuleSerializer(many=True)

def get_kml_url(self, obj):
return reverse('sensitivity:sensitivearea_kml_detail', kwargs={'lang': get_language(), 'pk': obj.pk})

def get_openair_url(self, obj):
return reverse('sensitivity:sensitivearea_openair_detail', kwargs={'lang': get_language(), 'pk': obj.pk})

class Meta:
model = sensitivity_models.SensitiveArea
fields = ('id', 'species', 'description', 'contact', 'published', 'publication_date', 'kml_url', 'attachments', 'rules')
fields = (
'id', 'species', 'description', 'contact', 'published', 'publication_date',
'kml_url', 'openair_url', 'attachments', 'rules'
)


class SensitiveAreaAPIGeojsonSerializer(GeoFeatureModelSerializer, SensitiveAreaAPISerializer):
Expand Down
2 changes: 1 addition & 1 deletion geotrek/sensitivity/tests/factories.py
Original file line number Diff line number Diff line change
Expand Up @@ -42,7 +42,7 @@ def practices(obj, create, extracted=None, **kwargs):
if not extracted:
practices = [
SportPracticeFactory.create(name="Practice1"),
SportPracticeFactory.create(name="Practice2")
SportPracticeFactory.create(name="Practice2"),
]
for practice in practices:
obj.practices.add(practice)
Expand Down
3 changes: 3 additions & 0 deletions geotrek/sensitivity/tests/test_functional.py
Original file line number Diff line number Diff line change
Expand Up @@ -29,11 +29,13 @@ def get_expected_geojson_attrs(self):
}

def get_expected_json_attrs(self):

return {
'attachments': [],
'contact': '<a href="mailto:[email protected]">[email protected]</a>',
'description': 'Blabla',
'kml_url': '/api/en/sensitiveareas/{}.kml'.format(self.obj.pk),
'openair_url': '/api/en/sensitiveareas/{}/openair'.format(self.obj.pk),
'publication_date': '2020-03-17',
'published': True,
'rules': [
Expand Down Expand Up @@ -127,6 +129,7 @@ def get_expected_json_attrs(self):
'contact': '<a href="mailto:[email protected]">[email protected]</a>',
'description': 'Blabla',
'kml_url': '/api/en/sensitiveareas/{}.kml'.format(self.obj.pk),
'openair_url': '/api/en/sensitiveareas/{}/openair'.format(self.obj.pk),
'publication_date': '2020-03-17',
'published': True,
'rules': [
Expand Down
Loading