Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

VODataservice support #5

Closed
wants to merge 11 commits into from
26 changes: 7 additions & 19 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,27 +1,24 @@
# vo-models

`vo-models` an open-source project to provide Python models and OpenAPI specifications for [IVOA](https://www.ivoa.net/) service protocols.
`vo-models` an open-source project to provide Python models for [IVOA](https://www.ivoa.net/) service protocols.

The project is designed to be used by IVOA members, service implementors, and developers to help facilitate the development of IVOA-compliant services and clients.

## Features

- **Pydantic-xml Models:** The project includes Python models for IVOA protocols, using [pydantic-xml](https://github.com/dapper91/pydantic-xml). Based on [Pydantic](https://docs.pydantic.dev/latest/), these models describe transactions for an IVOA protocol, such as UWS, and feature automatic validation, parsing and serialization of XML data for use with Python clients and web frameworks.

- **OpenAPI Specifications:** The project includes OpenAPI definitions for IVOA protocols. The use of OpenAPI provides a standardized and machine-readable way to describe the IVOA protocols. OpenAPI specifications offer benefits such as automatic documentation generation, and automatic client and server code generation.

- **Expandability:** The project is designed with future expansion in mind. Plans include extending the schema and models to cover other IVOA standards.

## Protocols

The following IVOA protocols are currently supported / under development:
The following IVOA protocols are currently supported:

- **UWS (Universal Worker Service) version 1.1**
- **VOSI (IVOA Support Interfaces) version 1.1**
- VOSI Availability
- **VODataService version 1.2**

- **UWS (Universal Worker Service) version 1.1:**
- Active development:
- OpenAPI Models
- Pydantic-XML Models
- Planned:
- OpenAPI Service Definition

## Installation

Expand Down Expand Up @@ -50,15 +47,6 @@ For active development, install the project in development mode:
pip install -e .[dev,test]
```

## Usage
### OpenAPI Schema

OpenAPI schema files representing IVOA protocol transactions can be found in the `vo/models/openapi` directory.

For each protocol, two files are provided: a `components.yml` file containing the JSON/XML schema definitions for request / response transactions, and a file named after the protocol (e.g. `uws.yml`) containing the OpenAPI specification for the protocol. The schema models, and the OpenAPI specification, are viewable in the [Swagger Editor](https://editor.swagger.io/).

*Note: Currently, the OpenAPI API definition files are not guaranteed to be complete. They are provided as a starting point for future development, and an example of how the schema definitions can be used.*

### Pydantic-XML Models

Python models using [pydantic-xml](https://github.com/dapper91/pydantic-xml), a library based on [Pydantic](https://docs.pydantic.dev/latest/), are provided in the `vo/models/xml` directory.
Expand Down
4 changes: 3 additions & 1 deletion pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ build-backend = "setuptools.build_meta"

[project]
name = "vo-models"
version = "0.1.2"
version = "0.2.0"
authors = [
{name = "Joshua Fraustro", email="[email protected]"},
{name = "MAST Archive Developers", email="[email protected]"}
Expand Down Expand Up @@ -40,4 +40,6 @@ dev = ["pylint"]
Homepage = "https://github.com/spacetelescope/vo-models"
Issues = "https://github.com/spacetelescope/vo-models/issues"

[tool.setuptools]
packages = ["vo_models"]

51 changes: 37 additions & 14 deletions tests/uws/uws_models_test.py
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
"""Tests for the XML serialization of UWS elements"""
"""Tests for UWS pydantic-xml models"""

from datetime import timezone as tz
from typing import Optional
Expand All @@ -8,7 +8,7 @@
from lxml import etree
from pydantic_xml import element

from vo_models.xml.uws import (
from vo_models.uws import (
ErrorSummary,
Jobs,
JobSummary,
Expand All @@ -18,8 +18,8 @@
Results,
ShortJobDescription,
)
from vo_models.xml.uws.types import ExecutionPhase
from vo_models.xml.voresource.types import UTCTimestamp
from vo_models.uws.types import ExecutionPhase
from vo_models.voresource.types import UTCTimestamp

UWS_NAMESPACE_HEADER = """xmlns:uws="http://www.ivoa.net/xml/UWS/v1.0"
xmlns:xlink="http://www.w3.org/1999/xlink"
Expand Down Expand Up @@ -55,14 +55,17 @@ def test_read_from_xml(self):
def test_write_to_xml(self):
"""Test writing to XML"""

error_summary = ErrorSummary(type="transient", has_detail=True, message="Invalid query.")
error_summary = ErrorSummary(
type="transient", has_detail=True, message="Invalid query."
)
error_summary_xml = error_summary.to_xml(encoding=str)

self.assertEqual(
canonicalize(self.test_error_summary_xml, strip_text=True),
canonicalize(error_summary_xml, strip_text=True),
)


class TestParameterType(TestCase):
"""Tests for the UWS Parameter complex type"""

Expand All @@ -84,7 +87,9 @@ def test_read_from_xml(self):
def test_write_to_xml(self):
"""Test writing to XML"""

parameter = Parameter(by_reference=False, id="param1", is_post=False, value="test_value")
parameter = Parameter(
by_reference=False, id="param1", is_post=False, value="test_value"
)
parameter_xml = parameter.to_xml(encoding=str)

self.assertEqual(
Expand Down Expand Up @@ -209,14 +214,19 @@ class TestShortJobDescriptionType(TestCase):
def test_read_from_xml(self):
"""Test reading from XML"""

short_job_description = ShortJobDescription.from_xml(self.test_short_job_description_xml)
short_job_description = ShortJobDescription.from_xml(
self.test_short_job_description_xml
)
self.assertEqual(short_job_description.job_id, "id1")
self.assertEqual(short_job_description.type, "simple")
self.assertEqual(short_job_description.href, "http://uri1")
self.assertEqual(short_job_description.phase, "PENDING")
self.assertEqual(short_job_description.run_id, "runId1")
self.assertEqual(short_job_description.owner_id, None)
self.assertEqual(short_job_description.creation_time, UTCTimestamp(1900, 1, 1, 1, 1, 1, tzinfo=tz.utc))
self.assertEqual(
short_job_description.creation_time,
UTCTimestamp(1900, 1, 1, 1, 1, 1, tzinfo=tz.utc),
)

def test_write_to_xml(self):
"""Test writing to XML"""
Expand Down Expand Up @@ -294,7 +304,9 @@ def test_validate(self):
param2=Parameter(id="param2", value="value2"),
param3=Parameter(id="param3", value="value3"),
)
parameters_xml = etree.fromstring(parameters.to_xml(skip_empty=True, encoding=str))
parameters_xml = etree.fromstring(
parameters.to_xml(skip_empty=True, encoding=str)
)
uws_schema.assertValid(parameters_xml)


Expand Down Expand Up @@ -337,9 +349,15 @@ def test_read_from_xml(self):
self.assertEqual(job_summary.owner_id, "ownerId1")
self.assertEqual(job_summary.phase, ExecutionPhase.PENDING.value)
self.assertEqual(job_summary.quote, None)
self.assertEqual(job_summary.creation_time, UTCTimestamp(1900, 1, 1, 1, 1, 1, tzinfo=tz.utc))
self.assertEqual(job_summary.start_time, UTCTimestamp(1900, 1, 1, 1, 1, 1, tzinfo=tz.utc))
self.assertEqual(job_summary.end_time, UTCTimestamp(1900, 1, 1, 1, 1, 1, tzinfo=tz.utc))
self.assertEqual(
job_summary.creation_time, UTCTimestamp(1900, 1, 1, 1, 1, 1, tzinfo=tz.utc)
)
self.assertEqual(
job_summary.start_time, UTCTimestamp(1900, 1, 1, 1, 1, 1, tzinfo=tz.utc)
)
self.assertEqual(
job_summary.end_time, UTCTimestamp(1900, 1, 1, 1, 1, 1, tzinfo=tz.utc)
)
self.assertEqual(job_summary.execution_duration, 0)
self.assertEqual(job_summary.destruction, UTCTimestamp(1900, 1, 1, 1, 1, 1, tzinfo=tz.utc))
self.assertEqual(len(job_summary.parameters.dict()), 2)
Expand Down Expand Up @@ -427,7 +445,10 @@ def test_read_from_xml(self):
self.assertEqual(jobs_element.jobref[0].phase, ExecutionPhase.PENDING)
self.assertEqual(jobs_element.jobref[0].run_id, None)
self.assertEqual(jobs_element.jobref[0].owner_id, None)
self.assertEqual(jobs_element.jobref[0].creation_time, UTCTimestamp(1900, 1, 1, 1, 1, 1, tzinfo=tz.utc))
self.assertEqual(
jobs_element.jobref[0].creation_time,
UTCTimestamp(1900, 1, 1, 1, 1, 1, tzinfo=tz.utc),
)

def test_write_to_xml(self):
"""Test writing to XML"""
Expand Down Expand Up @@ -462,5 +483,7 @@ def test_validate(self):
)
]
)
jobs_element_xml = etree.fromstring(jobs_element.to_xml(skip_empty=True, encoding=str))
jobs_element_xml = etree.fromstring(
jobs_element.to_xml(skip_empty=True, encoding=str)
)
uws_schema.assertValid(jobs_element_xml)
Loading