Skip to content

Commit

Permalink
docs: updated README
Browse files Browse the repository at this point in the history
  • Loading branch information
jonrandahl committed Dec 9, 2024
1 parent a2e7990 commit c17978c
Showing 1 changed file with 49 additions and 38 deletions.
87 changes: 49 additions & 38 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,41 +1,42 @@
# Epimorphics Data Services API gem

This gem provides a Ruby API for back-end data services used in the HMLR
linked data applications. Specifically, it allows a simple expression language
to be used to specify queries into an [RDF data cube](https://www.w3.org/TR/vocab-data-cube/),
in which a collection of data readings, known as _measures_ are organised into a
hyper-cube of two or more _dimensions_.
This gem provides a Ruby API for back-end data services used in the HMLR linked
data applications. Specifically, it allows a simple expression language to be
used to specify queries into an [RDF data
cube](https://www.w3.org/TR/vocab-data-cube/), in which a collection of data
readings, known as _measures_ are organised into a hyper-cube of two or more
_dimensions_.

## History

Originally, the expression language used by this gem was interpreted directly by
the [DsAPI](https://github.com/epimorphics/data-API/wiki). DsAPI presented a RESTful
API in which data expressions could be translated systematically into SPARQL
expressions, executed against a remote SPARQL endpoint, and then results
the [DsAPI](https://github.com/epimorphics/data-API/wiki). DsAPI presented a
RESTful API in which data expressions could be translated systematically into
SPARQL expressions, executed against a remote SPARQL endpoint, and then results
returned to the caller in a compact JSON format.

In 2021, we took the decision to retire the DsAPI codebase, which has not been
actively maintained for some time. In its place, we now expect to use
[Sapi-NT](https://github.com/epimorphics/sapi-nt). Sapi-NT performs a similar
function, in that it provides a RESTful API in which compact queries are translated
into SPARQL expressions, and the results are available in (amongst other formats)
JSON encoding. However, the input to Sapi-NT, in which we articulate the projection
of the underlying hypercube that we require is encoded as URL parameters in an
HTTP GET request. DsAPI, in contrast, expects the input query to be POSTed as a
JSON expression.

To minimise changes to the client applications in which this gem is used, we have
implemented a shim layer that accepts DsAPI expressions and re-codes them as Sapi-NT
URLs. Similarly, differences in the returned JSON results formats are also ironed
out by this shim layer. It is possible to do this because, once the designs of the
applications had settled, the HMLR apps only use a subset of the expressive power
of the DsAPI expression language.
function, in that it provides a RESTful API in which compact queries are
translated into SPARQL expressions, and the results are available in (amongst
other formats) JSON encoding. However, the input to Sapi-NT, in which we
articulate the projection of the underlying hypercube that we require is encoded
as URL parameters in an HTTP GET request. DsAPI, in contrast, expects the input
query to be POSTed as a JSON expression.

To minimise changes to the client applications in which this gem is used, we
have implemented a shim layer that accepts DsAPI expressions and re-codes them
as Sapi-NT URLs. Similarly, differences in the returned JSON results formats are
also ironed out by this shim layer. It is possible to do this because, once the
designs of the applications had settled, the HMLR apps only use a subset of the
expressive power of the DsAPI expression language.

It would be possible to simplify this code further, at the expense of needing to
make changes to the calling application code. At the time, we did not believe this
to be a cost-effective change, and no benefit to end-users (the internals of the
query language are not exposed to end-users). This calculation may be different in
future.
make changes to the calling application code. At the time, we did not believe
this to be a cost-effective change, and no benefit to end-users (the internals
of the query language are not exposed to end-users). This calculation may be
different in future.

---

Expand Down Expand Up @@ -71,10 +72,13 @@ Inspecting 21 files

### Tests

You will need to have started the [HMLR Data API](https://github.com/epimorphics/lr-data-api)
locally. To do so follow the instructions in the repository's [README](https://github.com/epimorphics/lr-data-api#run)
You will need to have started the [HMLR Data
API](https://github.com/epimorphics/lr-data-api) locally. To do so follow the
instructions in the repository's
[README](https://github.com/epimorphics/lr-data-api#run)

Once the API is started you can invoke the tests with the simple command below[^1]:
Once the API is started you can invoke the tests with the simple command
below[^1]:

```sh
rake test
Expand All @@ -87,32 +91,38 @@ instance of the HMLR Data API from a non-default port:
API_URL=http://localhost:8080 rake test
```

_N.B If `API_URL` environment variable is not set it will default to `http://localhost:8888`_
_N.B If `API_URL` environment variable is not set it will default to
`http://localhost:8888`_

---

### Publishing the gem to the GitHub Package Registry
### Publishing the gem to the Epimorphics GitHub Package Registry

This gem is now published to the Epimorphics section of the GitHub Package
Registry (GPR). Previously we linked directly to the GitHub repo in the
`Gemfile`s of applications consuming this library, but this practice is now
anti-preferred.

Note that in order to publish to the Epimorphics section of the GPR, you'll
need a GitHub personal access token (PAT). There are [instructions on the Epimorphics
Note that in order to publish to the Epimorphics section of the GPR, you'll need
a GitHub personal access token (PAT). There are [instructions on the Epimorphics
wiki](https://github.com/epimorphics/internal/wiki/Ansible-CICD#creating-a-pat-for-gpr-access)
for creating a new PAT if you don't have one. Once created, you can use the
same PAT in multiple projects, you don't need to create a new one each time.
for creating a new PAT if you don't have one. Once created, you can use the same
PAT in multiple projects, you don't need to create a new one each time.

At present, publishing is a manual step for Gem maintainers. The process is:

1. Make the required code changes, and have them reviewed by other members of
the team
2. Update `CHANGELOG.md` with the changes. Update
`lib/data_services_api/version.rb` following semantic version principles
3. Check that the gem builds correctly: `make gem`
4. Publish the new gem to GPR: `make publish`
5. Check on the [GitHub Package
3. Check that the gem builds correctly via the `make gem` target
- This will run the tests and build the gem locally; however, the local gem
will be ignored by the `.gitignore` file and not included in the recorded
code changes in the repository.
4. Push the changes to the `main` branch via a pull request
5. On PR merge, create a new release in GitHub by triggering the `Publish`
GitHub Action workflow manually.
6. Check on the [GitHub Package
Registry](https://github.com/orgs/epimorphics/packages?repo_name=data_services_api)
to see that the new gem has been published.

Expand All @@ -122,7 +132,8 @@ This gem integrates with Prometheus monitoring by emitting the following
`ActiveSupport::Notificaion`s:

- `response.api` - API response, including status code and duration
- `connection_failure.api` - failure to connect to the API, with exception detail
- `connection_failure.api` - failure to connect to the API, with exception
detail
- `service_exception.api` - failure to process the API response

[^1]: You may need to preface the `rake test` command with `bundle exec` if you
Expand Down

0 comments on commit c17978c

Please sign in to comment.