Skip to content

Commit

Permalink
ci: finalize release v1.10.0
Browse files Browse the repository at this point in the history
  • Loading branch information
jobulcke authored Nov 21, 2023
2 parents a54dd45 + d9dca8a commit 717c9fa
Show file tree
Hide file tree
Showing 168 changed files with 1,712 additions and 581 deletions.
5 changes: 5 additions & 0 deletions docs/_ldio/ldio-core/ldio-http-requester.md
Original file line number Diff line number Diff line change
Expand Up @@ -38,3 +38,8 @@ This requester supports the below config:
enabled: true
max-requests-per-minute: 500
```
## Retry
When retries are enabled, the following statuses are always retried, regardless of the configured statuses-to-retry:
- 5xx (500 and above)
- 429
4 changes: 4 additions & 0 deletions docs/_ldio/ldio-inputs/art/ldes-client-connector.svg
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
49 changes: 49 additions & 0 deletions docs/_ldio/ldio-inputs/ldio-ldes-client-connector.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,49 @@
---
layout: default
parent: LDIO Inputs
title: LDES Client
---

# LDIO Ldes Client Connector

***be.vlaanderen.informatievlaanderen.ldes.ldio.LdioLdesClientConnector***

An EDC (Eclipse dataspace Connector) LDIO wrapper component for the [LDI LDES Client building block](../../core/ldi-inputs/ldes-client)

This component adds EDC support to [the ldio ldes client](./ldio-ldes-client.md). If you'd like to know how to configure the LDES Client,
we refer to [the ldio ldes client](./ldio-ldes-client.md).
The additional functionality provided by this component makes it possible to use the Ldes Client to consume an LDES through an EDC connector.
This component exposes two endpoints:

1. http://<host>:<port>/<pipelines.name>/transfer
The Ldio component will start the data transfer with the connector. You have to send the transfer request to
the LdioLdesClientConnector instead of the EDC consumer connector. The LDIO Ldes Client Connector will start the transfer
with the connector and also keep the transfer alive while consuming the LDES (e.g. request a new token when it expires).
3. http://<host>:<port>/<pipelines.name>/token
This endpoint should never be called directly. This is the callback to be provided in the transfer request.
The EDC connector will use this callback endpoint to provide the LDES Client with a token.

![img](./art/ldes-client-connector.svg)

## Config

| Property | Description | Required | Default | Example | Supported values |
|:-----------------------|:----------------------------------------------------------------------------------------------------------------|:---------|:-------------|:----------------------------------------------------------------|:--------------------|
| connector-transfer-url | The transfer url of the EDC connector which has to be called to start a transfer | Yes | N/A | http://consumer-connector:29193/management/v2/transferprocesses | HTTP and HTTPS urls |
| proxy-url-to-replace | Makes it possible to proxy a part of the url of the LDES**. Indicates which part of the url should be replaced. | No | empty string | http://ldes-behind-connectors.dev | string |
| proxy-url-replacement | Makes it possible to proxy a part of the url of the LDES**. Indicates the replacement url part. | No | memory | http://consumer-connector:29193 | string |
** The url mentioned here are the actual url's used by the LDES Server (hostname). These are included in the results bodies to indicate relations, etc. This is a temporary solution until the client and server support relative urls.


## Examples

```yaml
input:
name: be.vlaanderen.informatievlaanderen.ldes.ldio.LdioLdesClientConnector
config:
url: http://consumer-connector:29291/public
connector-transfer-url: http://consumer-connector:29193/management/v2/transferprocesses
proxy-url-to-replace: http://localhost:8081/devices
proxy-url-replacement: http://consumer-connector:29291/public
source-format: application/n-quads
```
Binary file removed docs/assets/images/vlaanderen_header.png
Binary file not shown.
2 changes: 1 addition & 1 deletion ldi-api/pom.xml
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@
<parent>
<groupId>be.vlaanderen.informatievlaanderen.ldes</groupId>
<artifactId>linked-data-interactions</artifactId>
<version>1.9.0</version>
<version>1.10.0-SNAPSHOT</version>
</parent>
<modelVersion>4.0.0</modelVersion>

Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
package be.vlaanderen.informatievlaanderen.ldes.ldi.types;

import org.apache.jena.rdf.model.Model;

import java.util.List;

public interface LdiOneToManyTransformer extends LdiComponent {
List<Model> transform(Model model);
}
Original file line number Diff line number Diff line change
Expand Up @@ -2,14 +2,12 @@

import org.apache.jena.rdf.model.Model;

import java.util.Collection;
import java.util.function.Function;

/**
* The LDI Transformer provides a user to take a linked data model (RDF) from
* the pipeline and perform transformations
* onto it. Afterwards, this model will be put back onto the pipeline towards a
* next transformer or an LDI Output.
*/
public interface LdiTransformer extends LdiComponent, Function<Model, Collection<Model>> {
public interface LdiOneToOneTransformer extends LdiComponent {
Model transform(Model model);
}
2 changes: 1 addition & 1 deletion ldi-core/file-archiving/pom.xml
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@
<parent>
<groupId>be.vlaanderen.informatievlaanderen.ldes.ldi</groupId>
<artifactId>ldi-core</artifactId>
<version>1.9.0</version>
<version>1.10.0-SNAPSHOT</version>
</parent>

<artifactId>file-archiving</artifactId>
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -23,4 +23,4 @@ void extractTimestamp() {
assertTrue(result.isAfter(LocalDateTime.now().minusMinutes(5)));
}

}
}
2 changes: 1 addition & 1 deletion ldi-core/geojson-to-wkt/pom.xml
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@
<parent>
<groupId>be.vlaanderen.informatievlaanderen.ldes.ldi</groupId>
<artifactId>ldi-core</artifactId>
<version>1.9.0</version>
<version>1.10.0-SNAPSHOT</version>
</parent>

<artifactId>geojson-to-wkt</artifactId>
Expand Down
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
package be.vlaanderen.informatievlaanderen.ldes.ldi;

import be.vlaanderen.informatievlaanderen.ldes.ldi.types.LdiTransformer;
import be.vlaanderen.informatievlaanderen.ldes.ldi.types.LdiOneToOneTransformer;
import org.apache.jena.datatypes.RDFDatatype;
import org.apache.jena.datatypes.TypeMapper;
import org.apache.jena.rdf.model.*;
Expand All @@ -11,7 +11,7 @@

import static be.vlaanderen.informatievlaanderen.ldes.ldi.WktConverter.GEOJSON_GEOMETRY;

public class GeoJsonToWktTransformer implements LdiTransformer {
public class GeoJsonToWktTransformer implements LdiOneToOneTransformer {

private final WktConverter wktConverter = new WktConverter();

Expand All @@ -20,7 +20,7 @@ public class GeoJsonToWktTransformer implements LdiTransformer {
* containing geosparql#wktLiteral
*/
@Override
public List<Model> apply(Model model) {
public Model transform(Model model) {
final List<Statement> geometryStatements = model.listStatements(null, GEOJSON_GEOMETRY, (RDFNode) null)
.toList();
geometryStatements.forEach(oldGeometryStatement -> {
Expand All @@ -29,7 +29,7 @@ public List<Model> apply(Model model) {
model.remove(createModelWithChildStatements(model, oldGeometryStatement));
model.add(newGeometryStatement);
});
return List.of(model);
return model;
}

private Statement createNewGeometryStatement(Statement oldStatement, Model geometryModel) {
Expand Down Expand Up @@ -60,5 +60,4 @@ private void addChildStatements(Model model, Resource subject, Set<Statement> st
statements.add(statement);
});
}

}
Loading

0 comments on commit 717c9fa

Please sign in to comment.