Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Bug new rdf-writer in be.vlaanderen.informatievlaanderen.ldes.ldio.LdioHttpOut #380

Closed
kselin opened this issue Oct 25, 2023 · 1 comment · Fixed by #384
Closed

Bug new rdf-writer in be.vlaanderen.informatievlaanderen.ldes.ldio.LdioHttpOut #380

kselin opened this issue Oct 25, 2023 · 1 comment · Fixed by #384
Assignees
Labels
bug Something isn't working component: core Issues related to LDI Building Blocks

Comments

@kselin
Copy link

kselin commented Oct 25, 2023

Describe the bug
Output is defined like below but still get following error:

ldio-workbench  | Exception in thread "pool-4-thread-20" java.lang.NullPointerException: Cannot invoke "String.length()" because "string" is null
ldio-workbench  |       at org.apache.jena.atlas.json.io.JSWriter.outputQuotedString(JSWriter.java:193)
ldio-workbench  |       at org.apache.jena.atlas.json.io.JSWriter.outputQuotedString(JSWriter.java:183)
ldio-workbench  |       at org.apache.jena.atlas.json.io.JsonWriter.visit(JsonWriter.java:221)
ldio-workbench  |       at org.apache.jena.atlas.json.JsonString.visit(JsonString.java:36)
ldio-workbench  |       at org.apache.jena.atlas.json.io.JsonWriter.writeObjectLong(JsonWriter.java:118)
ldio-workbench  |       at org.apache.jena.atlas.json.io.JsonWriter.visit(JsonWriter.java:71)
ldio-workbench  |       at org.apache.jena.atlas.json.JsonObject.visit(JsonObject.java:39)
ldio-workbench  |       at org.apache.jena.atlas.json.JsonValue.output(JsonValue.java:67)
ldio-workbench  |       at org.apache.jena.atlas.json.JsonValue.toString(JsonValue.java:58)
ldio-workbench  |       at be.vlaanderen.informatievlaanderen.ldes.ldi.rdf.formatter.JsonLdWriter.getFramedContext(JsonLdWriter.java:39)
ldio-workbench  |       at be.vlaanderen.informatievlaanderen.ldes.ldi.rdf.formatter.JsonLdWriter.write(JsonLdWriter.java:25)
ldio-workbench  |       at be.vlaanderen.informatievlaanderen.ldes.ldio.LdioHttpOut.accept(LdioHttpOut.java:36)
ldio-workbench  |       at be.vlaanderen.informatievlaanderen.ldes.ldio.LdioHttpOut.accept(LdioHttpOut.java:18)
ldio-workbench  |       at be.vlaanderen.informatievlaanderen.ldes.ldio.services.LdiSender.lambda$accept$0(LdiSender.java:34)
ldio-workbench  |       at java.base/java.util.stream.ForEachOps$ForEachOp$OfRef.accept(ForEachOps.java:183)
ldio-workbench  |       at java.base/java.util.AbstractList$RandomAccessSpliterator.forEachRemaining(AbstractList.java:720)
ldio-workbench  |       at java.base/java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:509)
ldio-workbench  |       at java.base/java.util.stream.ForEachOps$ForEachTask.compute(ForEachOps.java:290)
ldio-workbench  |       at java.base/java.util.concurrent.CountedCompleter.exec(CountedCompleter.java:754)
ldio-workbench  |       at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373)
ldio-workbench  |       at java.base/java.util.concurrent.ForkJoinTask.invoke(ForkJoinTask.java:686)
ldio-workbench  |       at java.base/java.util.stream.ForEachOps$ForEachOp.evaluateParallel(ForEachOps.java:159)
ldio-workbench  |       at java.base/java.util.stream.ForEachOps$ForEachOp$OfRef.evaluateParallel(ForEachOps.java:173)
ldio-workbench  |       at java.base/java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:233)
ldio-workbench  |       at java.base/java.util.stream.ReferencePipeline.forEach(ReferencePipeline.java:596)
ldio-workbench  |       at java.base/java.util.stream.ReferencePipeline$Head.forEach(ReferencePipeline.java:765)
ldio-workbench  |       at be.vlaanderen.informatievlaanderen.ldes.ldio.services.LdiSender.accept(LdiSender.java:34)
ldio-workbench  |       at be.vlaanderen.informatievlaanderen.ldes.ldio.services.LdiSender.accept(LdiSender.java:18)
ldio-workbench  |       at java.base/java.lang.Iterable.forEach(Iterable.java:75)
ldio-workbench  |       at be.vlaanderen.informatievlaanderen.ldes.ldio.services.ComponentExecutorImpl.lambda$transformLinkedData$1(ComponentExecutorImpl.java:32)
ldio-workbench  |       at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
ldio-workbench  |       at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
ldio-workbench  |       at java.base/java.lang.Thread.run(Thread.java:833)

outputs:
        - name: be.vlaanderen.informatievlaanderen.ldes.ldio.LdioHttpOut
          config:
            endpoint: https://webhook.site/2b3799e0-95a8-4731-af0b-8751598991a1
            # content-type: application/ld+json
            rdf-writer:
              content-type: application/ld+json
              frame-type: http://def.isotc211.org/iso19156/2011/Observation#OM_Observation

Expected behavior
Expect to see framed json-ld output.

Full config

orchestrator:
  pipelines:
    - name: observations
      description: "This pipeline will take data from crowdscan ldes and send to kafka"
      input:
        name: be.vlaanderen.informatievlaanderen.ldes.ldi.client.LdioLdesClient
        config:
          url: https://test.crowdscan.be/ldes-beacon/observations/by-page-1w?pageNumber=1/2
          sourceFormat: text/turtle
          retries:
            enabled: true
      transformers:
        - name: be.vlaanderen.informatievlaanderen.ldes.ldi.SparqlConstructTransformer
          config:
            query: |
              PREFIX terms: <http://purl.org/dc/terms/>
              PREFIX ngsi-ld: <https://uri.etsi.org/ngsi-ld/>
              PREFIX OM: <http://def.isotc211.org/iso19156/2011/Observation#>
              PREFIX time: <http://www.w3.org/2006/time#>
              PREFIX sosa: <http://www.w3.org/ns/sosa/> 
              PREFIX seb: <https://data.vlaanderen.be/ns/sensoren-en-bemonstering#>
              PREFIX schema: <http://schema.org/>
              PREFIX prov: <http://www.w3.org/ns/prov#>
              CONSTRUCT {
                ?obs a OM:OM_Observation ;
                OM:OM_Observation.featureOfInterest [
                  a ngsi-ld:Relationship;
                  ngsi-ld:hasObject ?featureOfInterest
                ] ;
                OM:OM_Observation.observedProperty  [
                  a ngsi-ld:Property;
                  ngsi-ld:hasValue ?observedProperty
                ] ;
                OM:OM_Observation.phenomenonTime  [
                  a ngsi-ld:Property ;
                  ngsi-ld:hasValue [
                    a time:Interval ;
                    time:hasEnd ?endTime ;
                  time:hasXSDDuration ?duration
                  ]
                ] ;
                OM:OM_Observation.procedure  [
                  a ngsi-ld:Property;
                  ngsi-ld:hasValue [
                    a sosa:Procedure ;
                    seb:Observatieprocedure.specificatie ?procedure
                  ]
                ] ;
                OM:OM_Observation.value  [
                    a ngsi-ld:Property;
                    ngsi-ld:hasValue [
                      a schema:QuantitativeValue ;
                      schema:value ?value
                  ]
                ] ;
                OM:OM_Observation.resultTime  [
                  a ngsi-ld:Property;
                  ngsi-ld:hasValue ?resultTime
                ] ;
                prov:generatedAtTime [
                  a ngsi-ld:Property;
                  ngsi-ld:hasValue ?generatedAtTime
                ] 
              }
              WHERE {
                ?versieObs a OM:OM_Observation ;
                terms:isVersionOf ?obs ; 
                OM:OM_Observation.featureOfInterest ?featureOfInterest ;
                OM:OM_Observation.observedProperty ?observedProperty ;
                OM:OM_Observation.phenomenonTime/time:hasEnd/time:inXSDDateTimeStamp ?endTime ;
                OM:OM_Observation.phenomenonTime/time:hasXSDDuration ?duration ;
                OM:OM_Observation.procedure/seb:Observatieprocedure.specificatie ?procedure ;
                OM:OM_Observation.result/schema:value ?value ;
                OM:OM_Observation.resultTime ?resultTime ;
                prov:generatedAtTime ?generatedAtTime .
              }
      outputs:
        - name: be.vlaanderen.informatievlaanderen.ldes.ldio.LdioHttpOut
          config:
            endpoint: https://webhook.site/2b3799e0-95a8-4731-af0b-8751598991a1
            # content-type: application/ld+json
            rdf-writer:
              content-type: application/ld+json
              frame-type: http://def.isotc211.org/iso19156/2011/Observation#OM_Observation

Docker-compose file

services:
  ldio-workbench:
    container_name: ldio-workbench
    image: ldio-framing (image from latest version develop branch)
    ports:
      - 8083:8083
    volumes:
      - ./ldio-config.yml:/ldio/application.yml:ro
@kselin kselin added the needs triage Issue needs to be evaluated by team label Oct 25, 2023
@github-project-automation github-project-automation bot moved this to 📋 Backlog in VSDS Backlog Oct 25, 2023
@Yalz Yalz self-assigned this Oct 27, 2023
@Yalz Yalz added bug Something isn't working component: core Issues related to LDI Building Blocks and removed needs triage Issue needs to be evaluated by team labels Oct 27, 2023
@Yalz Yalz closed this as completed Oct 27, 2023
@github-project-automation github-project-automation bot moved this from 📋 Backlog to 👀 In review in VSDS Backlog Oct 27, 2023
@Yalz Yalz reopened this Oct 27, 2023
@github-project-automation github-project-automation bot moved this from 👀 In review to 📋 Backlog in VSDS Backlog Oct 27, 2023
@Yalz Yalz linked a pull request Oct 27, 2023 that will close this issue
@Yalz
Copy link
Collaborator

Yalz commented Nov 9, 2023

This issue is now fixed as part of the latest LDIO image

@Yalz Yalz closed this as completed Nov 9, 2023
@github-project-automation github-project-automation bot moved this from 📋 Backlog to 👀 In review in VSDS Backlog Nov 9, 2023
@Yalz Yalz moved this from 👀 In review to ✅ Done in VSDS Backlog Nov 9, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working component: core Issues related to LDI Building Blocks
Projects
Archived in project
Development

Successfully merging a pull request may close this issue.

2 participants