Merge branch 'refs/heads/master' into 20-update-vo-dml-standard-document

.github/workflows/test.yml

@@ -26,23 +26,30 @@ jobs:
           java-version: '11'
           distribution: 'temurin'
       - name: Build java runtime with Gradle
-        uses: gradle/[email protected]
+        uses: gradle/actions/setup-gradle@v3
         with:
           cache-disabled: false
           arguments: |
             :java:publishToMavenLocal  
             -PskipSigning=True
       - name: Build ivoa model with Gradle
-        uses: gradle/[email protected]
+        uses:  gradle/actions/setup-gradle@v3
         with:
           cache-disabled: false
           arguments: |
             :ivoa:publishToMavenLocal  
             -PskipSigning=True
       - name: Test sample model with Gradle
-        uses: gradle/[email protected]
+        uses: gradle/actions/setup-gradle@v3
         with:
           cache-disabled: false
           arguments: |
-            :gradletooling:vodml-sample:test
+            :sample:test
+            -PskipSigning=True
+      - name: Validate sample model with Gradle
+        uses: gradle/actions/setup-gradle@v3
+        with:
+          cache-disabled: false
+          arguments: |
+            :sample:vodmlValidate
             -PskipSigning=True

README.md

@@ -7,6 +7,8 @@ language.
 
 There is a [Guide to creating models with VO-DML tooling](https://ivoa.github.io/vo-dml/)
 
+[![PDF-Preview](https://img.shields.io/badge/VODML_WD_1.1-PDF-blue)](../../releases/download/WD-1.1/VO-DML.pdf)
+
 The various subdirectories
 
 * [doc](./doc)      : The IVOA standard document source and the vodml tools guide.
@@ -21,5 +23,6 @@ The various subdirectories
 ![main test](https://github.com/ivoa/vo-dml/actions/workflows/test.yml/badge.svg)
 ![site build](https://github.com/ivoa/vo-dml/actions/workflows/site.yml/badge.svg)
 [![gradle plugin](https://img.shields.io/gradle-plugin-portal/v/net.ivoa.vo-dml.vodmltools?label=gradle%20plugin)](https://plugins.gradle.org/plugin/net.ivoa.vo-dml.vodmltools)
+[![Standard Document PDF Preview generation](https://github.com/ivoa/vo-dml/actions/workflows/std.yml/badge.svg)](https://github.com/ivoa/vo-dml/actions/workflows/std.yml)
 
 _note that this project was moved from https://volute.g-vo.org/svn/trunk/projects/dm/vo-dml/_

build.gradle.kts

@@ -1,10 +1,10 @@
 //IMPL cannot get this composite build to work - have resorted to sequence of commands in github CI
 
-tasks.register("doAll"){
-    dependsOn(gradle.includedBuild("ivoa").task(":jar"))
-    finalizedBy(gradle.includedBuild("gradletooling").task(":vodml-sample:test"))
-    description = "builds and installs the runtime library and then runs unit tests on code generated from sample model"
-}
+//tasks.register("doAll"){
+//    dependsOn(gradle.includedBuild("ivoa").task(":jar"))
+//    finalizedBy(gradle.includedBuild("gradletooling").task(":vodml-sample:test"))
+//    description = "builds and installs the runtime library and then runs unit tests on code generated from sample model"
+//}
 
 // would like rto do something like this
 //tasks.register<GradleBuild>("doAll"){

doc/guide/JavaCodeGeneration.md

@@ -98,7 +98,7 @@ At this point the overall model object is suitable to be serialized.
 On reading in a model instance, it is possible to extract any top level ObjectTypes
 with the `public <T> List<T> getContent(Class<T> c)` method.
 
-!!! Note "contained references"
+!!! note "contained references"
 
     There is some provisional support for "contained references" when cloning an object - the API for this 
     is subject to change, but an example is used in [copyTest](https://github.com/ivoa/vo-dml/blob/519f213830c0a946a0d63be65552067bfc707f1d/tools/gradletooling/sample/src/test/java/org/ivoa/dm/lifecycle/LifeCycleDetailedTest.java#L99C10-L99C18) using the `createContext()` and `updateClonedReferences()` methods either side of an object clone with a copy constructor.

doc/guide/QuickStart.md

@@ -10,14 +10,12 @@ If starting a completely new data model then the [Template DM Project](https://g
 If adapting an existing data model repository then
 
 1. Edit a `build.gradle.kts` file with reference to the plugin (note substitute ![latest published version](https://img.shields.io/gradle-plugin-portal/v/net.ivoa.vo-dml.vodmltools?label=latest%20published%20version) below)
-
 ```kotlin
 plugins {
     id("net.ivoa.vo-dml.vodmltools") version "0.x.x"
 }
 ```
 2. create a `settings.gradle.kts` - it is possible just to copy the [template version](https://github.com/ivoa/DataModelTemplate/blob/master/settings.gradle.kts) and just edit the `rootProject.name`.
-
 3. create the binding file for the model (see [below](#binding-files)) for more detail 
 
 There is nothing else that needs to be done if the VO-DML files in the default place 
@@ -32,10 +30,8 @@ gradle vodmlValidate
 ```
 will attempt to validate the model and print any errors.
 
-* vodmlValidate - runs validation on the models.
-*
-* vodmlJavaGenerate - generate java classes. See [generated code guide](JavaCodeGeneration.md) for details of how to use the generated java code to serialize instances to XML and RDB.
- Note that it is also possible to test models using the generated code.
+If the validation is successful you can produce [various derived products](Transformers.md). Developing your VO-DML model further is discussed [here](modelling/modellingIntro.md). 
+
 
 ## Detailed configuration
 
@@ -47,6 +43,8 @@ vodml {
 ```
 section in the `build.gradle.kts` file.
 
+The various sub-properties that can be set are
+
 * _vodmlDir_ - the default is `src/main/vo-dml`
   ```kotlin
   vodmlDir.set(file("vo-dml"))
@@ -59,6 +57,7 @@ section in the `build.gradle.kts` file.
 * _outputSiteDir_ - where the [mkdocs](https://www.mkdocs.org) suitable model description is created by the `gradle vodmlSite` command - default `build/generated/docs/vodml-site`.
 * _outputJavaDir_ - where the generated Java is created - the default is `build/generated/sources/vodml/java/` and it should not 
   be necessary to ever alter this as gradle will integrate this automatically into the various source paths.
+* _outputSchemaDir_ - where the XML and JSON schema are generated to - the default is `build/generated/sources/vodml/schema/` - this is automatically included in the classpath and the output jar.
 * _catalogFile_ - in general it is not necessary to set this, as the plugin will create a catalogue file automatically from the vodmlDir and vodmlFiles properties (as well as including files in any dependencies that also contain VO-DML models)
   A catalogue file is necessary as the rest of the tooling is designed to use only the filename (no path) for inclusions and references.
   If it is desired to create a file manually for a special purpose, then the file should have the format as below - it should be noted that all references to model files will have to be specified if this is done.

doc/guide/Serialization.md

@@ -30,7 +30,7 @@ For the [small example model](https://github.com/ivoa/vo-dml/tree/master/models/
 ```xml
 <ser:myModelModel xmlns:ser="http://ivoa.net/vodml/sample/serialization" >
     <refs>
-        <refa id="id_0">
+        <refa _id="id_0">
             <val>a value</val>
         </refa>
         <refb>
@@ -45,6 +45,8 @@ For the [small example model](https://github.com/ivoa/vo-dml/tree/master/models/
     </SomeContent>
 </ser:myModelModel>
 ```
+Note that because the integer parts of the id might have come from database indices, which 
+typically are only unique per table, then the string part of the id is made up from the part of the vodml-id of the type.
 
 ## JSON
 

doc/guide/Transformers.md

@@ -26,6 +26,13 @@ The models are also transformed into schema that describe the various serializat
 VO-DML tooling is to be able to exchange instances of the models between different computer languages, with 
 all the source code and schema automatically generated.
 
-The gradle plugin does not currently have a task directly to generate XML and RDB schema from the models, however, this can be done
-indirectly from the generated Java code as can be seen from the [Small java example](https://github.com/ivoa/vo-dml/tree/master/gradletooling/sample/src/main/java/WriteSampleSchema.java).
+The gradle task
 
+```shell
+gradle vodmlSchema
+```
+will generate XML and JSON schema for the model. These schema files will automatically be included withing the jar file for the model.
+
+# Transformation to VO-DML
+
+The transformation of other data model representations to VO-DML is discussed elsewhere. 

doc/guide/modelling/TransformingToVODML.md

@@ -0,0 +1,8 @@
+Creating VO-DML from other Data Models
+======================================
+
+It is possible to start with an XML schema that can be transferred to VODSL with 
+
+```shell
+gradle vodmlXsdToVodsl --xsd mymodel.xsd --dsl mymodel.vodsl
+```

doc/guide/modelling/UML.md

@@ -3,8 +3,8 @@ UML
 
 ## Configuring the UML tool
 
-In general it is necessary to configure a UML tool with the IVOA "profile" to restrict the metamodel to that
-used by VO-DML.
+In general it is necessary to configure a UML tool with the IVOA "profile" to restrict
+the metamodel to that used by VO-DML.
 
 TBC
 

doc/guide/modelling/designIntro.md

@@ -1,4 +1,11 @@
 Designing Models
 ================
 
-## Testing serialization.
+Creating "good" data model designs is a art rather than a science. Within the context of the IVOA reusing existing recommended models is a prerequisite for a model be considered good.
+
+TBC
+
+## Testing serialization.
+
+The java runtime has some functionality for [roundtrip testing](../JavaCodeGeneration.md#testing-models) the various serializations 
+which can be a good first level test as to whether your model is a "good design".

doc/guide/modelling/modellingIntro.md

@@ -13,3 +13,29 @@ by editing in a text editor (and the [model schema](https://github.com/ivoa/vo-d
 * [VODSL](VODSL.md)
 * [UML](UML.md)
 
+
+It is possible to import existing data models by;
+
+1. Adding the published jar as a dependency in the `build.gradle.kts` file, e.g.
+```kotlin
+dependencies {
+    implementation("org.javastro.ivoa.vo-dml:ivoa-base:1.0")
+}
+```
+the tooling then makes sure that the model files are available for local import.
+1.  Importing into the model file, and there are two cases whether VO-DML or VODSL is considered the 'working source'
+
+    1. VO-DML - the tooling only requires the file name (it automatically creates an XML catalog which can resolve to  the absolute path)
+```xml
+<import>
+    <name>ivoa</name>
+    <url>IVOA-v1.0.vo-dml.xml</url>
+</import>
+
+```
+
+    1. VODSL - a relative path import should be used - the tooling automatically creates a vodsl file to import into the `build/tmp` directory so that the
+   exact form of the relative path depends on where the including source file is located.
+```
+include "../../build/tmp/IVOA-v1.0.vodsl"
+```

doc/mkdocs.yml

@@ -13,7 +13,7 @@ nav:
   - Getting Started:
       - Installation: Installation.md
       - Configuration: QuickStart.md
-  - Transformers:
+  - Derived Products:
       - Introduction: Transformers.md
       - Documentation: Documentation.md
       - Java Generation: JavaCodeGeneration.md
@@ -24,10 +24,13 @@ nav:
       - VODSL : modelling/VODSL.md
       - Eclipse Editor: modelling/EclipseVODSL.md
       - UML : modelling/UML.md
+      - Generation from other sources : modelling/TransformingToVODML.md
   - Model Design:
       - Introduction : modelling/designIntro.md
 
 markdown_extensions:
+  - admonition
+  - pymdownx.details
   - pymdownx.superfences:
       custom_fences:
         - name: mermaid

gradle/wrapper/gradle-wrapper.properties

@@ -1,6 +1,6 @@
 distributionBase=GRADLE_USER_HOME
 distributionPath=wrapper/dists
-distributionUrl=https\://services.gradle.org/distributions/gradle-8.2.1-bin.zip
+distributionUrl=https\://services.gradle.org/distributions/gradle-8.7-bin.zip
 networkTimeout=10000
 validateDistributionUrl=true
 zipStoreBase=GRADLE_USER_HOME

gradlew

@@ -83,7 +83,8 @@ done
 # This is normally unused
 # shellcheck disable=SC2034
 APP_BASE_NAME=${0##*/}
-APP_HOME=$( cd "${APP_HOME:-./}" && pwd -P ) || exit
+# Discard cd standard output in case $CDPATH is set (https://github.com/gradle/gradle/issues/25036)
+APP_HOME=$( cd "${APP_HOME:-./}" > /dev/null && pwd -P ) || exit
 
 # Use the maximum available, or set MAX_FD != -1 to use that value.
 MAX_FD=maximum
@@ -144,15 +145,15 @@ if ! "$cygwin" && ! "$darwin" && ! "$nonstop" ; then
     case $MAX_FD in #(
       max*)
         # In POSIX sh, ulimit -H is undefined. That's why the result is checked to see if it worked.
-        # shellcheck disable=SC3045
+        # shellcheck disable=SC2039,SC3045
         MAX_FD=$( ulimit -H -n ) ||
             warn "Could not query maximum file descriptor limit"
     esac
     case $MAX_FD in  #(
       '' | soft) :;; #(
       *)
         # In POSIX sh, ulimit -n is undefined. That's why the result is checked to see if it worked.
-        # shellcheck disable=SC3045
+        # shellcheck disable=SC2039,SC3045
         ulimit -n "$MAX_FD" ||
             warn "Could not set maximum file descriptor limit to $MAX_FD"
     esac
@@ -201,11 +202,11 @@ fi
 # Add default JVM options here. You can also use JAVA_OPTS and GRADLE_OPTS to pass JVM options to this script.
 DEFAULT_JVM_OPTS='"-Xmx64m" "-Xms64m"'
 
-# Collect all arguments for the java command;
-#   * $DEFAULT_JVM_OPTS, $JAVA_OPTS, and $GRADLE_OPTS can contain fragments of
-#     shell script including quotes and variable substitutions, so put them in
-#     double quotes to make sure that they get re-expanded; and
-#   * put everything else in single quotes, so that it's not re-expanded.
+# Collect all arguments for the java command:
+#   * DEFAULT_JVM_OPTS, JAVA_OPTS, JAVA_OPTS, and optsEnvironmentVar are not allowed to contain shell fragments,
+#     and any embedded shellness will be escaped.
+#   * For example: A user cannot expect ${Hostname} to be expanded, as it is an environment variable and will be
+#     treated as '${Hostname}' itself on the command line.
 
 set -- \
         "-Dorg.gradle.appname=$APP_BASE_NAME" \

gradlew.bat

@@ -43,11 +43,11 @@ set JAVA_EXE=java.exe
 %JAVA_EXE% -version >NUL 2>&1
 if %ERRORLEVEL% equ 0 goto execute
 
-echo.
-echo ERROR: JAVA_HOME is not set and no 'java' command could be found in your PATH.
-echo.
-echo Please set the JAVA_HOME variable in your environment to match the
-echo location of your Java installation.
+echo. 1>&2
+echo ERROR: JAVA_HOME is not set and no 'java' command could be found in your PATH. 1>&2
+echo. 1>&2
+echo Please set the JAVA_HOME variable in your environment to match the 1>&2
+echo location of your Java installation. 1>&2
 
 goto fail
 
@@ -57,11 +57,11 @@ set JAVA_EXE=%JAVA_HOME%/bin/java.exe
 
 if exist "%JAVA_EXE%" goto execute
 
-echo.
-echo ERROR: JAVA_HOME is set to an invalid directory: %JAVA_HOME%
-echo.
-echo Please set the JAVA_HOME variable in your environment to match the
-echo location of your Java installation.
+echo. 1>&2
+echo ERROR: JAVA_HOME is set to an invalid directory: %JAVA_HOME% 1>&2
+echo. 1>&2
+echo Please set the JAVA_HOME variable in your environment to match the 1>&2
+echo location of your Java installation. 1>&2
 
 goto fail
 

models/ivoa/build.gradle.kts

@@ -1,5 +1,5 @@
 plugins {
-    id("net.ivoa.vo-dml.vodmltools") version "0.4.4"
+    id("net.ivoa.vo-dml.vodmltools") version "0.5.1"
 //    id ("com.diffplug.spotless") version "5.17.1"
     `maven-publish`
     id("io.github.gradle-nexus.publish-plugin") version "1.3.0"