Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Separate default engine parameters for different engines in the service info #185

Open
vinjana opened this issue Sep 6, 2022 · 2 comments

Comments

@vinjana
Copy link

vinjana commented Sep 6, 2022

The API supports multiple workflow engines per WES server. But in the service info the default_workflow_engine_parameters is just a flat list of dictionaries like this

"default_workflow_engine_parameters": [
    {
      "name": "string",
      "type": "string",
      "default_value": "string"
    },
    ...
  ]

Does this mean that all engines must support the same set of parameters with the same defaults? Even if these are very different engines like Nextflow, Snakemake and Toil? This would seem little reasonable.

Wouldn't it make sense to either define separate lists of default parameters for different engines (or even versions)? This could be done in a global default_workflow_engine_parameters structure. E.g.

"default_workflow_engine_parameters": {
   "engineA":  [
        {
          "name": "string",
          "type": "string",
          "default_value": "string"
       },
       ...
    ],
    ...
}

Or it could be done together with some new definition of workflow_engines (following #122):

"workflow_engines": [
    {
      "name": "cwltool",
      "version": "1.0.20181201184214",
      "workflow_types": [
        {
          "language_type": "CWL",
          "language_version": "v1.0"
        }
      ],
      "default_parameters": [
         {
           "name": "string",
            "type": "string",
           "default_value": "string"
        },
        ...
      ]
    }
  ]
@patmagee
Copy link
Contributor

@vinjana another approach that implementations could do is devise their own set of workflow_engine_parameters and map those onto different engines themselves. This does still imply that various engines need to have some sort of property that a given parameter can be mapped to

But since we are starting to have multiple implementations with multiple engines behind the scenes, it is starting to appear that we will need a better strategy for this. I think the second approach you suggest makes a lot of sense. This would allow a way for the API to define the set of parameters that each engine supports independently of each other but also still keep the global default_workflow_engine_parameters and allow the WES api to have it's own set of parameters

@uniqueg
Copy link
Contributor

uniqueg commented Sep 27, 2022

I also think that the specs should account for this use case and would be very happy to see a PR for this (also preferring option 2 like @patmagee).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants