Declare more fields as nullable

[edgarrmondragon]

From a Slack conversation:

Trying to run tap-dbt and I'm running into some issues that I think would apply more generally to other taps too, but I couldn't find an answer in the forums or slack:

1. Allowing NULL types globally: The expected schema for most of the fields in tap-dbt does not allow NULL values. For example in the runs stream, the field dbt_version is configured to expect a type of string like:

          "dbt_version": {
            "type": "string"
          }

The solution would involve one of two alternatives:

• Manually add the "null" type in all the schemas
• Refactor the schemas to use the SDK typing helpers, which make fields nullable by default.

---

• Slack
• discuss.meltano.com

[mjsqu]

I think the schema is being derived from the content retrieved from: tap_dbt.client.OPENAPI_URL. For example the following block is retrieved for the Jobs endpoint:

    Job:
      type: "object"
      required:
        - id
        - account_id
        - project_id
        - environment_id
        - dbt_version
        - name
        - execute_steps
        - state
        - triggers
        - settings
        - schedule
      properties:
        id:
          type: integer
          nullable: true
          description: Must be `null` when creating a new Job
...
        dbt_version:
          type: string
          nullable: true
          description: Overrides the dbt_version specified on the attached Environment if provided
          example: 0.17.1

Whereas the extracted schema is:

      "schema": {
        "properties": {
          "id": {
            "description": "Must be `null` when creating a new Job",
            "type": "integer"
          },
...
          "dbt_version": {
            "description": "Overrides the dbt_version specified on the attached Environment if provided",
            "type": "string"
          }

nullable does not appear to be carried over when converting the yaml to the json schema.