Slack Hook Notification

Release Notes

For: uc-cdis/dcf-dataservice

Notes since tag: 1.6.0

Notes to tag/commit: a51fd5f

Generated: 2022-10-05

New Features

• Implemented slack webhook, send notification at beginning and completion of
  scripts. Webhook URL configuration in dcf_dataservice_settings file on
  dcfprod. (#111)

Package updates, Google Dataflow

Release Notes

For: uc-cdis/dcf-dataservice

Notes since tag: 1.5.0

Notes to tag/commit: 383fef9

Generated: 2022-08-11

Bug Fixes

• redaction wasn't deleting indexd records unless the gs or s3 url was still
  active, it would error out on the exception due to misnamed error property
  (#109)

Google Replicate package updates

Release Notes

For: uc-cdis/dcf-dataservice

Notes since tag: 1.4.1

Notes to tag/commit: 7d6b097

Generated: 2022-07-19

New Features

• Updated packages for google replication script (#110)
• Added GitHub workflow for building images (#110)
• Changes to Docker file for pip install/update (#110)

Dependency Updates

• google-auth (#110)
• requests (#110)
• urllib3 (#110)
• google-cloud-core (#110)
• google-resumable-media (#110)
• google-cloud-storage (#110)
• httplib2 (#110)
• pyparsing (#110)
• apache-beam[gcp] (#110)
• cdislogging (#110)
• cdiserrors (#110)
• indexclient (#110)

1.4.1

Release Notes

For: uc-cdis/dcf-dataservice

Notes since tag: 1.4.0

Notes to tag/commit: 3c66879

Generated: 2022-02-07

Bug Fixes

• Fix incorrent loggin keyword from indexd_urls to indexd_url (#104)

Package updates and Google replication change

Release Notes

For: uc-cdis/dcf-dataservice

Notes since tag: 1.3.2

Notes to tag/commit: cbe6469

Generated: 2022-02-02

Bug Fixes

• Fix dependency issues (#103)

Improvements

• don't list all of indexd or what's in bucket, just ingest what's in the
  provided manifest (#102)

Dependency Updates

• python_dateutil==2.8.0 (#103)
• requests==2.22.0 (#103)
• boto3>=1.9.111<2.0.0 (#103)
• retry<=0.9.2 (#103)
• google-auth==1.6.3 (#103)
• google-resumable-media==0.4.1 (#103)
• google-cloud-storage==1.6.0 (#103)
• httplib2==0.12.0 (#103)
• pyparsing==2.4.2 (#103)
• apache-beam[gcp]==2.16.0 (#103)
• setuptools==41.4.0 (#103)

Update packages for pip install error

Release Notes

For: uc-cdis/dcf-dataservice

Notes since tag: 1.3.1

Notes to tag/commit: bfd8519

Generated: 2021-12-14

Bug Fixes

• Change versions of packages, including apache-beam due to error during pip
  install from old beam package (#101)

Dependency Updates

• python_dateutil==2.8.2 (#101)
• requests>=2.26.0<3.0.0 (#101)
• boto3==1.20.24 (#101)
• google-auth==2.3.3 (#101)
• google-cloud==0.34.0 (#101)
• google-resumable-media==2.1.0 (#101)
• google-cloud-storage==1.43.0 (#101)
• apache-beam[gcp]==2.34.0 (#101)
• setuptools==59.6.0 (#101)

Dependency Updates

Release Notes

For: uc-cdis/dcf-dataservice

Notes since tag: 1.3.0

Notes to tag/commit: b8c0bd7

Generated: 2021-12-09

Improvements

• Change logic so that when FORCE_CREATE_MANIFEST is true, the resulting
  manifest file does not contain a FAIL_ prefix (#99)
• Add more logs surrounding creating validation output manifest (#99)

Dependency Updates

• Change order of install for apache beam and setuptools packages based on
  https://stackoverflow.com/questions/61078477/apache-beam-2-19-0-not-running-on-cloud-dataflow-anymore-due-to-could-not-find-a
  as a solution to error during google replicate job (#100)

Validation with force manifest creation option

Release Notes

For: uc-cdis/dcf-dataservice

Notes since tag: 1.2.7

Notes to tag/commit: 66bd89b

Generated: 2021-11-17

New Features

• Add logic for modifying validation process when files are expected to be
  redacted (#98)
• Validation process still error out as it did previously, but when
  FORCE_CREATE_MANIFEST True it will proceed to create a manifest output
  instead of halt the entire validation process (#98)

1.2.7

Release Notes

For: uc-cdis/dcf-dataservice

Notes since tag: 1.2.6

Notes to tag/commit: 168f1c3

Generated: 2021-10-31

Bug Fixes

• Fixed newer version of urllib does api calls differently which was
  breaking the way we called the GDC api (#97)

Bucket naming refactor

Release Notes

For: uc-cdis/dcf-dataservice

Notes since tag: 1.2.5

Notes to tag/commit: b8dcce8

Generated: 2021-10-25

Improvements

• Improved the way we separate bucket accounts (#96)
• Move bucket account list to dcf_dataservice_settings (#96)
• Improved bucket handling logic (#96)