v0.4.35

• default base docker image set to CALDP_20220406_CAL_final
• default crds update to hst_1002.pmap
• bugfix for model ingest lambda pprint (missing optional param "sort_dicts" after downgrade to 3.7 runtime)

v0.4.34

• pytest via moto (no need for aws creds)
• fix ami-rotation issues caused by yum
• updated memory model training
• bugfix; blackboard 'createdAt' default value supplied (hotfixed previously)
• significantly reduce frequency of blackboard lambda to avoid problems with the file in the storage gateway
• refactor terraform for ami rotation user-data
• terraform package version bumps
• remove terraformed ecr repo in each account; use central ecr via ssm param instead
• significant refactor of deployment scripts to handle central ecr and image promotion/deletion that goes with it
• cache refresh period increased to 9 minutes from 5 (continued gateway tweaks)

v0.4.33

• force java update in Predict image for log4j vulnerability

v0.4.32

• force update of nss due to critical security vulnerability

v0.4.31

• base image update to CALDP_20211129_CAL_final

v0.4.30

• readme updates regarding modeling of job resource requirements
• automated AMI rotation via lambda launching EC2 with user-data that runs terraform
• messages can now contain payloads to override certain job params
• various cert fixes (order of some installations in docker and user-data had to be moved around)
• job predict lambda now uses data from dynamodb for an ipppssoot, if it is present already from a previous processing run
• tensorflow update to 2.6.2
• various bugfixes for the ML model in the lambda and training (feature params were not guaranteed to be in the same order)
• re-introduced tainting off JobPredict lambda, which uses docker backend, to prevent cached image being used by lambda

v0.4.29 - dynamodb

• upon job completion, a lambda posts the training data for each job into a dynamodb. Removes the need for someone to manually scrape the training data after a processing run, and persists the data in dynamo.
• The Batch training job also backs up the contents to S3 upon completion
• tensorflow updated to 2.5 for security reasons. Requires python 3.9 in the docker image, which requires building python from source as there's no 3.9 binary
• updated base image to CALDP_20210827_CAL_final and default CRDS context to hst_0943.pmap

hotfix for unexpected newline chars in AWS API response

• The AWS Batch API returns some long strings with newline characters in them. We were writing these strings to the blackboardAWS snapshot without sanitizing the newlines. This was causing the file to occasionally be formatted incorrectly, breaking the on-premise poller.

hotfix for model training user

• adds correct user in model training docker image

AMI rotation improvements

• simple python script to check for running Batch jobs from ci node
• AMI rotation script now pulls keywords from ssm params to ensure correct terraform source version