metadata schema update webhook fails in github

[amnonkhen]

Overview

The github webhook that pushes changes from metadata schema repo to aws s3 where they are served to schema users is showing a 502 error code.

TLDR

• This is low priority because it does not impact functionality. The error the webhook sees is merely noise.
• The webhook successfully updates the schema files in the s3 bucket. the problem is only with the way the lambda function builds the output it returns to the api gateway.

Analysis

• created aws suppurt ticket 13590404591
• possible articles:
  • https://repost.aws/knowledge-center/malformed-502-api-gateway

Results

From conversations with aws support and careful debugging of the lambda function and api gateway calls, I understand the following:

1. the webhook hits the api gateway resource
2. the gateeway resource triggers the lambda function
3. the lambda functions runs
4. the lambda function updates ingest with the schema changes.
   • This is evident by checking the latestSchemas api call and searching for a recently modified schema, such as [email protected] updated 22/8/2023
5. the lambda function does not update the changes in the bucket for dev. Prod & Staging, however, are ok:
   • This is evident by checking the schema resource. Instead of the json schema, it shows the default page with all the schema files, which does not contain.
   • dev - this happends becuase there is no "develop" branch.
   • staging
   • prod
6. The lambda function returns output in json format where the body attribute is also json, but should be a json encoded string. This is the reason it fails to be parsed by the api gateway, which results in the 502 status code seen in the github console.
7. In addition the lambda function runs on an outdated python 3.6

logs and screenshots

see errors and screenshot

See error screenshot and text:
Private Zenhub Image

Response 502

Request

Headers
Content-Length: 36
Content-Type: application/json
Date: Thu, 17 Aug 2023 16:06:54 GMT
Via: 1.1 9546eb427ef2137803aed00cad4fc426.cloudfront.net (CloudFront)
X-Amz-Apigw-Id: Jz_QTE1NoAMFmKA=
X-Amz-Cf-Id: P080rf67bD-gg-8gkAuUAJvZTdtOLBmAR_NBpVquwDEcoftFDV8YRQ==
X-Amz-Cf-Pop: IAD12-P1
X-Amzn-Errortype: InternalServerErrorException
X-Amzn-Requestid: 22eb38e1-85cc-4187-8e8c-30ef3238cf27
X-Cache: Error from cloudfront

Body
{"message": "Internal server error"}
Due to an incident, webhooks were not delivered from 2023-08-15 17:54:00 to 2023-08-15 22:47:00