isa-tab-exporter

What?

The aim of isa-tab-exporter is to provide the refinery-platform with a means to utilize the Python3-based ISA-tools API while it remains in the land of Python2, but development has been done in a general manner so that anyone facing this same issue could reuse this code.

How?

isa-tab-exporter provides terraform code that will create/manage AWS infrastructure allowing for POST requests containing valid ISA-JSON to be converted into a full blown ISA-Tab Archive .zip file and returned to the end user.

Specifically, an API Gateway deployment is created that accepts these POST requests, and proxies said requests to a Lambda function that it triggers which will execute the ISA-tools API code to transform the ISA-JSON to a valid ISA-Tab Archive .zip file.

Pre-Reqs:

• terraform
• docker
• Python 3.6.x

Running Tests:

• pip install -r requirements-test.txt
• nosetests

Note: Some of the provided tests are end-to-end tests and are a bit slow. Run fast or slow tests in isolation with the following commands:

# Just the speedy ones
$ nosetests -a '!slow'

# Just the slow ones
$ nosetests -a 'slow'

Deployment:

The deploy.sh script will build the Lambda's Python requirements using an Amazon Linux docker container, and provision the AWS infrastructure using Terraform.

$ ./deployment/scripts/deploy.sh

...

Apply complete! Resources: 20 added, 0 changed, 0 destroyed.

Outputs:

api_gateway_deployment_invoke_url = https://XXXXXXXXX.execute-api.us-east-1.amazonaws.com/development/isa-tab-export

See the deployment README for more deployment related information.

CI/CD:

Currently Travis is used as the CI provder

• Python code is linted using flake8 and formatted using black
• Terraform code is formatted and validated using its own utilities
• Python tests are run using nosetests
• Code coverage metrics are collected and available here
• The latest version of the API Gateway deployment and Lambda function is deployed

Example Usage:

cURL

$ curl -X POST \
  -O \ # -O, --remote-name   Write output to a file named as the remote file
  -J \ #  -J, --remote-header-name Use the header-provided filename
  https://XXXXXXXXXX.execute-api.us-east-1.amazonaws.com/development/isa-tab-export \
  -d '{
    "isatab_filename": "My Cool ISA-Tab",
    "isatab_contents": <Valid ISA-JSON content (Take a peek at test_data/isa_json/*)>
}'

  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100  4641  100  4604  100    37    767      6  0:00:06  0:00:06 --:--:--  1301
curl: Saved to filename 'My Cool ISA-Tab.zip'

Postman

---

Lambda & Triggers

APIGateway Setup