Remove ftp image and build (#2070)

* Remove ftp functionality https://eaflood.atlassian.net/browse/IWTF-4278 Remove ftp functionality as package we use in fulfilment and pocl jobs (ssh2-sftp-client) has critical vulnerability * remove ssh2 sftp client * remove ssh2 sftp client and fix lint * remove reference * fix tests * fix tests and lint * update tests * refactor * refactor * update tests * refactor tests * s3spec * undefined token * rename file and undo removal of stores3 * rename file and undo removal of stores3 * add missing mock * Remove ssh2 mock * Remove ftp image and build https://eaflood.atlassian.net/browse/IWTF-4279 remove any FTP images from our Docker config * remove functionality from config * remove ftp reference * remove extra line
DEFRA · Nov 25, 2024 · 0469101 · 0469101
1 parent 3882521
commit 0469101
Show file tree

Hide file tree

Showing 8 changed files with 10 additions and 169 deletions.
diff --git a/docker/env/fulfilment_job.env.example b/docker/env/fulfilment_job.env.example
@@ -20,11 +20,6 @@ FULFILMENT_S3_BUCKET=fulfilment-audit
 FULFILMENT_FILE_SIZE=5000
 
 # FTP Settings
-FULFILMENT_FTP_HOST=host.docker.internal
-FULFILMENT_FTP_PORT=2222
-FULFILMENT_FTP_PATH=/share/fulfilment
-FULFILMENT_FTP_USERNAME=test_sftp_user
-FULFILMENT_FTP_KEY_SECRET_ID=/dev/fsh/local/sftp/ssh_ed25519_host_key
 FULFILMENT_SEND_UNENCRYPTED_FILE=true
 FULFILMENT_PGP_PUBLIC_KEY_SECRET_ID=/dev/fsh/local/
 

diff --git a/docker/infrastructure.yml b/docker/infrastructure.yml
@@ -107,30 +107,3 @@ services:
     deploy:
       restart_policy:
         condition: on-failure
-
-  #######################################################
-  # Test SFTP server
-  #######################################################
-  ftp:
-    image: rod_licensing/ftp:${TAG:-latest}
-    build:
-      dockerfile: Dockerfile
-      context: ./resources/infrastructure/sftp
-    depends_on:
-      - localstack
-    ports:
-      - '2222:22'
-    volumes:
-      - ./volumes/sftp:/home/test_sftp_user/share
-    environment:
-      SFTP_USER: test_sftp_user
-      SFTP_FOLDERS: share/pocl;share/fulfilment
-      AWS_SECRETSMANAGER_ENDPOINT: http://host.docker.internal:4566
-      AWS_DEFAULT_REGION: eu-west-2
-      AWS_ACCESS_KEY_ID: local
-      AWS_SECRET_ACCESS_KEY: local
-      SSH_HOST_ED25519_SECRET_ID: /dev/fsh/local/sftp/ssh_ed25519_host_key
-      SSH_HOST_RSA_SECRET_ID: /dev/fsh/local/sftp/ssh_rsa_host_key
-    deploy:
-      restart_policy:
-        condition: on-failure
diff --git a/docker/resources/infrastructure/sftp/Dockerfile b/docker/resources/infrastructure/sftp/Dockerfile
diff --git a/docker/resources/infrastructure/sftp/files/entrypoint.sh b/docker/resources/infrastructure/sftp/files/entrypoint.sh
diff --git a/docker/resources/infrastructure/sftp/files/sshd_config b/docker/resources/infrastructure/sftp/files/sshd_config
diff --git a/docker/volumes/sftp/fulfilment/README.md b/docker/volumes/sftp/fulfilment/README.md
diff --git a/docker/volumes/sftp/pocl/README.md b/docker/volumes/sftp/pocl/README.md
diff --git a/packages/fulfilment-job/README.md b/packages/fulfilment-job/README.md
@@ -19,21 +19,16 @@ provider.
 
 # Environment variables
 
-| name                                | description                                                                               | required | default | valid                                                                   | notes |
-| ----------------------------------- | ----------------------------------------------------------------------------------------- | :------: | ------- | ----------------------------------------------------------------------- | ----- |
-| NODE_ENV                            | Node environment                                                                          |    no    |         | development, test, production                                           |       |
-| FULFILMENT_FILE_SIZE                | The maximum number of records written to an aggregated fulfilment file                    |   yes    |         |                                                                         |       |
-| FULFILMENT_FTP_HOST                 | The hostname of the target FTP server                                                     |   yes    |         |                                                                         |       |
-| FULFILMENT_FTP_PORT                 | The port of the FTP service on the target server                                          |   yes    |         |                                                                         |       |
-| FULFILMENT_FTP_PATH                 | The base path under which files should be written to the FTP server                       |   yes    |         |                                                                         |       |
-| FULFILMENT_FTP_USERNAME             | The username used to authenticate with the FTP server                                     |   yes    |         |                                                                         |       |
-| FULFILMENT_FTP_KEY_SECRET_ID        | The ID of the secret in AWS secrets manager which contains the SSH key for authentication |   yes    |         |                                                                         |       |
-| FULFILMENT_S3_BUCKET                | The name of the AWS S3 bucket in which to stage and aggregate fulfilment data             |   yes    |         |                                                                         |       |
-| FULFILMENT_SEND_UNENCRYPTED_FILE    | Flag for whether to send the unencrypted fulfilment file                                  |    no    | false   | true, false, 0, 1                                                       |       |
-| FULFILMENT_PGP_PUBLIC_KEY_SECRET_ID | The secret id for the file encryption public key                                          |   yes    |         |                                                                         |       |
-| DEBUG                               | Use to enable output of debug information to the console                                  |   yes    |         | fulfilment:\*, fulfilment:staging, fulfilment:transport, fulfilment:ftp |       |
-| AIRBRAKE_HOST                       | URL of airbrake host                                                                      |    no    |         |                                                                         |       |
-| AIRBRAKE_PROJECT_KEY                | Project key for airbrake logging                                                          |    no    |         |                                                                         |       |
+| name                                | description                                                                   | required | default | valid                                                                   | notes |
+| ----------------------------------- | ----------------------------------------------------------------------------- | :------: | ------- | ----------------------------------------------------------------------- | ----- |
+| NODE_ENV                            | Node environment                                                              |    no    |         | development, test, production                                           |       |
+| FULFILMENT_FILE_SIZE                | The maximum number of records written to an aggregated fulfilment file        |   yes    |         |                                                                         |       |
+| FULFILMENT_S3_BUCKET                | The name of the AWS S3 bucket in which to stage and aggregate fulfilment data |   yes    |         |                                                                         |       |
+| FULFILMENT_SEND_UNENCRYPTED_FILE    | Flag for whether to send the unencrypted fulfilment file                      |    no    | false   | true, false, 0, 1                                                       |       |
+| FULFILMENT_PGP_PUBLIC_KEY_SECRET_ID | The secret id for the file encryption public key                              |   yes    |         |                                                                         |       |
+| DEBUG                               | Use to enable output of debug information to the console                      |   yes    |         | fulfilment:\*, fulfilment:staging, fulfilment:transport, fulfilment:ftp |       |
+| AIRBRAKE_HOST                       | URL of airbrake host                                                          |    no    |         |                                                                         |       |
+| AIRBRAKE_PROJECT_KEY                | Project key for airbrake logging                                              |    no    |         |                                                                         |       |
 
 ### See also: