From e0335225029220040fae4f6222fedd96f4a93dda Mon Sep 17 00:00:00 2001 From: awat31 Date: Wed, 12 Jun 2024 14:43:37 +0100 Subject: [PATCH 1/5] Updated MFT docs --- docs/services/mft/quickstart.md | 5 ++++- docs/services/mft/sftp.md | 3 --- docs/services/mft/using-the-mft.md | 27 ++++++++++++++++++++++++++- mkdocs.yml | 7 +++---- 4 files changed, 33 insertions(+), 9 deletions(-) delete mode 100644 docs/services/mft/sftp.md diff --git a/docs/services/mft/quickstart.md b/docs/services/mft/quickstart.md index 6904e50ba..688996055 100644 --- a/docs/services/mft/quickstart.md +++ b/docs/services/mft/quickstart.md @@ -6,7 +6,7 @@ The EIDF MFT can be accessed at [https://eidf-mft.epcc.ed.ac.uk](https://eidf-mf ## How it works -The MFT provides a 'drop' zone for the project. All users in a given project will have access to the same shared transfer area. They will have the ability to upload, download, and delete files from the project's transfer area. This area is linked to a directory within the projects space on the shared backend storage. +The MFT provides a 'drop zone' for the project. All users in a given project will have access to the same shared transfer area. They will have the ability to upload, download, and delete files from the project's transfer area. This area is linked to a directory within the projects space on the shared backend storage. Files which are uploaded are owned by the Linux user 'nobody' and the group ID of whatever project the file is being uploaded to. They have the permissions:
Owner = rw
@@ -19,3 +19,6 @@ Once the file is opened on the VM, the user that opened it will become the owner By default a project won't have access to the MFT, this has to be enabled. Currently this can be done by the PI sending a request to the EIDF Helpdesk. Once the project is enabled within the MFT, every user with the project will be able to log into the MFT using their usual EIDF credentials. + +Once MFT access has been enabled for a project, PIs can give a project user access to the MFT. +A new 'eidf-mft' machine option will be available for each user within the portal, which the PI can select to grant the user access to the MFT. \ No newline at end of file diff --git a/docs/services/mft/sftp.md b/docs/services/mft/sftp.md deleted file mode 100644 index bd7b2cc9e..000000000 --- a/docs/services/mft/sftp.md +++ /dev/null @@ -1,3 +0,0 @@ -# SFTP - -Coming Soon diff --git a/docs/services/mft/using-the-mft.md b/docs/services/mft/using-the-mft.md index 4fb7a25e8..0b7e93e29 100644 --- a/docs/services/mft/using-the-mft.md +++ b/docs/services/mft/using-the-mft.md @@ -1,6 +1,6 @@ # Using the MFT Web Portal -## Logging in +## Logging in to the web browser When you reach the MFT [home page](https://eidf-mft.epcc.ed.ac.uk) you can log in using your usual VM project credentials. @@ -21,3 +21,28 @@ File egress can be done in the reverse way. By placing the file into the project Directories can be created within the project transfer directory, for example with 'Import' and 'Export' to allow for better file management. Files deleted from either the MFT portal or from the VM itself will remove it from the other, as both locations point at the same file. It's only stored in one place, so modifications made from either place will remove the file. + + +## SFTP + +Once a project and user have access to the MFT, they can connect to it using SFTP as well as through the web browser. + +This can be done by logging into the MFT URL with the user's project account: + + ```bash + + sftp [EIDF username]@eidf-mft.epcc.ed.ac.uk + +``` + +## SCP + +Files can be scripted to be upload to the MFT using SCP. + +To copy a file to the project MFT area using SCP: + +```bash + + scp /path/to/file [EIDF username]@eidf-mft.epcc.ed.ac.uk:/ + +``` \ No newline at end of file diff --git a/mkdocs.yml b/mkdocs.yml index d4788b3a8..ef40702d1 100644 --- a/mkdocs.yml +++ b/mkdocs.yml @@ -45,10 +45,6 @@ nav: - "Virtual Desktop Interface": access/virtualmachines-vdi.md - "SSH Access to VMs": access/ssh.md - "VM Flavours": services/virtualmachines/flavours.md - #- "Managed File Transfer": - # - "Quickstart": services/mft/quickstart.md - # - "Using the MFT": services/mft/using-the-mft.md - # - "SFTP": services/mft/sftp.md - "Policies": services/virtualmachines/policies.md # - "Managed JupyterHub": # - "QuickStart": services/jhub/quickstart.md @@ -82,6 +78,9 @@ nav: - "Data Management Services": - "Data Catalogue": - "Metadata information": services/datacatalogue/metadata.md + - "Managed File Transfer": + - "Quickstart": services/mft/quickstart.md + - "Using the MFT": services/mft/using-the-mft.md - "Safe Haven Services": - "Overview": safe-haven-services/overview.md - "Network Access Controls": safe-haven-services/network-access-controls.md From ca26bdfe059ceda7759008f55ba78e41b0347ab9 Mon Sep 17 00:00:00 2001 From: awat31 Date: Fri, 9 Aug 2024 14:56:22 +0100 Subject: [PATCH 2/5] Web Servers page --- docs/services/virtualmachines/policies.md | 11 +++++++++++ 1 file changed, 11 insertions(+) diff --git a/docs/services/virtualmachines/policies.md b/docs/services/virtualmachines/policies.md index c526f030e..aa7ce1b4f 100644 --- a/docs/services/virtualmachines/policies.md +++ b/docs/services/virtualmachines/policies.md @@ -39,3 +39,14 @@ We strongly advise that you keep copies of any critical data on an alternative s ## Patching of User VMs The EIDF team updates and patches the hypervisors and the cloud management software as part of the EIDF Maintenance sessions. It is the responsibility of project PIs to keep the VMs in their projects up to date. VMs running the Ubuntu and Rocky operating systems automatically install security patches and alert users at log-on (via SSH) to reboot as necessary for the changes to take effect. They also encourage users to update packages. + +## Customer-run outward facing web services + +PIs can apply to run an outward-facing service; that is a webservice on port 443, running on a project-owned VM. The policy requires the customer to accept the following conditions: + +Agreement that the customer will automatically apply security patches, run regular maintenance, and have named contacts who can act should we require it. +Agreement that should EPCC detect any problematic behaviour (of users or code), we reserve the right to remove web access. +Agreement that the customer understands all access is filtered and gated by EPCC’s Firewalls and NGINX (or other equivalent software) server such that there is no direct exposure to the internet of their application. +Agreement that the customer owns the data, has permission to expose it, and that it will not bring UoE into disrepute. + +Pis can apply for such a service on application and also at any time by contacing the EIDF Service Desk. \ No newline at end of file From fc9790ccb4a028d9602ad50c721d7bf65b9a451f Mon Sep 17 00:00:00 2001 From: awat31 Date: Mon, 19 Aug 2024 13:35:53 +0100 Subject: [PATCH 3/5] Remove nav to MFT docs --- mkdocs.yml | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/mkdocs.yml b/mkdocs.yml index 507780766..d5b1f2964 100644 --- a/mkdocs.yml +++ b/mkdocs.yml @@ -81,9 +81,9 @@ nav: - "Tutorial": services/s3/tutorial.md - "Data Catalogue": - "Metadata information": services/datacatalogue/metadata.md - - "Managed File Transfer": - - "Quickstart": services/mft/quickstart.md - - "Using the MFT": services/mft/using-the-mft.md + #- "Managed File Transfer": + # - "Quickstart": services/mft/quickstart.md + # - "Using the MFT": services/mft/using-the-mft.md - "Safe Haven Services": - "Overview": safe-haven-services/overview.md - "Network Access Controls": safe-haven-services/network-access-controls.md From 408d3949e5ac3035eacbee7d633942340f028736 Mon Sep 17 00:00:00 2001 From: Amy Krause Date: Mon, 19 Aug 2024 13:43:34 +0100 Subject: [PATCH 4/5] fix trailing whitespace (precommit) --- docs/services/mft/quickstart.md | 2 +- docs/services/mft/using-the-mft.md | 2 +- docs/services/virtualmachines/policies.md | 4 ++-- 3 files changed, 4 insertions(+), 4 deletions(-) diff --git a/docs/services/mft/quickstart.md b/docs/services/mft/quickstart.md index 688996055..1d0fc5294 100644 --- a/docs/services/mft/quickstart.md +++ b/docs/services/mft/quickstart.md @@ -21,4 +21,4 @@ By default a project won't have access to the MFT, this has to be enabled. Curre Once the project is enabled within the MFT, every user with the project will be able to log into the MFT using their usual EIDF credentials. Once MFT access has been enabled for a project, PIs can give a project user access to the MFT. -A new 'eidf-mft' machine option will be available for each user within the portal, which the PI can select to grant the user access to the MFT. \ No newline at end of file +A new 'eidf-mft' machine option will be available for each user within the portal, which the PI can select to grant the user access to the MFT. diff --git a/docs/services/mft/using-the-mft.md b/docs/services/mft/using-the-mft.md index 0b7e93e29..b82f34e37 100644 --- a/docs/services/mft/using-the-mft.md +++ b/docs/services/mft/using-the-mft.md @@ -45,4 +45,4 @@ To copy a file to the project MFT area using SCP: scp /path/to/file [EIDF username]@eidf-mft.epcc.ed.ac.uk:/ -``` \ No newline at end of file +``` diff --git a/docs/services/virtualmachines/policies.md b/docs/services/virtualmachines/policies.md index aa7ce1b4f..621e36ed7 100644 --- a/docs/services/virtualmachines/policies.md +++ b/docs/services/virtualmachines/policies.md @@ -40,7 +40,7 @@ We strongly advise that you keep copies of any critical data on an alternative s The EIDF team updates and patches the hypervisors and the cloud management software as part of the EIDF Maintenance sessions. It is the responsibility of project PIs to keep the VMs in their projects up to date. VMs running the Ubuntu and Rocky operating systems automatically install security patches and alert users at log-on (via SSH) to reboot as necessary for the changes to take effect. They also encourage users to update packages. -## Customer-run outward facing web services +## Customer-run outward facing web services PIs can apply to run an outward-facing service; that is a webservice on port 443, running on a project-owned VM. The policy requires the customer to accept the following conditions: @@ -49,4 +49,4 @@ Agreement that should EPCC detect any problematic behaviour (of users or code), Agreement that the customer understands all access is filtered and gated by EPCC’s Firewalls and NGINX (or other equivalent software) server such that there is no direct exposure to the internet of their application. Agreement that the customer owns the data, has permission to expose it, and that it will not bring UoE into disrepute. -Pis can apply for such a service on application and also at any time by contacing the EIDF Service Desk. \ No newline at end of file +Pis can apply for such a service on application and also at any time by contacing the EIDF Service Desk. From 1c762c5d2122f7ee6d003c355805ad6b7a5606ce Mon Sep 17 00:00:00 2001 From: Aaron Watkins <49282648+awat31@users.noreply.github.com> Date: Mon, 19 Aug 2024 14:03:36 +0100 Subject: [PATCH 5/5] Remove Whitespace --- docs/services/mft/using-the-mft.md | 1 - 1 file changed, 1 deletion(-) diff --git a/docs/services/mft/using-the-mft.md b/docs/services/mft/using-the-mft.md index b82f34e37..8b7429960 100644 --- a/docs/services/mft/using-the-mft.md +++ b/docs/services/mft/using-the-mft.md @@ -22,7 +22,6 @@ File egress can be done in the reverse way. By placing the file into the project Directories can be created within the project transfer directory, for example with 'Import' and 'Export' to allow for better file management. Files deleted from either the MFT portal or from the VM itself will remove it from the other, as both locations point at the same file. It's only stored in one place, so modifications made from either place will remove the file. - ## SFTP Once a project and user have access to the MFT, they can connect to it using SFTP as well as through the web browser.