Skip to content

Commit

Permalink
Proof read
Browse files Browse the repository at this point in the history
  • Loading branch information
adamrtalbot committed Dec 2, 2024
1 parent 8f1e262 commit d039f6a
Showing 1 changed file with 16 additions and 15 deletions.
31 changes: 16 additions & 15 deletions platform_versioned_docs/version-24.2/compute-envs/azure-batch.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -293,8 +293,8 @@ Your Seqera compute environment uses resources that you may be charged for in yo

If not described below, please use the default settings.

1. **Account**:You must have an existing Azure Batch account. Ideally, you would already have demonstrated you can run an Azure Batch task within this account. Any type of account is compatible.
1. *Quota*: You must check you have sufficient quota for the number of pools, jobs and vCPUs per series. See [Azure Batch service quotas and limits][az-batch-quotas] for more information.
1. **Account**: You must have an existing Azure Batch account. Ideally, you would already have demonstrated you can run an Azure Batch task within this account. Any type of account is compatible.
1. **Quota**: You must check you have sufficient quota for the number of pools, jobs and vCPUs per series. See [Azure Batch service quotas and limits][az-batch-quotas] for more information.
1. On the Azure Batch page of the Azure Portal, select **Pools** and then **+ Add**.
1. **Name**: Enter a Pool ID and Display Name. The ID is the one we will refer to in the Seqera Platform and/or Nextflow.
1. **Identity**: Select **User assigned** to use a managed identity for the pool. Click the "Add" for User-assigned managed identity and select the Managed Identity with the correct permissions to the storage account and Batch account.
Expand Down Expand Up @@ -322,31 +322,32 @@ $TargetDedicatedNodes = targetPoolSize;
$NodeDeallocationOption = taskcompletion;
```

1. **Start task**: This is the task that will run on each VM when it joins the pool. This can be used to install additional software on the VM.When using Batch Forge, this is used to install `azcopy` for staging files onto and off the node. Select **Enabled** and add the following command line to install `azcopy`:
1. **Start task**: This is the task that will run on each VM when it joins the pool. This can be used to install additional software on the VM. When using Batch Forge, this is used to install `azcopy` for staging files onto and off the node. Select **Enabled** and add the following command line to install `azcopy`:

```shell
bash -c "chmod +x azcopy && mkdir $AZ_BATCH_NODE_SHARED_DIR/bin/ && cp azcopy $AZ_BATCH_NODE_SHARED_DIR/bin/"
```

Click **Resource files** then select *Http url**. For the URL, add `https://nf-xpack.seqera.io/azcopy/linux_amd64_10.8.0/azcopy` and for File path type `azcopy`. Every other setting can be left default.
Click **Resource files** then select **Http url**. For the URL, add `https://nf-xpack.seqera.io/azcopy/linux_amd64_10.8.0/azcopy` and for File path type `azcopy`. Every other setting can be left default.

:::note
When not using Fusion, every node **must** have `azcopy` installed.
:::

1. **Task Slots**: Set task slots to the number of vCPUs the machine has, e.g. select `4` for a `Standard_D4_v3` VM size.
1. **Task scheduling policy**: This can be set to `Pack` or `Spread`. `Pack` will attempt to schedule tasks from the same job on the same VM, while `Spread` will attempt to distribute tasks evenly across VMs.
1. **Virtual Network**: If using a virtual network, you can select it here. Be sure to select the correct virtual network and subnet. Be aware the virtual machines are required to:
- Pull containers from the relevant container registry (e.g. quay.io, docker.io)
- Copy data from and to Azure Storage using `azcopy`
- Communicate with the head node (running Nextflow) and Seqera Platform to relay log files and information.
As such, very restrictive networking may prevent pipelines running successfully.
1. **Mount configuration**: Nextflow *only* supports Azure File Shares. Select `Azure Files Share`, then add the following details:
- **Source**: Use the URL in the format `https://${accountName}.file.core.windows.net/${fileShareName}`
- **Relative mount path**: The path to the directory where the file share will be mounted on the VM.
- **Add the Storage account name** and **Storage account key** (managed identity is not supported).

Leave the node pool to start and create a single Azure VM. Monitor to make sure the VM correctly starts and if it raises an error be sure to check it and correct any mistakes. This may require you to create a new Azure Node pool.
1. **Virtual Network**: If using a virtual network, you can select it here. Be sure to select the correct virtual network and subnet. The virtual machines require:
- Access to container registries (e.g. quay.io, docker.io) to pull containers
- Access to Azure Storage to copy data using `azcopy`
- Access to any remote files required by the pipeline e.g. AWS S3.
- Communication with the head node (running Nextflow) and Seqera Platform to relay logs and information
Note that overly restrictive networking may prevent pipelines from running successfully.
1. **Mount configuration**: Nextflow *only* supports Azure File Shares. Select `Azure Files Share`, then add:
- **Source**: URL in format `https://${accountName}.file.core.windows.net/${fileShareName}`
- **Relative mount path**: Path where the file share will be mounted on the VM
- **Storage account name** and **Storage account key** (managed identity is not supported)

Leave the node pool to start and create a single Azure VM. Monitor the VM to ensure it starts correctly. If any errors occur, check and correct them - you may need to create a new Azure node pool if issues persist.

The following settings can be modified after creating a pool:

Expand Down

0 comments on commit d039f6a

Please sign in to comment.