09-from-hpc.qmd

# From HPC to Cloud

## Key Players in High Performance Computing

![Key Players in HPC](images/hpc_players.png){#fig-hpc-players}

- Our Machine
- Head Node
- HPC Worker
- Shared Storage

## Analogies with HPC

|Component|HPC|DNAnexus|
|---------|---|--------|
|**Driver/Requestor**|Head Node of Cluster|API Server|
|**Submission Script Language**|PBS/SLURM|dx-toolkit|
|**Worker**|Requested from private pool of machines in cluster|Requested from Pool of Machines from AWS/Azure|
|**Shared Storage**|Shared File System (Lustre, GPFS)|Project Storage|
|**Worker I/O**|Handled by Shared File System|Transferred to/from Project Storage by Worker|

## HPC vs. DNAnexus Commands

|Task   |dx-toolkit |PBS   |SLURM    |
|-------|-----------|------|---------|
|**Run Job** |`dx run <app-id> <script>`|`qsub <script>`|`sbatch <script>`|
|**Monitor Jobs** |`dx find jobs`|`qstat`|`squeue`|
|**Kill Job**|`dx terminate <jobid>`|`qdel <jobid>`|`scancel <jobid>`|