-
Notifications
You must be signed in to change notification settings - Fork 0
/
intro.html
162 lines (162 loc) · 7.96 KB
/
intro.html
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
<html xmlns="http://www.w3.org/1999/xhtml">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
<meta http-equiv="Content-Style-Type" content="text/css" />
<meta name="generator" content="pandoc" />
<meta name="author" content="August 7, 2018" />
<title>Data Carpentry workshop: Compational Resources at Berkeley and Beyond</title>
<style type="text/css">code{white-space: pre;}</style>
</head>
<body>
<div id="header">
<h1 class="title">Data Carpentry workshop: Compational Resources at Berkeley and Beyond</h1>
<h2 class="author">August 7, 2018</h2>
<h3 class="date">Chris Paciorek</h3>
</div>
<h1 id="outline">Outline</h1>
<p>Berkeley Research IT helps researchers through its Berkeley Research Computing (BRC) and Research Data Management (RDM) programs.</p>
<p>This (brief) session will cover the following topics:</p>
<ul>
<li>Savio campus cluster
<ul>
<li>Getting access to the system - faculty computing allowance, condo</li>
<li>Savio computing nodes</li>
<li>Cluster job submission/scheduling</li>
</ul></li>
<li>Disk storage at Berkeley
<ul>
<li>Where can I put my stuff</li>
<li>Data transfer</li>
</ul></li>
<li>National resources (XSEDE)</li>
<li>How to get help</li>
</ul>
<h1 id="system-capabilities-and-hardware">System capabilities and hardware</h1>
<p>Berkeley Research Computing runs the campus cluster, Savio.</p>
<ul>
<li>Savio is a >380-node, >8000-core, >169000-gpu-core Linux cluster rated at >350 peak teraFLOPS.
<ul>
<li>about 174 compute nodes provided by the UC Berkeley for general access</li>
<li>about 211 compute nodes contributed by researchers in the Condo program</li>
</ul></li>
</ul>
<h1 id="getting-access-to-the-system---fca-and-condo">Getting access to the system - FCA and condo</h1>
<ul>
<li>All regular Berkeley faculty can request 300,000 service units (roughly core-hours) per year through the <a href="http://research-it.berkeley.edu/services/high-performance-computing/faculty-computing-allowance">Faculty Computing Allowance (FCA)</a></li>
<li>Researchers can also purchase nodes for their own priority access and gain access to the shared Savio infrastructure and to the ability to <em>burst</em> to additional nodes through the <a href="http://research-it.berkeley.edu/services/high-performance-computing/condo-cluster-program">condo cluster program</a></li>
<li>Instructors can request an <a href="http://research-it.berkeley.edu/programs/berkeley-research-computing/instructional-computing-allowance">Instructional Computing Allowance (ICA)</a>.</li>
</ul>
<p>Faculty/principal investigators can allow researchers working with them to get user accounts with access to the FCA or condo resources available to the faculty member.</p>
<h1 id="savio-computing-nodes">Savio computing nodes</h1>
<p><a href="https://research-it.berkeley.edu/services/high-performance-computing">Savio</a> provides access to the following types of computational resources:</p>
<ul>
<li>full 20/24/28 core nodes scheduled per-node</li>
<li>'htc' nodes scheduled per-core</li>
<li>GPU nodes (scheduled per GPU)</li>
<li>big-memory nodes (scheduled per node)</li>
<li>Jupyter notebooks</li>
<li>a visualization/remote desktop node</li>
</ul>
<p>Let's take a look at the hardware specifications of the computing nodes on the cluster <a href="https://research-it.berkeley.edu/services/high-performance-computing/user-guide/savio-user-guide">(see the <em>Hardware Configuration</em> section of this document)</a>.</p>
<p>The nodes are divided into several pools, called partitions. These partitions have different restrictions and costs associated with them <a href="https://research-it.berkeley.edu/services/high-performance-computing/user-guide/savio-user-guide">(see the <em>Scheduler Configuration</em> section of this document)</a>. Any job you submit must be submitted to a partition to which you have access.</p>
<h1 id="submitting-jobs-accounts-and-partitions">Submitting jobs: accounts and partitions</h1>
<p>All computations are done by submitting jobs to the scheduling software that manages jobs on the cluster, called SLURM.</p>
<ul>
<li>interactive jobs</li>
<li>batch/background jobs</li>
</ul>
<p>Here's an example job script for a batch job. You'll need to modify the various "--" flags for your own work.</p>
<pre><code>#!/bin/bash
# Job name:
#SBATCH --job-name=test
#
# Account:
#SBATCH --account=fc_paciorek
#
# Partition:
#SBATCH --partition=savio2
#
# Number of tasks (e.g., MPI tasks, here 2 nodes worth)
#SBATCH --ntasks=48
#
# Wall clock limit (30 minutes here):
#SBATCH --time=00:30:00
#
## Command(s) to run:
module load python/3.6
python calc.py >& calc.out</code></pre>
<h1 id="genomics-on-savio-things-to-keep-in-mind">Genomics on Savio: things to keep in mind</h1>
<ul>
<li>some standard genomics software may not be installed; in some cases we can help</li>
<li>three-day time limit for FCAs (but not condos)</li>
<li>long queue for jobs up to 10 days but limited cores and not high memory</li>
<li>memory limits sometimes are an issue</li>
</ul>
<h1 id="disk-space-options-on-savio-and-on-campus-broadly">Disk space options on Savio and on campus broadly</h1>
<p>Here are some options for moderate-large disk storage options:</p>
<ul>
<li>Savio project storage: 200 GB, backed up</li>
<li>Savio scratch: 1.5 PB shared across all users, not backed up, subject to removal</li>
<li>Savio condo (purchase) storage: roughly $6000 per 42 TB</li>
<li>Berkeley Box: unlimited, 15 GB file size limit</li>
<li>bDrive (Berkeley Google drive): unlimited</li>
</ul>
<p>More details on Savio storage are here <a href="https://research-it.berkeley.edu/services/high-performance-computing/user-guide/savio-user-guide">here in the <em>Storage and Backup</em> section</a>.</p>
<h1 id="data-transfer-for-large-data">Data transfer for large data</h1>
<p>Some options include:</p>
<ul>
<li><a href="https://research-it.berkeley.edu/services/high-performance-computing/using-globus-connect-savio">Globus</a> (to/from Savio, laptop, XSEDE)</li>
<li><a href="https://research-it.berkeley.edu/services/research-data-management-service/take-advantage-unlimited-bdrive-storage-using-rclone">rclone</a> (to/from bDrive and Berkeley Box)</li>
</ul>
<h1 id="nsfs-xsede-to-infinity-and-beyond">NSF's XSEDE: To infinity and beyond</h1>
<p><a href="https://xsede.org">XSEDE</a> provides free access to NSF-funded clusters around the US, e.g.,</p>
<ul>
<li>Bridges and Comet: Savio-like clusters
<ul>
<li>CPUS, GPUs, big-memory</li>
<li>also generally have time limits</li>
</ul></li>
<li>Jetstream cloud environment:
<ul>
<li>long-running VMs</li>
<li>GUIs</li>
<li>science gateways (e.g., interface to your own project's products)</li>
</ul></li>
<li>there are other clusters as well</li>
</ul>
<h1 id="accessing-xsede">Accessing XSEDE</h1>
<p>Several modes of (free) access:</p>
<ul>
<li>access through BRC (email [email protected])</li>
<li>startup allocation via application - proof of concept allocation, think 50000-100000 core-hours</li>
<li>research allocation via application</li>
</ul>
<p>BRC can help with all of these.</p>
<h1 id="how-to-get-additional-help">How to get additional help</h1>
<ul>
<li>For technical issues and questions about using Savio:
<ul>
<li>[email protected]</li>
</ul></li>
<li>For questions about computing resources in general, including cloud computing:
<ul>
<li>[email protected] or [email protected]</li>
</ul></li>
<li>For questions about data management (including HIPAA-protected data):
<ul>
<li>[email protected] or [email protected]</li>
</ul></li>
<li>Office hours for any of the above topics:
<ul>
<li>Tues. 10-12, Wed. 1:30-3, Thur. 9:30-11:30 in AIS (Dwinelle 117)</li>
</ul></li>
</ul>
<p>Don't hesitate to contact us; we're friendly, even with basic questions.</p>
<h1 id="upcoming-events">Upcoming events</h1>
<ul>
<li>Savio intro workshop Thursday September 17</li>
<li>Other trainings planned for the fall</li>
</ul>
</body>
</html>