Skip to content

Commit 578d20f

Browse files
author
Trevor Keller
committed
Merge branch 'nersc'
2 parents ac9b0bc + e181924 commit 578d20f

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

46 files changed

+733
-139
lines changed

MAINTENANCE.md

Lines changed: 5 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -38,9 +38,8 @@ of the HPC Carpentry community.
3838

3939
### Slack
4040

41-
The main Carpentries Slack channel is reachable at
42-
[swcarpentry.slack.com][swc-slack]. To become a member, you can
43-
trigger in invitation from [here][slack-invite].
41+
HPC Carpentry participates in the The Carpentries' Slack channel.
42+
Click through for a [Slack invite][slack-invite].
4443

4544
You can go directly to the HPC Carpentry sub-channel at
4645
[#hpc-carpentry][hpc-slack]. These are the things to bookmark —
@@ -91,8 +90,8 @@ assigned to `CHECKERS` near the bottom of `bin/lesson_check.py`.
9190
[project-github]: https://github.com/hpc-carpentry
9291
[coord-repo]: https://github.com/hpc-carpentry/coordination
9392
[mainsite-repo]: https://github.com/hpc-carpentry/hpc-carpentry.org
94-
[swc-slack]: https://swcarpentry.slack.com
95-
[hpc-slack]: https://swcarpentry.slack.com/#hpc-carpentry
96-
[slack-invite]: https://swc-slack-invite.herokuapp.com
93+
[swc-slack]: https://carpentries.slack.com
94+
[hpc-slack]: https://carpentries.slack.com/#hpc-carpentry
95+
[slack-invite]: https://slack-invite.carpentries.org/
9796
[topicbox]: https://carpentries.topicbox.com
9897
[zenodo]: https://zenodo.org/

config.yaml

Lines changed: 9 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -1,7 +1,7 @@
11
#------------------------------------------------------------
22
# Values for this lesson.
33
#------------------------------------------------------------
4-
4+
---
55
# Which carpentry is this (swc, dc, lc, or cp)?
66
# swc: Software Carpentry
77
# dc: Data Carpentry
@@ -17,7 +17,7 @@ carpentry: 'cp'
1717
# carpentry_description: "Custom Carpentry"
1818

1919
# Overall title for pages.
20-
title: 'Introduction to High-Performance Computing'
20+
title: 'Introduction to High-Performance Computing at NERSC'
2121

2222
# Date the lesson was created (YYYY-MM-DD, this is empty by default)
2323
created: 2017-03-24
@@ -33,7 +33,7 @@ life_cycle: 'alpha'
3333
license: 'CC-BY 4.0'
3434

3535
# Link to the source repository for this lesson
36-
source: 'https://github.com/ocaisa/probable-pancake'
36+
source: 'https://github.com/hpc-workshops/nersc-hpc-intro'
3737

3838
# Default branch of your lesson
3939
branch: 'main'
@@ -65,7 +65,7 @@ contact: 'team@carpentries.org'
6565

6666
# Order of episodes in your lesson
6767
episodes:
68-
- 10-hpc-intro.md
68+
- 10-hpc-intro.Rmd
6969
- 11-connecting.Rmd
7070
- 12-cluster.Rmd
7171
- 13-scheduler.Rmd
@@ -78,20 +78,23 @@ episodes:
7878

7979
# Information for Learners
8080
learners:
81+
- setup.md
8182

8283
# Information for Instructors
8384
instructors:
85+
- instructor-notes.Rmd
8486

8587
# Learner Profiles
8688
profiles:
89+
- learner-profiles.md
8790

8891
# Customisation ---------------------------------------------
8992
#
9093
# This space below is where custom yaml items (e.g. pinning
9194
# sandpaper and varnish versions) should live
9295

9396

94-
carpentry_description: Lesson Description
95-
url: 'https://ocaisa.github.io/probable_pancake'
97+
carpentry_description: "HPC Intro at NERSC"
98+
url: 'https://hpc-workshops.github.io/perlmutter-hpc-intro'
9699
analytics: carpentries
97100
lang: en

episodes/10-hpc-intro.md renamed to episodes/10-hpc-intro.Rmd

Lines changed: 25 additions & 29 deletions
Original file line numberDiff line numberDiff line change
@@ -22,15 +22,15 @@ Frequently, research problems that use computing can outgrow the capabilities
2222
of the desktop or laptop computer where they started:
2323

2424
- A statistics student wants to cross-validate a model. This involves running
25-
the model 1000 times -- but each run takes an hour. Running the model on
25+
the model 1000 times but each run takes an hour. Running the model on
2626
a laptop will take over a month! In this research problem, final results are
2727
calculated after all 1000 models have run, but typically only one model is
2828
run at a time (in **serial**) on the laptop. Since each of the 1000 runs is
2929
independent of all others, and given enough computers, it's theoretically
3030
possible to run them all at once (in **parallel**).
3131
- A genomics researcher has been using small datasets of sequence data, but
3232
soon will be receiving a new type of sequencing data that is 10 times as
33-
large. It's already challenging to open the datasets on a computer --
33+
large. It's already challenging to open the datasets on a computer
3434
analyzing these larger datasets will probably crash it. In this research
3535
problem, the calculations required might be impossible to parallelize, but a
3636
computer with **more memory** would be required to analyze the much larger
@@ -54,7 +54,7 @@ problems in parallel**.
5454

5555
## Jargon Busting Presentation
5656

57-
Open the [HPC Jargon Buster](../files/jargon#p1)
57+
Open the [HPC Jargon Buster](files/jargon.html#p1)
5858
in a new tab. To present the content, press `C` to open a **c**lone in a
5959
separate window, then press `P` to toggle **p**resentation mode.
6060

@@ -71,48 +71,44 @@ results.
7171
## Some Ideas
7272

7373
- Checking email: your computer (possibly in your pocket) contacts a remote
74-
machine, authenticates, and downloads a list of new messages; it also
75-
uploads changes to message status, such as whether you read, marked as
76-
junk, or deleted the message. Since yours is not the only account, the
77-
mail server is probably one of many in a data center.
78-
- Searching for a phrase online involves comparing your search term against
79-
a massive database of all known sites, looking for matches. This "query"
74+
machine, authenticates, and downloads a list of new messages; it also uploads
75+
changes to message status, such as whether you read, marked as junk, or
76+
deleted the message. Since yours is not the only account, the mail server is
77+
probably one of many in a data center.
78+
- Searching for a phrase online involves comparing your search term against a
79+
massive database of all known sites, looking for matches. This "query"
8080
operation can be straightforward, but building that database is a
8181
[monumental task][mapreduce]! Servers are involved at every step.
82-
- Searching for directions on a mapping website involves connecting your
83-
(A) starting and (B) end points by [traversing a graph][dijkstra] in
84-
search of the "shortest" path by distance, time, expense, or another
85-
metric. Converting a map into the right form is relatively simple, but
86-
calculating all the possible routes between A and B is expensive.
82+
- Searching for directions on a mapping website involves connecting your (A)
83+
starting and (B) end points by [traversing a graph][dijkstra] in search of
84+
the "shortest" path by distance, time, expense, or another metric. Converting
85+
a map into the right form is relatively simple, but calculating all the
86+
possible routes between A and B is expensive.
8787

8888
Checking email could be serial: your machine connects to one server and
8989
exchanges data. Searching by querying the database for your search term (or
90-
endpoints) could also be serial, in that one machine receives your query
91-
and returns the result. However, assembling and storing the full database
92-
is far beyond the capability of any one machine. Therefore, these functions
93-
are served in parallel by a large, ["hyperscale"][hyperscale] collection of
94-
servers working together.
95-
96-
90+
endpoints) could also be serial, in that one machine receives your query and
91+
returns the result. However, assembling and storing the full database is far
92+
beyond the capability of any one machine. Therefore, these functions are served
93+
in parallel by a large, ["hyperscale"][hyperscale] collection of servers
94+
working together.
9795

9896
:::::::::::::::::::::::::
9997

10098
::::::::::::::::::::::::::::::::::::::::::::::::::
10199

102-
103-
104100
[mapreduce]: https://en.wikipedia.org/wiki/MapReduce
105101
[dijkstra]: https://en.wikipedia.org/wiki/Dijkstra%27s_algorithm
106102
[hyperscale]: https://en.wikipedia.org/wiki/Hyperscale_computing
107103

108-
109104
:::::::::::::::::::::::::::::::::::::::: keypoints
110105

111-
- High Performance Computing (HPC) typically involves connecting to very large computing systems elsewhere in the world.
112-
- These other systems can be used to do work that would either be impossible or much slower on smaller systems.
106+
- High Performance Computing (HPC) typically involves connecting to very large
107+
computing systems elsewhere in the world.
108+
- These other systems can be used to do work that would either be impossible or
109+
much slower on smaller systems.
113110
- HPC resources are shared by multiple users.
114-
- The standard method of interacting with such systems is via a command line interface.
111+
- The standard method of interacting with such systems is via a command line
112+
interface.
115113

116114
::::::::::::::::::::::::::::::::::::::::::::::::::
117-
118-

episodes/11-connecting.Rmd

Lines changed: 11 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -38,15 +38,17 @@ results.
3838
If you have ever opened the Windows Command Prompt or macOS Terminal, you have
3939
seen a CLI. If you have already taken The Carpentries' courses on the UNIX
4040
Shell or Version Control, you have used the CLI on your *local machine*
41-
extensively. The only leap to be made here is to open a CLI on a *remote machine*,
42-
while taking some precautions so that other folks on the network can't see (or
43-
change) the commands you're running or the results the remote machine sends
44-
back. We will use the Secure SHell protocol (or SSH) to open an encrypted
45-
network connection between two machines, allowing you to send \& receive text
46-
and data without having to worry about prying eyes.
47-
48-
![](/fig/connect-to-remote.svg){max-width="50%" alt="Connect to cluster"}
49-
41+
extensively. The only leap to be made here is to open a CLI on a *remote
42+
machine*, while taking some precautions so that other folks on the network
43+
can't see (or change) the commands you're running or the results the remote
44+
machine sends back. We will use the Secure SHell protocol (or SSH) to open an
45+
encrypted network connection between two machines, allowing you to send \&
46+
receive text and data without having to worry about prying eyes.
47+
48+
![connect-to-remote.svg](/fig/connect-to-remote.svg){
49+
max-width="50%"
50+
alt="Connect to cluster. "
51+
}
5052

5153
SSH clients are usually command-line tools, where you provide the remote
5254
machine address as the only required argument. If your username on the remote

episodes/12-cluster.Rmd

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -75,7 +75,7 @@ sends back. We will use the Secure SHell protocol (or SSH) to open an encrypted
7575
network connection between two machines, allowing you to send & receive text
7676
and data without having to worry about prying eyes.
7777

78-
![Connect to cluster](fig/connect-to-remote.svg){alt-text="Connect to cluster"}
78+
![Connect to cluster](fig/connect-to-remote.svg){alt="Connect to cluster"}
7979

8080
Make sure you have a SSH client installed on your laptop. Refer to the
8181
[setup](../index.md) section for more details. SSH clients are

episodes/13-hpcc-scheduler/hpcc/section2.rmd

Lines changed: 0 additions & 8 deletions
This file was deleted.
Lines changed: 54 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,54 @@
1+
# ----------------
2+
# NERSC Perlmutter
3+
# ----------------
4+
# If you need an account, please visit <nersc.gov>
5+
#
6+
# Compute responsibly.
7+
---
8+
snippets: "NERSC_Perlmutter_slurm"
9+
10+
local:
11+
prompt: "local:~$"
12+
bash_shebang: "#!/bin/bash"
13+
14+
remote:
15+
name: "Perlmutter"
16+
host_id: "Perlmutter"
17+
login: "saul.nersc.gov"
18+
host: "saul"
19+
node: "pm001"
20+
location: "saul.nersc.gov"
21+
homedir: "/home"
22+
user: "user"
23+
prompt: 'remote:~$'
24+
prompt_work: "user@saul:/work/user"
25+
module_python3: "Python"
26+
bash_shebang: "#!/bin/bash"
27+
28+
sched:
29+
name: "Slurm"
30+
reservation: "trn010"
31+
submit:
32+
name: "sbatch"
33+
options: ""
34+
queue:
35+
debug: "smnode"
36+
testing: "cpubase_bycore_b1"
37+
status: "squeue"
38+
flag:
39+
user: "-u user"
40+
interactive: "--pty /bin/bash"
41+
histdetail: "-l -j"
42+
name: "--job-id="
43+
time: "--time="
44+
queue: "--constraint="
45+
nodes: "--nodes="
46+
tasks: "--ntasks="
47+
del: "scancel"
48+
interactive: "srun"
49+
info: "sinfo"
50+
comment: "#SBATCH"
51+
hist: "sacct -u user"
52+
hist_filter: ""
53+
partition: "cpu"
54+
qos: "debug"
Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,6 @@
1+
```output
2+
PARTITION AVAIL TIMELIMIT NODES STATE NODELIST
3+
cpubase_bycore_b1* up infinite 4 idle node[1-2],smnode[1-2]
4+
node up infinite 2 idle node[1-2]
5+
smnode up infinite 2 idle smnode[1-2]
6+
```

0 commit comments

Comments
 (0)