forked from johannesgerer/jburkardt-cpp
-
Notifications
You must be signed in to change notification settings - Fork 0
/
mpi.html
683 lines (626 loc) · 20.7 KB
/
mpi.html
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
575
576
577
578
579
580
581
582
583
584
585
586
587
588
589
590
591
592
593
594
595
596
597
598
599
600
601
602
603
604
605
606
607
608
609
610
611
612
613
614
615
616
617
618
619
620
621
622
623
624
625
626
627
628
629
630
631
632
633
634
635
636
637
638
639
640
641
642
643
644
645
646
647
648
649
650
651
652
653
654
655
656
657
658
659
660
661
662
663
664
665
666
667
668
669
670
671
672
673
674
675
676
677
678
679
680
681
682
683
<html>
<head>
<title>
MPI - C++ Examples
</title>
</head>
<body bgcolor="#EEEEEE" link="#CC0000" alink="#FF3300" vlink="#000055">
<h1 align = "center">
MPI <br> C++ Examples
</h1>
<hr>
<p>
<b>MPI</b>
is a directory of C++ programs which
illustrate the use of the Message Passing Interface
for parallel programming.
</p>
<p>
<b>MPI</b> is a library of message passing routines. The library
allows a user to write a program in a familiar language, such as
C, C++, FORTRAN77 or FORTRAN90, and carry out a computation in
parallel on an arbitrary number of cooperating computers.
</p>
<h3 align = "center">
Overview of MPI
</h3>
<p>
A remarkable feature of <b>MPI</b> is that <i>the user writes
a single program which runs on all the computers</i>. However,
because each computer is assigned a unique identifying number,
it is possible for different actions to occur on different
machines, even though they run the same program:
<pre><code>
if ( I am processor A ) then
add a bunch of numbers
else if ( I am processor B ) then
multipy a matrix times a vector
end
</code></pre>
</p>
<p>
Another feature of <b>MPI</b> is that the data stored on each
computer is entirely separate from that stored on other computers.
If one computer needs data from another, or wants to send
a particular value to all the other computers, it must
explicitly call the appropriate library routine requesting
a data transfer. Depending on the library routine called,
it may be necessary for both sender and receiver to be
"on the line" at the same time (which means that one will
probably have to wait for the other to show up), or it is
possible for the sender to send the message to a buffer, for
later delivery, allowing the sender to proceed immediately
to further computation.
</p>
<p>
Here is a simple example of what a piece of the program would
look like, in which the number <b>X</b> is presumed to have been
computed by processor <b>A</b> and needed by processor <b>B</b>:
<pre><code>
if ( I am processor A ) then
call MPI_Send ( X )
else if ( I am processor B ) then
call MPI_Recv ( X )
end
</code></pre>
</p>
<p>
Often, an <b>MPI</b> program is written so that one computer
supervises the work, creating data, issuing it to the
worker computers, and gathering and printing the results at the
end. Other models are also possible.
</p>
<p>
It should be clear that a program using MPI to execute in parallel
will look much different from a corresponding sequential version.
The user must divide the problem data among the different processes,
rewrite the algorithm to divide up work among the processes,
and add explicit calls to transfer values as needed from the
process where a data item "lives" to a process that needs that
value.
</p>
<p>
A C++ program, subroutine or function that calls any
MPI function, or uses an MPI-defined variable, must include
the line
<pre><code>
include "mpi.h"
</code></pre>
so that the types of the MPI variables are defined.
</p>
<p>
You probably compile and link your program with a single command,
as in
<pre><code>
g++ myprog.C
</code></pre>
Depending on the computer that you are using, you may be able
to compile an MPI program with a similar command, which automatically
locates the include file and the compiled libraries that you will
need. This command is likely to be:
<pre><code>
mpiCC myprog.c
</code></pre>
</p>
<h3 align = "center">
Interactive MPI Runs
</h3>
<p>
Some systems allow users to run an MPI program interactively.
You do this with the <b>mpirun</b> command:
<pre><code>
mpirun -np 4 a.out
</code></pre>
This command requests that the executable program <i>a.out</i>
be run, <i>right now</i>, using 4 processors.
</p>
<p>
The <b>mpirun</b> command may be a convenience for beginners,
with very small jobs, but this is not the way to go once you
have a large lengthy program to run! Also, what actually happens
can vary from machine to machine. When you ask for 4 processors,
for instance,
<ul>
<li>
in the best case, <b>mpirun</b> automatically finds three other
machines just like the one you are one, copies your program
to them, and starts your program on all four.
</li>
<li>
in a less good case, there are four processors on your current
machine, so the memory is divided up among them and your program
runs;
</li>
<li>
in a worse case, there are less than four processors available,
so, as necessary, one processor will "time share", and run two
or more of your processes alternately.
</li>
</ul>
</p>
<h3 align = "center">
Licensing:
</h3>
<p>
The computer code and data files described and made available on this web page
are distributed under
<a href = "../../txt/gnu_lgpl.txt">the GNU LGPL license.</a>
</p>
<h3 align = "center">
Languages:
</h3>
<p>
<b>MPI</b> examples are available in
<a href = "../../c_src/mpi/mpi.html">a C version</a> and
<a href = "../../cpp_src/mpi/mpi.html">a C++ version</a> and
<a href = "../../f77_src/mpi/mpi.html">a FORTRAN77 version</a> and
<a href = "../../f_src/mpi/mpi.html">a FORTRAN90 version</a>.
</p>
<h3 align = "center">
Related Data and Programs:
</h3>
<p>
<a href = "../../cpp_src/communicator_mpi/communicator_mpi.html">
COMMUNICATOR_MPI</a>,
a C++ program which
creates new communicators involving a subset of initial
set of MPI processes in the default communicator MPI_COMM_WORLD.
</p>
<p>
<a href = "../../cpp_src/heat_mpi/heat_mpi.html">
HEAT_MPI</a>,
a C++ program which
solves the 1D Time Dependent Heat Equation using MPI.
</p>
<p>
<a href = "../../cpp_src/hello_mpi/hello_mpi.html">
HELLO_MPI</a>,
a C++ program which
prints out "Hello, world!" using the MPI parallel programming environment.
</p>
<p>
<a href = "../../examples/moab/moab.html">
MOAB</a>,
examples which
illustrate the use of the MOAB job scheduler for a computer cluster.
</p>
<p>
<a href = "../../cpp_src/mpi_stubs/mpi_stubs.html">
MPI_STUBS</a>,
a C++ library which
contains "stub" MPI routines which allow
a user to compile, load, and possibly run an MPI program on a
serial machine.
</p>
<p>
<a href = "../../cpp_src/multitask_mpi/multitask_mpi.html">
MULTITASK_MPI</a>,
a C++ program which
demonstrates how to "multitask", that is, to execute several unrelated
and distinct tasks simultaneously, using MPI for parallel execution.
</p>
<p>
<a href = "../../cpp_src/poisson_serial/poisson_serial.html">
POISSON_SERIAL</a>,
a C++ program which
computes an approximate solution to the Poisson equation in a rectangle,
and is intended as the starting point for the creation of a parallel version.
</p>
<p>
<a href = "../../cpp_src/prime_mpi/prime_mpi.html">
PRIME_MPI</a>,
a C++ program which
counts the number of primes between 1 and N, using MPI for parallel execution.
</p>
<p>
<a href = "../../c_src/pthreads/pthreads.html">
PTHREADS</a>,
C programs which
illustrate the use of the POSIX thread library to carry out
parallel program execution.
</p>
<p>
<a href = "../../cpp_src/quad_mpi/quad_mpi.html">
QUAD_MPI</a>,
a C++ program which
approximates an integral using a quadrature rule, and carries out the
computation in parallel using MPI.
</p>
<p>
<a href = "../../cpp_src/random_mpi/random_mpi.html">
RANDOM_MPI</a>,
a C++ program which
demonstrates one way to generate the same sequence of random numbers
for both sequential execution and parallel execution under MPI.
</p>
<p>
<a href = "../../cpp_src/ring_mpi/ring_mpi.html">
RING_MPI</a>,
a C++ program which
uses the MPI parallel programming environment, and measures the time
necessary to copy a set of data around a ring of processes.
</p>
<p>
<a href = "../../cpp_src/satisfy_mpi/satisfy_mpi.html">
SATISFY_MPI</a>,
a C++ program which
demonstrates, for a particular circuit, an exhaustive search
for solutions of the circuit satisfiability problem, using MPI to
carry out the calculation in parallel.
</p>
<p>
<a href = "../../cpp_src/search_mpi/search_mpi.html">
SEARCH_MPI</a>,
a C++ program which
searches integers between A and B for a value J such that F(J) = C,
using MPI for parallel execution.
</p>
<p>
<a href = "../../cpp_src/task_division/task_division.html">
TASK_DIVISION</a>,
a C++ library which
implements a simple procedure for smoothly dividing T tasks among
P processors; such a method can be useful in MPI and other parallel
environments, particularly when T is not an exact multiple of P,
and when the processors can be indexed starting from 0 or from 1.
</p>
<p>
<a href = "../../cpp_src/wave_mpi/wave_mpi.html">
WAVE_MPI</a>,
a C++ program which
uses finite differences and MPI to estimate a solution to the
wave equation.
</p>
<h3 align = "center">
Reference:
</h3>
<p>
<ol>
<li>
William Gropp, Steven Huss-Lederman, Andrew Lumsdaine, Ewing Lusk,
Bill Nitzberg, William Saphir, Marc Snir,<br>
MPI: The Complete Reference,<br>
Volume II: The MPI-2 Extensions,<br>
Second Edition,<br>
MIT Press, 1998,<br>
ISBN13: 978-0-262-57123-4,<br>
LC: QA76.642.M65.
</li>
<li>
William Gropp, Ewing Lusk, Anthony Skjellum,<br>
Using MPI: Portable Parallel Programming with the
Message-Passing Interface,<br>
Second Edition,<br>
MIT Press, 1999,<br>
ISBN: 0262571323,<br>
LC: QA76.642.G76.
</li>
<li>
William Gropp, Ewing Lusk, Rajiv Thakur,<br>
Using MPI-2: Advanced Features of the Message-Passing
Interface,<br>
Second Edition,<br>
MIT Press, 1999,<br>
ISBN: 0262571331,<br>
LC: QA76.642.G762.
</li>
<li>
Stan Openshaw, Ian Turton,<br>
High Performance Computing and the Art of Parallel Programming:
an Introduction for Geographers, Social Scientists, and
Engineers,<br>
Routledge, 2000,<br>
ISBN: 0415156920,<br>
LC: QA76.88.O64.
</li>
<li>
Peter Pacheco,<br>
Parallel Programming with MPI,<br>
Morgan Kaufman, 1996,<br>
ISBN: 1558603395,<br>
LC: QA76.642.P3.
</li>
<li>
Sudarshan Raghunathan,<br>
Making a Supercomputer Do What You Want: High Level Tools for
Parallel Programming,<br>
Computing in Science and Engineering,<br>
Volume 8, Number 5, September/October 2006, pages 70-80.
</li>
<li>
Marc Snir, Steve Otto, Steven Huss-Lederman, David Walker,
Jack Dongarra,<br>
MPI: The Complete Reference,<br>
Volume I: The MPI Core,<br>
Second Edition,<br>
MIT Press, 1998,<br>
ISBN: 0-262-69216-3,<br>
LC: QA76.642.M65.
</li>
<li>
Scott Vetter, Yukiya Aoyama, Jun Nakano,<br>
RS/600 SP: Practical MPI Programming,<br>
IBM Redbooks, 1999,<br>
ISBN: 0738413658.
</li>
</ol>
</p>
<h3 align = "center">
Examples and Tests:
</h3>
<p>
<b>BONES</b> passes a vector of real data from one process to
another. It was used as an example in an introductory MPI workshop.
<ul>
<li>
<a href = "bones_mpi.cpp">bones_mpi.cpp</a>,
the source code;
</li>
<li>
<a href = "bones.sh">bones.sh</a>,
a script to compile and run the program using MPIRUN.
</li>
<li>
<a href = "bones_output.txt">bones_output.txt</a>,
the output file;
</li>
<li>
<a href = "bones_fsu.sh">bones_fsu.sh</a>,
a script to compile and run the program on the FSU HPC cluster.
</li>
<li>
<a href = "bones_fsu_output.txt">bones_fsu_output.txt</a>,
the output file;
</li>
</ul>
</p>
<p>
<b>BUFFON</b> demonstrates how parallel Monte Carlo
processes can set up distinct random number streams.
<ul>
<li>
<a href = "buffon_mpi.cpp">buffon_mpi.cpp</a>,
the source code;
</li>
<li>
<a href = "buffon.sh">buffon.sh</a>,
a script to compile and run the program using MPIRUN.
</li>
<li>
<a href = "buffon_output.txt">buffon_output.txt</a>,
the output file;
</li>
<li>
<a href = "buffon_fsu.sh">buffon_fsu.sh</a>,
a script to compile and run the program on the FSU HPC cluster.
</li>
<li>
<a href = "buffon_fsu_output.txt">buffon_fsu_output.txt</a>,
the output file;
</li>
</ul>
</p>
<p>
<b>DAY1</b> works out exercise #3 assigned after day 1 of a
workshop on MPI. The instructions were to have process 1 generate some
integers, send them to process 3 which used some of those values to
generate some real numbers which were then sent
back to process 1.
<ul>
<li>
<a href = "day1_mpi.cpp">day1_mpi.cpp</a>,
the source code;
</li>
<li>
<a href = "day1.sh">day1.sh</a>,
a script to compile and run the program using MPIRUN.
</li>
<li>
<a href = "day1_output.txt">day1_output.txt</a>,
the output file;
</li>
<li>
<a href = "day1_fsu.sh">day1_fsu.sh</a>,
a script to compile and run the program on the FSU HPC cluster.
</li>
<li>
<a href = "day1_fsu_output.txt">day1_fsu_output.txt</a>,
the output file;
</li>
</ul>
</p>
<p>
<b>INTERVALS</b> estimates an integral by dividing an interval
into subintervals, and having the servant processes estimate
the integral over each subinterval.
<ul>
<li>
<a href = "intervals_mpi.cpp">intervals_mpi.cpp</a>,
the source code;
</li>
<li>
<a href = "intervals.sh">intervals.sh</a>,
a script to compile and run the program using MPIRUN.
</li>
<li>
<a href = "intervals_output.txt">intervals_output.txt</a>,
the output file;
</li>
<li>
<a href = "intervals_fsu.sh">intervals_fsu.sh</a>,
a script to compile and run the program on the FSU HPC cluster.
</li>
<li>
<a href = "intervals_fsu_output.txt">intervals_fsu_output.txt</a>,
the output file;
</li>
</ul>
</p>
<p>
<b>MATVEC</b> computes a matrix-vector product c = A * b,
giving each process a copy of the vector b, and using self-scheduling
to let any process have the next row of A to work on when it is ready.
Arrays are allocated dynamically. The "math.h" include file
is needed, as is the run-time math library.
<ul>
<li>
<a href = "matvec_mpi.cpp">matvec_mpi.cpp</a>,
the source code;
</li>
<li>
<a href = "matvec.sh">matvec.sh</a>,
a script to compile and run the program using MPIRUN.
</li>
<li>
<a href = "matvec_output.txt">matvec_output.txt</a>,
the output file;
</li>
<li>
<a href = "matvec_fsu.sh">matvec_fsu.sh</a>,
a script to compile and run the program on the FSU HPC cluster.
</li>
<li>
<a href = "matvec_fsu_output.txt">matvec_fsu_output.txt</a>,
the output file;
</li>
</ul>
</p>
<p>
<b>MONTE CARLO</b> computes PI by the Monte Carlo method, testing
whether points in the unit square are in the unit circle.
<ul>
<li>
<a href = "monte_carlo_mpi.cpp">monte_carlo_mpi.cpp</a>,
the source code;
</li>
<li>
<a href = "monte_carlo.sh">monte_carlo.sh</a>,
a script to compile and run the program using MPIRUN.
</li>
<li>
<a href = "monte_carlo_output.txt">monte_carlo_output.txt</a>,
the output file;
</li>
<li>
<a href = "monte_carlo_fsu.sh">monte_carlo_fsu.sh</a>,
a script to compile and run the program on the FSU HPC cluster.
</li>
<li>
<a href = "monte_carlo_fsu_output.txt">monte_carlo_fsu_output.txt</a>,
the output file;
</li>
</ul>
</p>
<p>
<b>QUADRATURE</b> integrates a function f(x) over an interval;
<ul>
<li>
<a href = "quadrature_mpi.cpp">quadrature_mpi.cpp</a>,
the source code;
</li>
<li>
<a href = "quadrature.sh">quadrature.sh</a>,
a script to compile and run the program using MPIRUN.
</li>
<li>
<a href = "quadrature_output.txt">quadrature_output.txt</a>,
the output file;
</li>
<li>
<a href = "quadrature_fsu.sh">quadrature_fsu.sh</a>,
a script to compile and run the program on the FSU HPC cluster.
</li>
<li>
<a href = "quadrature_fsu_output.txt">quadrature_fsu_output.txt</a>,
the output file;
</li>
</ul>
</p>
<p>
<b>SEARCH</b> searches a list of numbers for all
occurrences of a target value.
<ul>
<li>
<a href = "search_mpi.cpp">search_mpi.cpp</a>,
the source code;
</li>
<li>
<a href = "search.sh">search.sh</a>,
a C-shell script to compile and run the program using MPIRUN.
</li>
<li>
<a href = "search_output.txt">search_output.txt</a>,
the output file;
</li>
<li>
<a href = "search_fsu.sh">search_fsu.sh</a>,
a script to compile and run the program on the FSU HPC cluster.
</li>
<li>
<a href = "search_fsu_output.txt">search_fsu_output.txt</a>,
the output file;
</li>
</ul>
</p>
<p>
<b>SUM</b> adds a list of numbers.
<ul>
<li>
<a href = "sum_mpi.cpp">sum_mpi.cpp</a>,
the source code;
</li>
<li>
<a href = "sum.sh">sum.sh</a>,
a script to compile and run the program using MPIRUN.
</li>
<li>
<a href = "sum_output.txt">sum_output.txt</a>,
the output file;
</li>
<li>
<a href = "sum_fsu.sh">sum_fsu.sh</a>,
a script to compile and run the program on the FSU HPC cluster.
</li>
<li>
<a href = "sum_fsu_output.txt">sum_fsu_output.txt</a>,
the output file;
</li>
</ul>
</p>
<p>
<b>TYPE</b> sets up a user-defined datatype, and sends and
receives data in this form.
<ul>
<li>
<a href = "type_mpi.cpp">type_mpi.cpp</a>,
the source code;
</li>
<li>
<a href = "type.sh">type.sh</a>,
a script to compile and run the program using MPIRUN.
</li>
<li>
<a href = "type_output.txt">type_output.txt</a>,
the output file;
</li>
<li>
<a href = "type_fsu.sh">type_fsu.sh</a>,
a script to compile and run the program on the FSU HPC cluster.
</li>
<li>
<a href = "type_fsu_output.txt">type_fsu_output.txt</a>,
the output file;
</li>
</ul>
</p>
<p>
You can go up one level to <a href = "../cpp_src.html">
the C++ source codes</a>.
</p>
<hr>
<i>
Last revised on 24 October 2011.
</i>
<!-- John Burkardt -->
</body>
</html>