@@ -262,7 +262,7 @@ To map processes:
262262
263263* ``--bynode ``: Launch processes one per node, cycling by node in a
264264 round-robin fashion. This spreads processes evenly among nodes and
265- assigns MPI_COMM_WORLD ranks in a round-robin, "by node" manner.
265+ assigns `` MPI_COMM_WORLD `` ranks in a round-robin, "by node" manner.
266266 (deprecated in favor of ``--map-by node ``)
267267
268268* ``--cpu-list <cpus> ``: Comma-delimited list of processor IDs to
@@ -272,7 +272,7 @@ To map processes:
272272 .. note :: You can run Run the hwloc ``lstopo(1)`` command to see a
273273 list of available cores and their logical IDs.
274274
275- To order processes' ranks in MPI_COMM_WORLD:
275+ To order processes' ranks in `` MPI_COMM_WORLD `` :
276276
277277* ``--rank-by <mode> ``: Rank in round-robin fashion according to the
278278 specified mode, defaults to slot. Supported options include
@@ -311,15 +311,15 @@ To manage standard I/O:
311311 specified filename. Any directories in the filename will
312312 automatically be created. Each output file will consist of
313313 ``filename.id ``, where the ``id `` will be the processes' rank in
314- MPI_COMM_WORLD, left-filled with zero's for correct ordering in
314+ `` MPI_COMM_WORLD `` , left-filled with zero's for correct ordering in
315315 listings. A relative path value will be converted to an absolute
316316 path based on the cwd where mpirun is executed. Note that this will
317317 not work on environments where the file system on compute nodes
318318 differs from that where :ref: `mpirun(1) <man1-mpirun >` is
319319 executed.
320320
321- * ``--stdin <rank> ``: The MPI_COMM_WORLD rank of the process that is
322- to receive stdin. The default is to forward stdin to MPI_COMM_WORLD
321+ * ``--stdin <rank> ``: The `` MPI_COMM_WORLD `` rank of the process that is
322+ to receive stdin. The default is to forward stdin to `` MPI_COMM_WORLD ``
323323 rank 0, but this option can be used to forward stdin to any
324324 process. It is also acceptable to specify none, indicating that no
325325 processes are to receive stdin.
@@ -329,7 +329,7 @@ To manage standard I/O:
329329
330330* ``--tag-output ``: Tag each line of output to stdout, stderr, and
331331 stddiag with ``[jobid, MCW_rank]<stdxxx> `` indicating the process
332- jobid and MPI_COMM_WORLD rank of the process that generated the
332+ jobid and `` MPI_COMM_WORLD `` rank of the process that generated the
333333 output, and the channel which generated it.
334334
335335* ``--timestamp-output ``: Timestamp each line of output to stdout,
@@ -342,7 +342,7 @@ To manage standard I/O:
342342 specified file.
343343
344344* ``--xterm <ranks> ``: Display the output from the processes
345- identified by their MPI_COMM_WORLD ranks in separate xterm
345+ identified by their `` MPI_COMM_WORLD `` ranks in separate xterm
346346 windows. The ranks are specified as a comma-separated list of
347347 ranges, with a ``-1 `` indicating all. A separate window will be created
348348 for each specified process.
@@ -422,7 +422,22 @@ Setting MCA parameters:
422422
423423* ``--mca <key> <value> ``: Send arguments to various MCA modules. See
424424 the :ref: `Setting MCA Parameters
425- <man1-mpirun-setting-mca-parameters>` section for mode details.
425+ <man1-mpirun-setting-mca-parameters>` section for more details.
426+
427+ .. note :: Open MPI will attempt to discern PMIx and PRRTE MCA
428+ parameters passed via ``--mca `` and handle them
429+ appropriately, but it may not always guess correctly. It
430+ is best to use ``--pmixmca `` and ``--prtemca `` when
431+ passing MCA parammeters to PMIx and PRRTE, respectively.
432+
433+ * ``--pmixmca <key> <value> ``: Send arguments to MCA modules in the
434+ PMIx subsystem. See the :ref: `Setting MCA Parameters
435+ <man1-mpirun-setting-mca-parameters>` section for more details.
436+
437+ * ``--prtemca <key> <value> ``: Send arguments to MCA modules in the
438+ PMIx Reference Runtime Environment (PRRTE) subsystem. See the
439+ :ref: `Setting MCA Parameters <man1-mpirun-setting-mca-parameters >`
440+ section for more details.
426441
427442* ``--tune <tune_file> ``: Specify a tune file to set arguments for
428443 various MCA modules and environment variables. See the :ref: `
@@ -532,11 +547,11 @@ generally useful to most Open MPI users:
532547 just before launch.
533548
534549* ``--launch-agent ``: Name of the executable that is to be used to
535- start processes on the remote nodes. The default is ``PRRTEd ``. This
550+ start processes on the remote nodes. The default is ``prted ``. This
536551 option can be used to test new daemon concepts, or to pass options
537552 back to the daemons without having mpirun itself see them. For
538- example, specifying a launch agent of ``PRRTEd -mca odls_base_verbose
539- 5 `` allows the developer to ask the ``PRRTEd `` for debugging output
553+ example, specifying a launch agent of ``prted --prtemca odls_base_verbose
554+ 5 `` allows the developer to ask the ``prted `` for debugging output
540555 without clutter from ``mpirun `` itself.
541556
542557* ``--report-state-on-timeout ``: When paired with the ``--timeout ``
@@ -716,7 +731,7 @@ options that describe mapping policies.
716731
717732Consider the same hostfile as above, again with ``-n 6 ``. The table
718733below lists a few ``mpirun `` variations, and shows which
719- MPI_COMM_WORLD ranks end up on which node:
734+ `` MPI_COMM_WORLD `` ranks end up on which node:
720735
721736.. list-table ::
722737 :header-rows: 1
@@ -832,7 +847,7 @@ Open MPI employs a three-phase procedure for assigning process locations
832847and ranks:
833848
834849#. **Mapping **: Assigns a default location to each process
835- #. **Ranking **: Assigns an MPI_COMM_WORLD rank value to each process
850+ #. **Ranking **: Assigns an `` MPI_COMM_WORLD `` rank value to each process
836851#. **Binding **: Constrains each process to run on specific processors
837852
838853The mapping step is used to assign a default location to each process
@@ -864,7 +879,7 @@ gives you detailed control over process binding as well. Rankfiles
864879are discussed :ref: `below <man1-mpirun-rankfiles >`.
865880
866881The second phase focuses on the ranking of the process within the
867- job's MPI_COMM_WORLD. Open MPI separates this from the mapping
882+ job's `` MPI_COMM_WORLD `` . Open MPI separates this from the mapping
868883procedure to allow more flexibility in the relative placement of MPI
869884processes. This is best illustrated by considering the following
870885cases where we used the ``--np 8 --map-by ppr:2:package --host aa:4,bb:4 `` option:
@@ -1013,10 +1028,12 @@ MCA parameters can be set not only on the mpirun command line, but
10131028alternatively in a system or user ``mca-params.conf `` file or as
10141029environment variables, as described in the :ref: `Setting MCA
10151030Parameters <man1-mpirun-setting-mca-parameters>`. These are MCA parameters for
1016- the PRRTE runtime so the command line argument ``--PRRTEmca `` must be used to
1017- pass the MCA parameter key/value pair. Alternatively, the MCA parameter key/
1018- value pair may be specific on the command line by prefixing the key with
1019- ``PRRTE_MCA_ ``. Some examples include:
1031+ the PRRTE runtime so the command line argument ``--prtemca ``
1032+ (yes, ``prte `` with a single ``r ``, not two ``r ``'s) must be used to
1033+ pass the MCA parameter key/value pair. Alternatively, the MCA parameter
1034+ key/value pair may be specific on the command line by prefixing the key with
1035+ ``PRTE_MCA_ `` (again, that is not a typo: ``PRTE `` not ``PRRTE ``).
1036+ Some examples include:
10201037
10211038.. list-table ::
10221039 :header-rows: 1
@@ -1166,7 +1183,7 @@ Rankfiles are text files that specify detailed information about how
11661183individual processes should be mapped to nodes, and to which
11671184processor(s) they should be bound. Each line of a rankfile specifies
11681185the location of one process (for MPI jobs, the process' "rank" refers
1169- to its rank in MPI_COMM_WORLD). The general form of each line in the
1186+ to its rank in `` MPI_COMM_WORLD `` ). The general form of each line in the
11701187rankfile is:
11711188
11721189.. code ::
@@ -1299,11 +1316,11 @@ Standard I/O
12991316 retained, or removed?
13001317
13011318Open MPI directs UNIX standard input to ``/dev/null `` on all processes
1302- except the MPI_COMM_WORLD rank 0 process. The MPI_COMM_WORLD rank 0
1319+ except the `` MPI_COMM_WORLD `` rank 0 process. The `` MPI_COMM_WORLD `` rank 0
13031320process inherits standard input from ``mpirun ``.
13041321
13051322.. note :: The node that invoked ``mpirun`` need not be the same as the
1306- node where the MPI_COMM_WORLD rank 0 process resides. Open
1323+ node where the `` MPI_COMM_WORLD `` rank 0 process resides. Open
13071324 MPI handles the redirection of ``mpirun ``'s standard input
13081325 to the rank 0 process.
13091326
@@ -1320,7 +1337,7 @@ example:
13201337
13211338 shell$ mpirun -n 2 my_app < my_input > my_output
13221339
1323- Note that in this example only the MPI_COMM_WORLD rank 0 process will
1340+ Note that in this example only the `` MPI_COMM_WORLD `` rank 0 process will
13241341receive the stream from ``my_input `` on stdin. The stdin on all the other
13251342nodes will be tied to ``/dev/null ``. However, the stdout from all nodes
13261343will be collected into the ``my_output `` file.
@@ -1643,15 +1660,15 @@ that job are designated "secondary" jobs):
16431660
16441661* If one or more processes in the primary job normally terminate with
16451662 non-zero exit status, ``mpirun `` returns the exit status of the
1646- process with the lowest MPI_COMM_WORLD rank to have a non-zero
1663+ process with the lowest `` MPI_COMM_WORLD `` rank to have a non-zero
16471664 status.
16481665
16491666* If all processes in the primary job normally terminate with exit
16501667 status 0, and one or more processes in a secondary job normally
16511668 terminate with non-zero exit status, ``mpirun ``:
16521669
16531670 #. Returns the exit status of the process with the lowest
1654- MPI_COMM_WORLD rank in the lowest jobid to have a non-zero
1671+ `` MPI_COMM_WORLD `` rank in the lowest jobid to have a non-zero
16551672 status, and
16561673 #. Outputs a message summarizing the exit status of the primary and
16571674 all secondary jobs.
@@ -1662,7 +1679,7 @@ that job are designated "secondary" jobs):
16621679 summary print statement.
16631680
16641681By default, the job will abort when any process terminates with
1665- non-zero status. The MCA parameter ``--PRRTEmca state_base_error_non_zero_exit ``
1682+ non-zero status. The MCA parameter ``--prtemca state_base_error_non_zero_exit ``
16661683can be set to "false" (or "0") to cause Open MPI to not abort a job if
16671684one or more processes return a non-zero status. In that situation the
16681685Open MPI records and notes that processes exited with non-zero
@@ -1705,7 +1722,7 @@ processes exited before calling :ref:`MPI_FINALIZE(3) <mpi_finalize>`.
17051722If an internal error occurred in mpirun, the corresponding error code
17061723is returned. In the event that one or more processes exit before
17071724calling :ref: `MPI_FINALIZE(3) <mpi_finalize >`, the return value of
1708- the MPI_COMM_WORLD rank of the process that mpirun first notices died
1725+ the `` MPI_COMM_WORLD `` rank of the process that mpirun first notices died
17091726before calling :ref: `MPI_FINALIZE(3) <mpi_finalize >` will be
17101727returned. Note that, in general, this will be the first process that
17111728died but is not guaranteed to be so.
0 commit comments