21.1.1. prte

prte — start a PRRTE distributed virtual machine (DVM)

21.1.1.1. SYNOPSIS

prte [options]

21.1.1.2. DESCRIPTION

prte instantiates an instance of the PMIx Reference Runtime Environment (PRRTE) distributed virtual machine (DVM). The prte process itself becomes the DVM controller; it starts a prted(1) daemon on each of the other nodes that make up the DVM, and then waits to be given jobs to run — by prun(1), by any other PMIx tool, or by an application calling PMIx_Spawn. The DVM persists until it is shut down by pterm(1).

The nodes of the DVM are those of the allocation the resource manager granted, if any, narrowed by --host, --hostfile and --default-hostfile; see --hostfile below.

prte does not launch an application itself: an executable given on its command line is an error. Use prterun(1) to start a DVM, run a single job in it, and shut it down again.

All of the text below is also available from the command itself: prte --help lists the options, and prte --help <option> prints the full description of one.

21.1.1.3. DIRECTIVES AND QUALIFIERS

Several options take a value that is a small language of its own: --map-by, --rank-by, --bind-to, --output, --display and --rtos. The words in that value come from a fixed vocabulary, listed with each option below, and are put together the same way for all of them.

Separators. A value is built from directives, each of which may be followed by qualifiers, and any word may carry a value of its own:

  • : separates a directive from its qualifiers, and one qualifier from the next: --map-by package:span:pe=2.

  • , separates one directive from the next, in the options that accept several: --output tag,timestamp or --display map,bind. A directive’s qualifiers follow it with : as usual, so --output tag,file=out:nocopy is the tag directive, then the file directive qualified by nocopy.

  • = separates a word from its value: pe=2, device=gpu, file=out.

A qualifier written after a , instead of a : is therefore not a qualifier of the directive before it. --map-by device=gpu,ndev=2 names a device called gpu,ndev=2, and since no device is named with a comma, it is refused with the spelling that was almost certainly meant:

$ prun --map-by device=gpu,ndev=2 ./a.out
The device named in a mapping request contains a comma:
  Given:  gpu,ndev=2
...
  device=gpu:ndev=2

--rtos takes no qualifiers, and its values are not split at : at all: a time is written with colons (--rtos timeout=1:30:00, one hour thirty minutes).

Abbreviations. Words are case-insensitive, and any of them may be shortened to a prefix that names only that word:

  • --map-by pack is --map-by package; --bind-to hwt is --bind-to hwthread; --map-by core:ov is --map-by core:oversubscribe.

  • A prefix that fits more than one word is refused, and the words it fits are listed, rather than one of them being picked:

    $ prun --bind-to n ./a.out
The --bind-to option was given a word that abbreviates more than one of the
words it accepts:
  Given:    n
  Matches:  none,numa

    So --bind-to n must be written no or nu; --map-by :s must be :sp (span) or :sh (shared); --map-by :i must be :inh (inherit) or :int (interleave); --rank-by s must be sl or sp; and --output ta must be tag, tag-d or tag-f.

  • A word given in full is always that word, even when it is also the start of a longer one: pe=2 is the pe qualifier, not pe-list, and --output tag is tag, not tag-detailed.

  • A word that merely begins with one in the vocabulary is refused, not read as the word it begins with. --map-by nodes, --bind-to cores and --map-by package:spanish are all errors.

Values. Each word takes no value, may take one, or requires one:

  • A word that takes no value refuses one. --map-by core:span=false is an error, not a request for span - to not ask for something, leave it out.

  • A word that requires a value refuses to go without: --map-by core:pe and --rtos timeout are errors, as is pe= with nothing after the =.

  • The directives of --output, --display and --rtos that are simply on or off may be given a truth value, and are on when given bare: --output tag and --output tag=true ask for tagging, and --output tag=false does not.

Examples.

# two GPUs per process; bind each to a core beside them
$ prun -n 2 --map-by device=gpu:ndev=2 --bind-to core ./a.out

# map by package across the nodes, two cpus per process
$ prun -n 8 --map-by package:span:pe=2 ./a.out

# the same, abbreviated
$ prun -n 8 --map-by pack:sp:pe=2 ./a.out

# tag the output, and also write it to files without copying it to
# the terminal
$ prun -n 4 --output tag,dir=/tmp/out:nocopy ./a.out

# show the map in a form a script can parse, and the bindings
$ prun -n 4 --display map:parseable,bind ./a.out

# stop the job if it runs longer than an hour and a half
$ prun -n 4 --rtos timeout=1:30:00 ./a.out

21.1.1.4. OPTIONS

A value may follow its option either as the next argument or after an = (--host a,b or --host=a,b).

General options

21.1.1.4.1. -h | --help [<option>]

Print the list of options, or the full help for the named option.

21.1.1.4.2. -v | --verbose

Enable typical debug options.

21.1.1.4.3. -V | --version

Print version and exit.

Debug options

21.1.1.4.4. --debug-daemons

Debug daemon output enabled. This is a somewhat limited stream of information normally used to simply confirm that the daemons started. Includes leaving the output streams open.

21.1.1.4.5. --debug-daemons-file

Debug daemon output is enabled and all output from the daemons is redirected into files with names of the form:

output-prted-<daemon-nspace>-<nodename>.log

These names avoid conflict on shared file systems. The files are located in the top-level session directory assigned to the DVM.

See the “Session directory” HTML documentation for additional details about the PRRTE session directory.

21.1.1.4.6. --leave-session-attached

Do not discard stdout/stderr of remote PRRTE daemons. The primary use for this option is to ensure that the daemon output streams (i.e., stdout and stderr) remain open after launch, thus allowing the user to see any daemon-generated error messages. Otherwise, the daemon will “daemonize” itself upon launch, thereby closing its output streams.

21.1.1.4.7. --display <directives>

The display command line directive must be accompanied by a comma-delimited list of case-insensitive options indicating what information about the job and/or allocation is to be displayed. The full directive need not be provided — only enough characters are required to uniquely identify the directive. For example, ALL is sufficient to represent the ALLOCATION directive — while MAP can not be used to represent MAP-DEVEL (though MAP-D would suffice).

Supported values include:

  • ALLOCATION displays the detected hosts and slot assignments for this job

  • BINDINGS displays the resulting bindings applied to processes in this job

  • MAP displays the resulting locations assigned to processes in this job

  • MAP-DEVEL displays a more detailed report on the locations assigned to processes in this job that includes local and node ranks, assigned bindings, and other data

  • TOPO[=LIST] displays the topology of each node in the provided semicolon-delimited list of nodes allocated to the job (defaults to all nodes). An empty list (TOPO=) is refused.

  • CPUS[=LIST] displays the available CPUs on the provided semicolon-delimited list of nodes (defaults to all nodes)

The display command line directive can include qualifiers by adding a colon (:) and any combination of one or more of the following (delimited by colons):

  • PARSEABLE directs that the output be provided in a format that is easily parsed by machines. Note that PARSABLE is also accepted as a typical spelling for the qualifier.

  • PHYSICAL directs that the output of the BINDINGS option be displayed using physical (instead of logical) CPU IDs.

Provided qualifiers will apply to all of the display directives unless noted. Note that directives and qualifiers are case-insensitive.

Every directive and qualifier above that asks a yes-or-no question — everything except TOPO and CPUS, which name a list of nodes — may also be given an explicit truth value:

--display map        the directive is requested
--display map=1      the same, said explicitly
--display map=0      the directive is NOT requested

True may also be written T, Y, TRUE, YES or ENABLE, and false F, N, FALSE, NO or DISABLE — case-insensitively, and as whole words (TR is not an abbreviation of TRUE).

A value that is neither true nor false is refused rather than guessed at: the truth test underneath reads anything it does not recognize as false, so map=maybe would otherwise quietly turn the display off.

--display describes the job as a whole: there is no such thing as one app context of an MPMD command line being displayed. It may therefore be written in any app context and applies to all of them. Two app contexts that ask for opposite things are refused, since there is no way to honor both.

21.1.1.4.8. --no-aggregate-help

Do not aggregate help output from multiple processes. PRRTE defaults to aggregating messages generated by its “help” subsystem so that only one is printed out per topic (along with the number of processes that reported the same issue). This is done to avoid users receiving a flood of one-per-process error messages, all containing the identical error report. Setting this option turns off the aggregation, thereby allowing the user to see duplicate reports from multiple processes.

MCA parameters

21.1.1.4.9. --prtemca <key> <value>

Pass a PRRTE MCA parameter.

Syntax: --prtemca <key> <value>, where key is the parameter name and value is the parameter value.

21.1.1.4.10. --pmixmca <key> <value>

Pass a PMIx MCA parameter

Syntax: --pmixmca <key> <value>, where key is the parameter name and value is the parameter value.

21.1.1.4.11. --tune <files>

Comma-delimited list of one or more files containing MCA parameters for tuning DVM and/or application operations. The option may be given more than once. A file named by a relative path is looked for first in the current directory and then among the parameter sets installed with PRRTE.

Syntax in the file is:

param = value

with one parameter per line. Empty lines and lines beginning with the # character are ignored, as is any whitespace around the = character. Quotes around the value are removed.

Each parameter is a generic MCA parameter, so it is treated exactly like --mca param value: it is applied to PRRTE if it belongs to a PRRTE framework, and otherwise to PMIx. A parameter that belongs to neither is an error, as is a parameter given twice with different values. A parameter given explicitly on the command line (--prtemca, --pmixmca) overrides the same parameter in a tune file.

DVM options

21.1.1.4.12. -H | --host <hosts>

Host syntax consists of a comma-delimited list of node names, each entry optionally containing a :N extension indicating the number of slots to assign to that entry:

--host node01:5,node02

In the absence of the slot extension, one slot will be assigned to the node. Duplicate entries are aggregated and the number of slots assigned to that node are summed together.

Note

A “slot” is the PRRTE term for an allocatable unit where we can launch a process. Thus, the number of slots equates to the maximum number of processes PRRTE may start on that node without oversubscribing it.

Given to a job, --host selects from the hosts already available to the DVM — those a resource manager allocated, or those the DVM was started with. It does not add any: naming a host that is not among them is an error. Use --add-host or --add-hostfile to bring a new host into a running DVM, or --activate to start a daemon on a host the allocation already contains.

The :N count applies to placement, and not merely to the size of the job: it is the number of processes that may be placed on that host, whatever mapping policy is in effect. Asking for more slots on a host than it has is an error under a resource manager, which decided how big the host is. Without one, the larger count is taken as the size of the host for that job only; the allocation itself is unchanged.

See the “Host specification” documentation for details about the format and content of hostfiles.

21.1.1.4.13. --hostfile <filename>

PRRTE supports several levels of user-specified host lists based on an established precedence order. Users can specify a default hostfile that contains a list of nodes to be used by the DVM. Only one default hostfile can be provided for a given DVM. In addition, users can specify a hostfile that contains a list of nodes to be used for a DVM, or can provide a comma-delimited list of nodes to be used for that DVM via the --host command line option.

The precedence order applied to these various options depends to some extent on the local environment. The following table illustrates how host and hostfile directives work together to define the set of hosts upon which a DVM will execute in the absence of a resource manager (RM):

Default hostfile

host

hostfile

Result

unset

unset

unset
The DVM will consist solely of the
local host where the DVM
was started.

unset

set

unset
Host option defines resource list for the DVM.

unset

unset

set
Hostfile option defines resource list for the DVM.

unset

set

set
Hostfile option defines resource list for the DVM,
then host filters the list to define the final
set of nodes to be used by the DVM

set

unset

unset
Default hostfile defines resource list for the DVM

set

set

unset
Default hostfile defines resource list for the DVM,
then host filters the list to define the final
set of nodes to be used by the DVM

set

set

set
Default hostfile defines resource list for the DVM,
then hostfile filters the list, and then host filters
the list to define the final set of nodes to be
used by the DVM

This changes somewhat in the presence of an RM as that entity specifies the initial allocation of nodes. In this case, the default hostfile, hostfile and host directives are all used to filter the RM’s specification so that a user can utilize different portions of the allocation for different DVMs. This is done according to the same precedence order as in the prior table, with the RM providing the initial pool of nodes.

Hostfiles (sometimes called “machine files”) are a combination of two things:

  1. A listing of hosts on which to launch processes.

  2. Optionally, limit the number of processes which can be launched on each host.

Hostfile syntax consists of one node name on each line, optionally including a designated number of “slots”:

# This is a comment line, and will be ignored
node01  slots=10
node13  slots=5

node15
node16
node17  slots=3
...

Blank lines and lines beginning with a # are ignored.

A node name may carry the account PRRTE is to use when reaching that node, written in front of it and separated by a single @:

user01@node01  slots=4

An entry may contain at most one @, and both the account and the node name must be given: a second @, or an @ with nothing on one side of it, is reported as a parse error naming the hostfile and the line it is on.

A node name written with a leading ^ is excluded rather than used. The ^ goes in front of the whole entry, account included:

node01  slots=4
node02  slots=4
node03  slots=4
^user01@node02

Where the hostfile also names nodes, an exclusion takes a node back out of what it named — the file above names node01 and node03. A hostfile given to a job in a running DVM may consist of nothing but exclusions, and then it selects every node of the allocation except the ones it excludes.

A “slot” is the PRRTE term for an allocatable unit where we can launch a process. See the section on definition of the term slot for a longer description of slots.

In the absence of the slot parameter, PRRTE will assign either the number of slots to be the number of CPUs detected on the node or the resource manager-assigned value if operating in the presence of an RM.

Important

If using a resource manager, the user-specified number of slots is capped by the RM-assigned value.

A hostfile given to a job that is being submitted to an already-running DVM selects within the DVM’s allocation: it names the subset of nodes that job may use, and a slots count smaller than the node’s own is the number of slots that job may take there. It says nothing about how big the node is, so it applies to that job alone — the node is back to its allocated size for the next job, which may be someone else’s. Changing the allocation is what --add-hostfile is for.

21.1.1.4.14. --default-hostfile <filename>

Specify a default hostfile.

Also see --hostfile.

21.1.1.4.15. --uniform-nodes

The uniform-nodes command line directive is used to indicate that the allocated nodes should be treated as having only one topology, so optimize the launch for that scenario. This includes ensuring that all CPU allocations are the same on each node, that each node contains the same number of devices and topological layers, etc.

Note

The runtime does not currently support mixes of chips with different endianness.

21.1.1.4.16. --rtos <directives> | --runtime-options <directives>

Given to prte, these runtime options apply to the DVM itself — for example, --rtos show-progress reports progress while the DVM’s daemons start, which is useful on large systems.

The --rtos command line directive must be accompanied by a comma-delimited list of case-insensitive options that control the runtime behavior of the job. The full directive need not be provided — only enough characters are required to uniquely identify the directive.

Runtime options are typically true or false, though this is not a requirement on developers. Since the value of each option may need to be set (e.g., to override a default set by MCA parameter), the syntax of the command line directive includes the use of an = character to allow inclusion of a value for the option. For example, one can set the ERROR-NONZERO-STATUS option to true by specifying it as ERROR-NONZERO-STATUS=1. A boolean option can be set to true using a non-zero integer, the single letter T or Y, or the whole word TRUE, YES or ENABLE; and to false using zero, the single letter F or N, or the whole word FALSE, NO or DISABLE. All of these are case-insensitive. Note that these are the whole words — TR is not an abbreviation of TRUE — and that a value which is neither true nor false is refused rather than guessed at.

Note that a boolean option will default to true if provided without a value. Thus, --rtos error-nonzero is sufficient to set the ERROR-NONZERO-STATUS option to true.

The --runtime-options command line directive is accepted as a synonym for --rtos.

Supported values include:

  • ERROR-NONZERO-STATUS[=(bool)]: if set to false, this directs the runtime to treat a process that exits with non-zero status as a normal termination. If set to true, the runtime will consider such an occurrence as an error termination and take appropriate action — i.e., the job will be terminated unless a runtime option directs otherwise. This option defaults to a true value if the option is given without a value.

  • DONOTLAUNCH: directs the runtime to map but not launch the specified job. This is provided to help explore possible process placement patterns before actually starting execution. No value need be passed as this is not an option that can be set by default in PRRTE.

  • DONOTSPAWN: directs the runtime to carry out the entire launch procedure — including starting any daemons it needs and delivering the job to them — but to not actually start the application processes, which are instead marked as having terminated. This is provided to help exercise the launch procedure itself.

  • SHOW-PROGRESS[=(bool)]: requests that the runtime provide progress reports on its startup procedure — i.e., the launch of its daemons in support of a job. This is typically used to debug DVM startup on large systems. This option defaults to a true value if the option is given without a value.

  • NOTIFYERRORS[=(bool)]: if set to true, requests that the runtime provide a PMIx event whenever a job encounters an error — e.g., a process fails. The event is to be delivered to each remaining process in the job. This option defaults to a true value if the option is given without a value. See --help notifications for more detail as to the PMIx event codes available for capturing failure events.

  • RECOVERABLE[=(bool)]: if set to true, this indicates that the application wishes to consider the job as recoverable — i.e., the application is assuming responsibility for recovering from any process failure. This could include application-driven spawn of a substitute process or internal compensation for the missing process. This option defaults to a true value if the option is given without a value.

  • AUTORESTART[=(bool)]: if set to true, this requests that the runtime automatically restart failed processes up to “max restarts” number of times. This option defaults to a true value if the option is given without a value.

  • CONTINUOUS[=(bool)]: if set to true, this informs the runtime that the processes in this job are to run until explicitly terminated. Processes that fail are to be automatically restarted up to “max restarts” number of times. Notification of process failure is to be delivered to all processes in the application. This is the equivalent of specifying RECOVERABLE, NOTIFYERRORS, and AUTORESTART options except that the runtime, not the application, assumes responsibility for process recovery. This option defaults to a true value if the option is given without a value.

  • MAX-RESTARTS=<int>: indicates the maximum number of times a given process is to be restarted. This can be set at the application or job level (which will then apply to all applications in that job).

  • EXEC-AGENT=<path> indicates the executable that shall be used to start an application process. The resulting command for starting an application process will be <path> app <app-argv>. The path may contain its own command line arguments.

  • DEFAULT-EXEC-AGENT: directs the runtime to use the system default exec agent to start an application process. No value need be passed as this is not an option that can be set by default in PRRTE.

  • OUTPUT-PROCTABLE[(=channel)]: directs the runtime to report the conventional debugger process table (includes PID and host location of each process in the application). Output is directed to stdout if the channel is -, stderr if +, or into the specified file otherwise. If no channel is specified, output will be directed to stdout.

  • STOP-ON-EXEC: directs the runtime to stop the application process(es) immediately upon exec’ing them. The directive will apply to all processes in the job.

  • STOP-IN-INIT: indicates that the runtime should direct the application process(es) to stop in PMIx_Init(). The directive will apply to all processes in the job.

  • STOP-IN-APP[=<breakpoint>]: indicates that the runtime should direct application processes to stop at some application-defined place and notify they are ready-to-debug. The directive will apply to all processes in the job. Given without a value, the processes stop at whichever such place they reach first. Given a value, that string is the identifier of the one breakpoint at which they are to stop: the runtime passes it to the application in the PMIX_BREAKPOINT environment variable and then waits for the application to report itself ready for debug, so it is up to the application to recognize the name and stop in the corresponding place. This is the one directive whose value may be something other than a truth value, and it is read as a truth value whenever it spells one — so a breakpoint cannot be named true, false, or any other spelling of a boolean.

  • TIMEOUT=<string>: directs the runtime to terminate the job after it has executed for the specified time. Time is specified in colon-delimited format — e.g., 01:20:13:05 to indicate 1 day, 20 hours, 13 minutes and 5 seconds. Time specified without colons will be assumed to have been given in seconds.

  • SPAWN-TIMEOUT=<string>: directs the runtime to terminate the job if job launch is not completed within the specified time. Time is specified in colon-delimited format — e.g., 01:20:13:05 to indicate 1 day, 20 hours, 13 minutes and 5 seconds. Time specified without colons will be assumed to have been given in seconds.

  • REPORT-STATE-ON-TIMEOUT[(=bool)]: directs the runtime to provide a detailed report on job and application process state upon job timeout. This option defaults to a true value if the option is given without a value.

  • GET-STACK-TRACES[(=bool)]: requests that the runtime provide stack traces on all application processes still executing upon timeout. This option defaults to a true value if the option is given without a value.

  • REPORT-CHILD-JOBS-SEPARATELY[(=bool)]: directs the runtime to report the exit status of any child jobs spawned by the primary job separately. If false, then the final exit status reported will be zero if the primary job and all spawned jobs exit normally, or the first non-zero status returned by either primary or child jobs. This option defaults to a true value if the option is given without a value.

  • AGGREGATE-HELP-MESSAGES[(=bool)]: directs the runtime to aggregate help messages, reporting each unique help message once accompanied by the number of processes that reported it. This option defaults to a true value if the option is given without a value.

  • FWD-ENVIRONMENT[(=bool)]: directs the runtime to forward the entire local environment in support of the application. This option defaults to a true value if the option is given without a value.

The --rtos command line option has no qualifiers.

Note

Directives are case-insensitive. FWD-ENVIRONMENT is the same as fwd-environment.

A value that is neither true nor false is refused rather than guessed at: the truth test underneath reads anything it does not recognize as false, so donotlaunch=maybe would otherwise quietly launch.

--rtos describes the job as a whole — there is no such thing as one app context of an MPMD command line not launching — so it may be written in any app context and applies to all of them. Two app contexts that ask for opposite things are refused, since there is no way to honor both.

21.1.1.4.17. --timeout <seconds>

Timeout DVM startup if time exceeds the specified number of seconds. The DVM startup will abort after the specified interval.

21.1.1.4.18. --daemonize

Daemonize the DVM daemons and controller into the background.

21.1.1.4.19. --no-ready-msg

Do not print a DVM ready message.

21.1.1.4.20. --system-server

Start the DVM controller and its daemons as the system server on their nodes.

21.1.1.4.21. --set-sid

Direct the DVM (controller and daemons) to separate from the current session.

21.1.1.4.22. --report-pid <arg>

Print the PID of this process: on stdout if the argument is -, on stderr if it is +, into the already-open file descriptor it names if it is a non-negative integer, and otherwise into the file it names.

21.1.1.4.23. --report-uri <arg>

Print the PMIx contact URI of this process: on stdout if the argument is -, on stderr if it is +, and otherwise into the file it names.

21.1.1.4.24. --keepalive <fd>

Pipe for DVM controller to monitor — DVM will terminate upon closure.

21.1.1.4.25. --singleton <id>

DVM is being started by a singleton process (i.e., one not started by a DVM) — the argument must be the PMIx ID of the singleton process that started us.

21.1.1.4.26. --launch-agent <executable>

Name of daemon executable used to start processes on remote nodes (default: prted). This is the executable the DVM controller shall start on each remote node when establishing the DVM.

21.1.1.4.27. --max-vm-size <size>

Maximum number of daemons to start — sets the maximum size of the DVM.

21.1.1.4.28. --prefix <dir>

Prefix to be used to look for PRRTE executables. PRRTE automatically sets the prefix for remote daemons if it was either configured with the --enable-prte-prefix-by-default option OR prte itself was executed with an absolute path to the prte command. This option overrides those settings, if present, and forces use of the provided path.

21.1.1.4.29. --noprefix

Disable automatic --prefix behavior. PRRTE automatically sets the prefix for remote daemons if it was either configured with the --enable-prte-prefix-by-default option OR prte itself was executed with an absolute path to the prte command. This option disables that behavior.

21.1.1.4.30. --pmix-prefix <dir>

Prefix to be used by a PRRTE executable to look for its PMIx installation on remote nodes. This is the location of the top-level directory for the installation. If the installation has not been moved, it would be the value given to “–prefix” when the installation was configured.

Note that PRRTE cannot determine the exact name of the library subdirectory under this location. For example, some systems will call it “lib” while others call it “lib64”. Accordingly, PRRTE will use the library subdirectory name of the PMIx installation used to build PRRTE.

21.1.1.4.31. --exec-agent <path>

Executable to be used to start an application process. The resulting command for starting an application process will be <path> app <app-argv>. The path may contain its own command line arguments.

Note that this can also be given as a runtime option (see --rtos), and that --rtos default-exec-agent returns the job to the system default agent.

21.1.1.4.32. -x <name>[=<value>]

Export an environment variable, optionally specifying a value. For example:

  • -x foo exports the environment variable foo and takes its value from the current environment.

  • -x foo=bar exports the environment variable name foo and sets its value to bar in the started processes.

  • -x foo* exports all current environmental variables starting with foo.

21.1.1.4.33. --forward-signals <signals>

Comma-delimited list of the signals (names or integers) to be forwarded to application processes (none = forward nothing).

The list replaces the default set rather than adding to it, so it names every signal that is to be forwarded. The default set, used when this option is not given, is SIGTSTP, SIGUSR1, SIGUSR2, SIGABRT, SIGALRM, and SIGCONT.

21.1.1.4.34. --allow-run-as-root

Allow execution as root (STRONGLY DISCOURAGED).

Running as root exposes the user to potentially catastrophic file system corruption and damage — e.g., if the user accidentally points the root of the session directory to a system required point, this directory and all underlying elements will be deleted upon job completion, thereby rendering the system inoperable.

It is recognized that some environments (e.g., containers) may require operation as root, and that the user accepts the risks in those scenarios. Accordingly, one can override PRRTE’s run-as-root protection by providing one of the following:

  • The --allow-run-as-root command line directive

  • Adding BOTH of the following environmental parameters:

    • PRTE_ALLOW_RUN_AS_ROOT=1

    • PRTE_ALLOW_RUN_AS_ROOT_CONFIRM=1

Again, we recommend this only be done if absolutely necessary.

21.1.1.5. DEPRECATED COMMAND LINE OPTIONS

The following options are still accepted, but will be removed in a future release; use the replacement shown.

--machinefile <filename>

Synonym for --hostfile.

--show-progress

Replaced by --rtos show-progress.

--hetero-nodes

Has no effect beyond a warning: heterogeneous nodes are detected automatically. Give --uniform-nodes if the nodes are known to be identical.

--debug

Has no effect; it is accepted, with a warning, so that old command lines still parse.

21.1.1.6. ENVIRONMENT

PRTE_ALLOW_RUN_AS_ROOT, PRTE_ALLOW_RUN_AS_ROOT_CONFIRM

When both are set to 1, permit execution as root — see --allow-run-as-root.

PRTE_MCA_<name>, PMIX_MCA_<name>

Set the PRRTE or PMIx MCA parameter <name>, as --prtemca and --pmixmca do on the command line.

21.1.1.7. EXIT STATUS

prte exits with status 0 when the DVM is shut down normally (for example, by pterm(1)), and non-zero if the DVM could not be started or terminated abnormally.

21.1.1.8. EXAMPLES

Start a DVM on the hosts listed in a hostfile, in the background, and write the controller’s URI to a file that tools can use to find it:

shell$ prte --hostfile myhosts --report-uri dvm.uri --daemonize

Start a DVM whose jobs default to mapping without launching, so placement can be explored with prun(1):

shell$ prte --rtos donotlaunch --daemonize
shell$ prun --display map -n 8 ./a.out

See also

prun(1), prterun(1), pterm(1), prted(1), prte_info(1), prte(5)