PetscDeviceContextJoin#

Converge a set of child contexts

Synopsis#

#include <petscdevice.h> 
PetscErrorCode PetscDeviceContextJoin(PetscDeviceContext dctx, PetscInt n, PetscDeviceContextJoinMode joinMode, PetscDeviceContext **dsub)

Not Collective, Asynchronous

Input Parameters#

dctx - A PetscDeviceContext to converge on
n - The number of sub contexts to converge
joinMode - The type of join to perform
dsub - The sub contexts to converge

Notes#

If PetscDeviceContextFork() creates n edges from a source node which all depend on the source node, then this routine is the exact mirror. That is, it creates a node (represented in dctx) which receives n edges (and optionally destroys them) which is dependent on the completion of all incoming edges.

If joinMode is PETSC_DEVICE_CONTEXT_JOIN_DESTROY. All contexts in dsub will be destroyed by this routine. Thus all sub contexts must have been created with the dctx passed to this routine.

If joinMode is PETSC_DEVICE_CONTEXT_JOIN_SYNC. All sub contexts will additionally wait on dctx after converging. This has the effect of “synchronizing” the outgoing edges. Note the sync suffix does NOT refer to the host, i.e. this routine does NOT call PetscDeviceSynchronize().

If joinMode is PETSC_DEVICE_CONTEXT_JOIN_NO_SYNC. dctx waits for all sub contexts but the sub contexts do not wait for one another or dctx afterwards.

DAG representations#

If joinMode is PETSC_DEVICE_CONTEXT_JOIN_DESTROY

  time ->

  -> dctx ---------/- |= CALL =| - dctx ->
  -> dsub[0] -----/
  ->  ... -------/
  -> dsub[n-1] -/

If joinMode is PETSC_DEVICE_CONTEXT_JOIN_SYNC

  time ->

  -> dctx ---------/- |= CALL =| -\----> dctx ------>
  -> dsub[0] -----/                \---> dsub[0] --->
  ->  ... -------/                  \--> ... ------->
  -> dsub[n-1] -/                    \-> dsub[n-1] ->

If joinMode is PETSC_DEVICE_CONTEXT_JOIN_NO_SYNC

  time ->

  -> dctx ----------/- |= CALL =| - dctx ->
  -> dsub[0] ------/----------------------->
  ->  ... --------/------------------------>
  -> dsub[n-1] --/------------------------->

Asynchronous API Notes#

This routine is explicitly marked as exhibiting asynchronous behavior. Asynchronous behavior implies that routines launching operations on (or associated with) a PetscDeviceContext may return to the caller before the operation has completed.

Sequential Consistency:

Operations using the same PetscDeviceContext which access objects or memory regions are ordered per the language specification.

Operations using separate PetscDeviceContexts which access the same object or memory region are strongly write-ordered. That is, the following operations:

write-write
write-read
read-write

are strongly ordered. Formally:

Given an operation A-B (e.g. A = write, B = read) on an object or memory region M such that A “happens-before” B, where A uses PetscDeviceContext X and B uses PetscDeviceContext Y, then B shall not begin before A completes. This implies that any side-effects resulting from A are also observed by B.

Note the omission of read-read; there is no implied ordering between separate PetscDeviceContexts for consecutive reads.

Operations using separate PetscDeviceContexts which access separate objects or memory regions may execute in an arbitrary order and offer no guarantee of sequential consistency.

Memory Consistency:

If this routine modifies the participating object(s) then – unless otherwise stated – the contents of any externally held references to internal data structures should be considered to be in an undefined state. A well-defined state can only be restored by re-acquiring these references through the appropriate API or by calling PetscDeviceContextSynchronize().

Unless otherwise stated, exceptions to this rule are:

References returned by the routine itself. If a routine returns a pointer, the value of the top-most pointer is guaranteed to always be valid. For example, given a routine which asynchronously allocates memory and returns a pointer to the memory, the value of said pointer is immediately valid but dereferencing the pointer may not be.
References to structures. If a routine returns a PetscFoo, or array thereof then the objects themselves are always valid (though their member variables PetscFoo->data may not be).