PetscSFConcatenate#
concatenate multiple PetscSF
into one
Synopsis#
#include "petscsf.h"
PetscErrorCode PetscSFConcatenate(MPI_Comm comm, PetscInt nsfs, PetscSF sfs[], PetscSFConcatenateRootMode rootMode, PetscInt leafOffsets[], PetscSF *newsf)
Input Parameters#
Output Parameters#
newsf - The resulting
PetscSF
Notes#
The communicator of all SFs in sfs must be comm.
Leaves are always concatenated locally, keeping them ordered by the input SF index and original local order. The offsets in leafOffsets are added to the original leaf indices. If all input SFs use contiguous leaf storage (ilocal = NULL), leafOffsets can be passed as NULL as well. In this case, NULL is also passed as ilocal to the resulting SF. If any input SF has non-null ilocal, leafOffsets is needed to distinguish leaves from different input SFs. In this case, user is responsible to provide correct offsets so that the resulting leaves are unique (otherwise an error occurs).
All root modes retain the essential connectivity condition#
If two leaves of the same input SF are connected (sharing the same root), they are also connected in the output SF.
Parameter rootMode controls how the input root spaces are combined.
For PETSCSF_CONCATENATE_ROOTMODE_SHARED
, the root space is considered the same for each input SF (checked in debug mode) and is also the same in the output SF.
For PETSCSF_CONCATENATE_ROOTMODE_LOCAL
and PETSCSF_CONCATENATE_ROOTMODE_GLOBAL
, the input root spaces are taken as separate and joined.
PETSCSF_CONCATENATE_ROOTMODE_LOCAL
joins the root spaces locally;
roots of sfs[0], sfs[1], sfs[2], … are joined on each rank separately, ordered by input SF and original local index, and renumbered contiguously.
PETSCSF_CONCATENATE_ROOTMODE_GLOBAL
joins the root spaces globally;
roots of sfs[0], sfs[1], sfs[2, … are joined globally, ordered by input SF index and original global index, and renumbered contiguously;
the original root ranks are ignored.
For both PETSCSF_CONCATENATE_ROOTMODE_LOCAL
and PETSCSF_CONCATENATE_ROOTMODE_GLOBAL
,
the output SF’s root layout is such that the local number of roots is a sum of the input SF’s local numbers of roots on each rank to keep the load balancing.
However, for PETSCSF_CONCATENATE_ROOTMODE_GLOBAL
, that means roots can move to different ranks.
Example#
We can use src/vec/is/sf/tests/ex18.c to compare the root modes. By running
make -C $PETSC_DIR/src/vec/is/sf/tests ex18
for m in {local,global,shared}; do
mpirun -n 2 $PETSC_DIR/src/vec/is/sf/tests/ex18 -nsfs 2 -n 2 -root_mode $m -sf_view
done
we generate two identical SFs sf_0 and sf_1,
PetscSF Object: sf_0 2 MPI processes
type: basic
rank #leaves #roots
[ 0] 4 2
[ 1] 4 2
leaves roots roots in global numbering
( 0, 0) <- ( 0, 0) = 0
( 0, 1) <- ( 0, 1) = 1
( 0, 2) <- ( 1, 0) = 2
( 0, 3) <- ( 1, 1) = 3
( 1, 0) <- ( 0, 0) = 0
( 1, 1) <- ( 0, 1) = 1
( 1, 2) <- ( 1, 0) = 2
( 1, 3) <- ( 1, 1) = 3
and pass them to PetscSFConcatenate()
along with different choices of rootMode, yielding different result_sf#
rootMode = local:
PetscSF Object: result_sf 2 MPI processes
type: basic
rank #leaves #roots
[ 0] 8 4
[ 1] 8 4
leaves roots roots in global numbering
( 0, 0) <- ( 0, 0) = 0
( 0, 1) <- ( 0, 1) = 1
( 0, 2) <- ( 1, 0) = 4
( 0, 3) <- ( 1, 1) = 5
( 0, 4) <- ( 0, 2) = 2
( 0, 5) <- ( 0, 3) = 3
( 0, 6) <- ( 1, 2) = 6
( 0, 7) <- ( 1, 3) = 7
( 1, 0) <- ( 0, 0) = 0
( 1, 1) <- ( 0, 1) = 1
( 1, 2) <- ( 1, 0) = 4
( 1, 3) <- ( 1, 1) = 5
( 1, 4) <- ( 0, 2) = 2
( 1, 5) <- ( 0, 3) = 3
( 1, 6) <- ( 1, 2) = 6
( 1, 7) <- ( 1, 3) = 7
rootMode = global:
PetscSF Object: result_sf 2 MPI processes
type: basic
rank #leaves #roots
[ 0] 8 4
[ 1] 8 4
leaves roots roots in global numbering
( 0, 0) <- ( 0, 0) = 0
( 0, 1) <- ( 0, 1) = 1
( 0, 2) <- ( 0, 2) = 2
( 0, 3) <- ( 0, 3) = 3
( 0, 4) <- ( 1, 0) = 4
( 0, 5) <- ( 1, 1) = 5
( 0, 6) <- ( 1, 2) = 6
( 0, 7) <- ( 1, 3) = 7
( 1, 0) <- ( 0, 0) = 0
( 1, 1) <- ( 0, 1) = 1
( 1, 2) <- ( 0, 2) = 2
( 1, 3) <- ( 0, 3) = 3
( 1, 4) <- ( 1, 0) = 4
( 1, 5) <- ( 1, 1) = 5
( 1, 6) <- ( 1, 2) = 6
( 1, 7) <- ( 1, 3) = 7
rootMode = shared:
PetscSF Object: result_sf 2 MPI processes
type: basic
rank #leaves #roots
[ 0] 8 2
[ 1] 8 2
leaves roots roots in global numbering
( 0, 0) <- ( 0, 0) = 0
( 0, 1) <- ( 0, 1) = 1
( 0, 2) <- ( 1, 0) = 2
( 0, 3) <- ( 1, 1) = 3
( 0, 4) <- ( 0, 0) = 0
( 0, 5) <- ( 0, 1) = 1
( 0, 6) <- ( 1, 0) = 2
( 0, 7) <- ( 1, 1) = 3
( 1, 0) <- ( 0, 0) = 0
( 1, 1) <- ( 0, 1) = 1
( 1, 2) <- ( 1, 0) = 2
( 1, 3) <- ( 1, 1) = 3
( 1, 4) <- ( 0, 0) = 0
( 1, 5) <- ( 0, 1) = 1
( 1, 6) <- ( 1, 0) = 2
( 1, 7) <- ( 1, 1) = 3
See Also#
PetscSF
, PetscSFCompose()
, PetscSFGetGraph()
, PetscSFSetGraph()
, PetscSFConcatenateRootMode
Level#
advanced
Location#
Index of all PetscSF routines
Table of Contents for all manual pages
Index of all manual pages