PetscSFConcatenate#
concatenate multiple PetscSF into one
Synopsis#
#include "petscsf.h"
PetscErrorCode PetscSFConcatenate(MPI_Comm comm, PetscInt nsfs, PetscSF sfs[], PetscSFConcatenateRootMode rootMode, PetscInt leafOffsets[], PetscSF *newsf)
Input Parameters#
Output Parameter#
newsf - The resulting
PetscSF
Notes#
The communicator of all PetscSFs in sfs must be comm.
Leaves are always concatenated locally, keeping them ordered by the input PetscSF index and original local order.
The offsets in leafOffsets are added to the original leaf indices.
If all input SFs use contiguous leaf storage (ilocal = NULL), leafOffsets can be passed as NULL as well.
In this case, NULL is also passed as ilocal to the resulting PetscSF.
If any input PetscSF has non-null ilocal, leafOffsets is needed to distinguish leaves from different input PetscSFs.
In this case, user is responsible to provide correct offsets so that the resulting leaves are unique (otherwise an error occurs).
All root modes retain the essential connectivity condition.
If two leaves of the same input PetscSF are connected (sharing the same root), they are also connected in the output PetscSF.
Parameter rootMode controls how the input root spaces are combined.
For PETSCSF_CONCATENATE_ROOTMODE_SHARED, the root space is considered the same for each input PetscSF (checked in debug mode)
and is also the same in the output PetscSF.
For PETSCSF_CONCATENATE_ROOTMODE_LOCAL and PETSCSF_CONCATENATE_ROOTMODE_GLOBAL, the input root spaces are taken as separate and joined.
PETSCSF_CONCATENATE_ROOTMODE_LOCAL joins the root spaces locally;
roots of sfs[0], sfs[1], sfs[2], … are joined on each rank separately, ordered by input PetscSF and original local index, and renumbered contiguously.
PETSCSF_CONCATENATE_ROOTMODE_GLOBAL joins the root spaces globally;
roots of sfs[0], sfs[1], sfs[2], … are joined globally, ordered by input PetscSF index and original global index, and renumbered contiguously;
the original root ranks are ignored.
For both PETSCSF_CONCATENATE_ROOTMODE_LOCAL and PETSCSF_CONCATENATE_ROOTMODE_GLOBAL,
the output PetscSF’s root layout is such that the local number of roots is a sum of the input PetscSF’s local numbers of roots on each rank
to keep the load balancing.
However, for PETSCSF_CONCATENATE_ROOTMODE_GLOBAL, roots can move to different ranks.
Example#
We can use src/vec/is/sf/tests/ex18.c to compare the root modes. By running
make -C $PETSC_DIR/src/vec/is/sf/tests ex18
for m in {local,global,shared}; do
mpirun -n 2 $PETSC_DIR/src/vec/is/sf/tests/ex18 -nsfs 2 -n 2 -root_mode $m -sf_view
done
we generate two identical PetscSFs sf_0 and sf_1,
PetscSF Object: sf_0 2 MPI processes
type: basic
rank #leaves #roots
[ 0] 4 2
[ 1] 4 2
leaves roots roots in global numbering
( 0, 0) <- ( 0, 0) = 0
( 0, 1) <- ( 0, 1) = 1
( 0, 2) <- ( 1, 0) = 2
( 0, 3) <- ( 1, 1) = 3
( 1, 0) <- ( 0, 0) = 0
( 1, 1) <- ( 0, 1) = 1
( 1, 2) <- ( 1, 0) = 2
( 1, 3) <- ( 1, 1) = 3
and pass them to PetscSFConcatenate() along with different choices of rootMode, yielding different result_sf:
rootMode = local:
PetscSF Object: result_sf 2 MPI processes
type: basic
rank #leaves #roots
[ 0] 8 4
[ 1] 8 4
leaves roots roots in global numbering
( 0, 0) <- ( 0, 0) = 0
( 0, 1) <- ( 0, 1) = 1
( 0, 2) <- ( 1, 0) = 4
( 0, 3) <- ( 1, 1) = 5
( 0, 4) <- ( 0, 2) = 2
( 0, 5) <- ( 0, 3) = 3
( 0, 6) <- ( 1, 2) = 6
( 0, 7) <- ( 1, 3) = 7
( 1, 0) <- ( 0, 0) = 0
( 1, 1) <- ( 0, 1) = 1
( 1, 2) <- ( 1, 0) = 4
( 1, 3) <- ( 1, 1) = 5
( 1, 4) <- ( 0, 2) = 2
( 1, 5) <- ( 0, 3) = 3
( 1, 6) <- ( 1, 2) = 6
( 1, 7) <- ( 1, 3) = 7
rootMode = global:
PetscSF Object: result_sf 2 MPI processes
type: basic
rank #leaves #roots
[ 0] 8 4
[ 1] 8 4
leaves roots roots in global numbering
( 0, 0) <- ( 0, 0) = 0
( 0, 1) <- ( 0, 1) = 1
( 0, 2) <- ( 0, 2) = 2
( 0, 3) <- ( 0, 3) = 3
( 0, 4) <- ( 1, 0) = 4
( 0, 5) <- ( 1, 1) = 5
( 0, 6) <- ( 1, 2) = 6
( 0, 7) <- ( 1, 3) = 7
( 1, 0) <- ( 0, 0) = 0
( 1, 1) <- ( 0, 1) = 1
( 1, 2) <- ( 0, 2) = 2
( 1, 3) <- ( 0, 3) = 3
( 1, 4) <- ( 1, 0) = 4
( 1, 5) <- ( 1, 1) = 5
( 1, 6) <- ( 1, 2) = 6
( 1, 7) <- ( 1, 3) = 7
rootMode = shared:
PetscSF Object: result_sf 2 MPI processes
type: basic
rank #leaves #roots
[ 0] 8 2
[ 1] 8 2
leaves roots roots in global numbering
( 0, 0) <- ( 0, 0) = 0
( 0, 1) <- ( 0, 1) = 1
( 0, 2) <- ( 1, 0) = 2
( 0, 3) <- ( 1, 1) = 3
( 0, 4) <- ( 0, 0) = 0
( 0, 5) <- ( 0, 1) = 1
( 0, 6) <- ( 1, 0) = 2
( 0, 7) <- ( 1, 1) = 3
( 1, 0) <- ( 0, 0) = 0
( 1, 1) <- ( 0, 1) = 1
( 1, 2) <- ( 1, 0) = 2
( 1, 3) <- ( 1, 1) = 3
( 1, 4) <- ( 0, 0) = 0
( 1, 5) <- ( 0, 1) = 1
( 1, 6) <- ( 1, 0) = 2
( 1, 7) <- ( 1, 1) = 3
See Also#
PetscSF, PetscSFCompose(), PetscSFGetGraph(), PetscSFSetGraph(), PetscSFConcatenateRootMode
Level#
advanced
Location#
Index of all PetscSF routines
Table of Contents for all manual pages
Index of all manual pages