petsc-3.9.4 2018-09-11

VecScatterCreate

Creates a vector scatter context.

Synopsis

#include "petscvec.h"    
PetscErrorCode VecScatterCreate(Vec xin,IS ix,Vec yin,IS iy,VecScatter *newctx)

Collective on Vec

Input Parameters

	xin	- a vector that defines the shape (parallel data layout of the vector) of vectors from which we scatter
	yin	- a vector that defines the shape (parallel data layout of the vector) of vectors to which we scatter
	ix	- the indices of xin to scatter (if NULL scatters all values)
	iy	- the indices of yin to hold results (if NULL fills entire vector yin)

Output Parameter

newctx -location to store the new scatter context

Options Database Keys: (uses regular MPI_Sends by default)

-vecscatter_view -Prints detail of communications

-vecscatter_view ::ascii_info -Print less details about communication

-vecscatter_ssend -Uses MPI_Ssend_init() instead of MPI_Send_init()

-vecscatter_rsend -use ready receiver mode for MPI sends

-vecscatter_merge -VecScatterBegin() handles all of the communication, VecScatterEnd() is a nop eliminates the chance for overlap of computation and communication

-vecscatter_sendfirst -Posts sends before receives

-vecscatter_packtogether -Pack all messages before sending, receive all messages before unpacking

-vecscatter_alltoall -Uses MPI all to all communication for scatter

-vecscatter_window -Use MPI 2 window operations to move data

-vecscatter_nopack -Avoid packing to work vector when possible (if used with -vecscatter_alltoall then will use MPI_Alltoallw()

-vecscatter_reproduce -insure that the order of the communications are done the same for each scatter, this under certain circumstances will make the results of scatters deterministic when otherwise they are not (it may be slower also).

                                                                                   --When packing is used--

                              MPI Datatypes (no packing)  sendfirst   merge        packtogether  persistent*

                               _nopack                   _sendfirst    _merge      _packtogether                -vecscatter_

----------------------------------------------------------------------------------------------------------------------------

   Message passing    Send       p                           X            X           X         always

                     Ssend       p                           X            X           X         always          _ssend

                     Rsend       p                        nonsense        X           X         always          _rsend

   AlltoAll  v or w              X                        nonsense     always         X         nonsense        _alltoall

   MPI_Win                       p                        nonsense        p           p         nonsense        _window

  Since persistent sends and receives require a constant memory address they can only be used when data is packed into the work vector

  because the in and out array may be different for each call to VecScatterBegin/End().

   p indicates possible, but not implemented. X indicates implemented

Notes

In calls to VecScatter() you can use different vectors than the xin and yin you used above; BUT they must have the same parallel data layout, for example, they could be obtained from VecDuplicate(). A VecScatter context CANNOT be used in two or more simultaneous scatters; that is you cannot call a second VecScatterBegin() with the same scatter context until the VecScatterEnd() has been called on the first VecScatterBegin(). In this case a separate VecScatter is needed for each concurrent scatter.

Currently the MPI_Send(), MPI_Ssend() and MPI_Rsend() all use PERSISTENT versions. (this unfortunately requires that the same in and out arrays be used for each use, this is why when not using MPI_alltoallw() we always need to pack the input into the work array before sending and unpack upon receeving instead of using MPI datatypes to avoid the packing/unpacking).

Both ix and iy cannot be NULL at the same time.

Level

intermediate

Location

src/vec/vscat/interface/vscreate.c

Examples

src/vec/vec/utils/tagger/examples/tutorials/ex1.c.html
src/dm/examples/tutorials/ex6.c.html
src/ksp/ksp/examples/tutorials/ex49.c.html
src/tao/pde_constrained/examples/tutorials/elliptic.c.html
src/tao/pde_constrained/examples/tutorials/parabolic.c.html
src/tao/pde_constrained/examples/tutorials/hyperbolic.c.html

Implementations