discovers communicating ranks given one-sided information, calling user-defined functions during rendezvous, returns requests


#include "petscsys.h"  
PetscErrorCode PetscCommBuildTwoSidedFReq(MPI_Comm comm, PetscMPIInt count, MPI_Datatype dtype, PetscMPIInt nto, const PetscMPIInt *toranks, const void *todata, PetscMPIInt *nfrom, PetscMPIInt **fromranks, void *fromdata, PetscMPIInt ntags, MPI_Request **toreqs, MPI_Request **fromreqs, PetscErrorCode (*send)(MPI_Comm, const PetscMPIInt[], PetscMPIInt, PetscMPIInt, void *, MPI_Request[], void *), PetscErrorCode (*recv)(MPI_Comm, const PetscMPIInt[], PetscMPIInt, void *, MPI_Request[], void *), void *ctx)


Input Parameters#

  • comm - communicator

  • count - number of entries to send/receive in initial rendezvous (must match on all ranks)

  • dtype - datatype to send/receive from each rank (must match on all ranks)

  • nto - number of ranks to send data to

  • toranks - ranks to send to (array of length nto)

  • todata - data to send to each rank (packed)

  • ntags - number of tags needed by send/recv callbacks

  • send - callback invoked on sending process when ready to send primary payload

  • recv - callback invoked on receiving process after delivery of rendezvous message

  • ctx - context for callbacks

Output Parameters#

  • nfrom - number of ranks receiving messages from

  • fromranks - ranks receiving messages from (length nfrom; caller should PetscFree())

  • fromdata - packed data from each rank, each with count entries of type dtype (length nfrom, caller responsible for PetscFree())

  • toreqs - array of nto*ntags sender requests (caller must wait on these, then PetscFree())

  • fromreqs - array of nfrom*ntags receiver requests (caller must wait on these, then PetscFree())


This memory-scalable interface is an alternative to calling PetscGatherNumberOfMessages() and PetscGatherMessageLengths(), possibly with a subsequent round of communication to send other data.

Basic data types as well as contiguous types are supported, but non-contiguous (e.g., strided) types are not.


  • **** -*** Hoefler, Siebert and Lumsdaine, The MPI_Ibarrier implementation uses the algorithm in Scalable communication protocols for dynamic sparse data exchange, 2010.

See Also#

PetscCommBuildTwoSided(), PetscCommBuildTwoSidedF(), PetscGatherNumberOfMessages(), PetscGatherMessageLengths()





Edit on GitLab

Index of all Sys routines
Table of Contents for all manual pages
Index of all manual pages