PetscCommBuildTwoSidedFReq#

discovers communicating ranks given one-sided information, calling user-defined functions during rendezvous, returns requests

Synopsis#

#include "petscsys.h"  
PetscErrorCode PetscCommBuildTwoSidedFReq(MPI_Comm comm, PetscMPIInt count, MPI_Datatype dtype, PetscMPIInt nto, const PetscMPIInt *toranks, const void *todata, PetscMPIInt *nfrom, PetscMPIInt **fromranks, void *fromdata, PetscMPIInt ntags, MPI_Request **toreqs, MPI_Request **fromreqs, PetscErrorCode (*send)(MPI_Comm, const PetscMPIInt[], PetscMPIInt, PetscMPIInt, void *, MPI_Request[], void *), PetscErrorCode (*recv)(MPI_Comm, const PetscMPIInt[], PetscMPIInt, void *, MPI_Request[], void *), void *ctx)

Collective

Input Parameters#

  • comm - communicator

  • count - number of entries to send/receive in initial rendezvous (must match on all ranks)

  • dtype - datatype to send/receive from each rank (must match on all ranks)

  • nto - number of ranks to send data to

  • toranks - ranks to send to (array of length nto)

  • todata - data to send to each rank (packed)

  • ntags - number of tags needed by send/recv callbacks

  • send - callback invoked on sending process when ready to send primary payload

  • recv - callback invoked on receiving process after delivery of rendezvous message

  • ctx - context for callbacks

Output Parameters#

  • nfrom - number of ranks receiving messages from

  • fromranks - ranks receiving messages from (length nfrom; caller should PetscFree())

  • fromdata - packed data from each rank, each with count entries of type dtype (length nfrom, caller responsible for PetscFree())

  • toreqs - array of nto*ntags sender requests (caller must wait on these, then PetscFree())

  • fromreqs - array of nfrom*ntags receiver requests (caller must wait on these, then PetscFree())

Notes#

This memory-scalable interface is an alternative to calling PetscGatherNumberOfMessages() and PetscGatherMessageLengths(), possibly with a subsequent round of communication to send other data, [HSL10].

Basic data types as well as contiguous types are supported, but non-contiguous (e.g., strided) types are not.

References#

HSL10

T. Hoefler, C. Siebert, and A. Lumsdaine. Scalable Communication Protocols for Dynamic Sparse Data Exchange. In Proceedings of the 2010 ACM SIGPLAN Symposium on Principles and Practice of Parallel Programming (PPoPP'10), 159–168. ACM, January 2010.

See Also#

PetscCommBuildTwoSided(), PetscCommBuildTwoSidedF(), PetscGatherNumberOfMessages(), PetscGatherMessageLengths()

Level#

developer

Location#

src/sys/utils/mpits.c

Index of all Sys routines
Table of Contents for all manual pages
Index of all manual pages