PetscCommBuildTwoSidedF#

discovers communicating ranks given one-sided information, calling user-defined functions during rendezvous

Synopsis#

#include "petscsys.h"  
PetscErrorCode PetscCommBuildTwoSidedF(MPI_Comm comm, PetscMPIInt count, MPI_Datatype dtype, PetscMPIInt nto, const PetscMPIInt *toranks, const void *todata, PetscMPIInt *nfrom, PetscMPIInt **fromranks, void *fromdata, PetscMPIInt ntags, PetscErrorCode (*send)(MPI_Comm, const PetscMPIInt[], PetscMPIInt, PetscMPIInt, void *, MPI_Request[], void *), PetscErrorCode (*recv)(MPI_Comm, const PetscMPIInt[], PetscMPIInt, void *, MPI_Request[], void *), void *ctx)

Collective

Input Parameters#

comm - communicator
count - number of entries to send/receive in initial rendezvous (must match on all ranks)
dtype - datatype to send/receive from each rank (must match on all ranks)
nto - number of ranks to send data to
toranks - ranks to send to (array of length nto)
todata - data to send to each rank (packed)
ntags - number of tags needed by send/recv callbacks
send - callback invoked on sending process when ready to send primary payload
recv - callback invoked on receiving process after delivery of rendezvous message
ctx - context for callbacks

Output Parameters#

nfrom - number of ranks receiving messages from
fromranks - ranks receiving messages from (length nfrom; caller should PetscFree())
fromdata - packed data from each rank, each with count entries of type dtype (length nfrom, caller responsible for PetscFree())

Notes#

This memory-scalable interface is an alternative to calling PetscGatherNumberOfMessages() and PetscGatherMessageLengths(), possibly with a subsequent round of communication to send other data, [HSL10].

Basic data types as well as contiguous types are supported, but non-contiguous (e.g., strided) types are not.

References#

[HSL10]

T. Hoefler, C. Siebert, and A. Lumsdaine. Scalable Communication Protocols for Dynamic Sparse Data Exchange. In Proceedings of the 2010 ACM SIGPLAN Symposium on Principles and Practice of Parallel Programming (PPoPP'10), 159–168. ACM, January 2010.

Level#

developer

Location#

src/sys/utils/mpits.c

Index of all Sys routines
Table of Contents for all manual pages
Index of all manual pages