Implementing a `CommunicationSpace`

Implementing a CommunicationSpace

Summary

#109 introduced the concept of multiple CommunicationSpaces. A CommunicationSpace defines what Kokkos Comm actually does when you call e.g. KokkosComm::send.

Your implementation is a struct (or other type) that represents your communication space, as well as partial specializations of another struct for each API.

struct representing CommunicationSpace

The first piece is a struct that represents your CommunicationSpace. We're still a bit fuzzy on the interface for this thing, and ultimately, it might be very small. It's main purpose is to serve as a tag for the partial specializations of each API struct.

For example, in the MPI CommunicationSpace, we define the following

namespace KokkosComm {

// TODO: not sure what members this thing needs
struct Mpi {
  static int world_size() { ... }
  static int world_rank() { ... }
};

template <>
struct Impl::is_communication_space<KokkosComm::Mpi> : public std::true_type {};

}

You can see Mpi has two static methods, but these methods are not required. The main point is that the struct Mpi exists.

struct Partial Specialization for each API

The APIs are actually implemented by partial specializations of structs. Conceptually, there is an internal interface that needs to be satisfied for each API:

namespace Impl {
template <KokkosExecutionSpace ExecSpace, KokkosView RecvView, CommunicationSpace CS>
struct Recv<RecvView, ExecSpace, CS> { ... };
}

In the above, CS is a type that represents the CommunicationSpace implementation. For example, for the Mpi CommunicationSpace, we create a partial specialization of that struct template (note, fewer template parameters and the use of Mpi):

namespace Impl {
template <KokkosExecutionSpace ExecSpace, KokkosView RecvView>
struct Recv<RecvView, ExecSpace, Mpi> { ... };
}