Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

RDMA memory registration #169

Open
jcarreira opened this issue Sep 4, 2017 · 1 comment
Open

RDMA memory registration #169

jcarreira opened this issue Sep 4, 2017 · 1 comment
Assignees

Comments

@jcarreira
Copy link
Owner

Right now the RDMA backend supports two methods for dealing with memory pinning:

  1. A pointer to memory (RDMAMem) previously pinned is passed to the RDMAClient to avoid having to register memory
  2. If no RDMAMem pointer is passed, Cirrus registers the memory region where the data passed to Cirrus lives

Currently, mostly version 2 is being used which is very expensive.

A potentially more efficient alternative would be to

  1. register a large memory region (e.g., 500MB) when a client is constructed
  2. serialize all writes and reads to that region by allocating memory from there (using external allocator provided by boost)
  3. if we run out of memory from that region we can resort to version 2 above
@jcarreira jcarreira self-assigned this Sep 12, 2017
@jcarreira
Copy link
Owner Author

The externall allocator from boost doesn't seem very efficient when compared against libc's.

It might still be faster to pre-pin+BoostAllocator than to pin memory every time.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant