Memory Spaces sub-section:1 Determining the relationship between Teams, Contexts, and Spaces

[naveen-rn]

Based on OpenSHMEM WG discussions, we would be making the following semantic updates in adding the new Memory Spaces feature.

Core Space Semantics:
Space Creation

1. Spaces are created and destroyed with Teams
2. Each Team can create more than one Space, through the Team configuration option
3. Each Space created along with the Team-split operation, can be passed a Space configuration property
4. Spaces can not be created separately from a Team
5. Spaces are automatically destroyed with a Team destroy operation

Space Management

1. Memory allocation (malloc, calloc, align) can be performed on a space only using the PEs which are part of the Team that created the Space
2. Similarly, memory management routines like free can be performed only using the PEs which are part of the Team that created the Space

Data Movement operations

1. All data movement operations like RMA, AMO, PWS, and collectives can be performed on any PEs which has access to the Space
2. Access to the Space can be obtained either by Space creation or from inheritance. Space creation meaning all contexts which are part of the Team that create the Space, would have access to the Space. Also, Spaces that are part of the parent Team will be available for the child Teams to perform data movement operations.

New routines:

/* Space configuration and Space object */
typedef shmem_space_t void *;
typedef struct {
   size_t sheap_size;
} shmem_space_config_t;

/* updates to Team configuration */
typedef struct {
   // existing team configuration parameters
   int                    num_contexts;
   // new space-specific configurations
   shmem_space_t        * spaces;
   shmem_space_config_t * space_configs;
   shmem_team_t         * team_select;
   int                    num_spaces;
} shmem_team_config_t;

/* Space-based Contexts */
shmem_ctx_config_t {
  shmem_team_t  team;
  shmem_space_t space;
};
int shmem_ext_ctx_create(IN long options, IN shmem_ctx_config_t *config, OUT shmem_ctx_t *ctx);

/* Space-based Memory management */
void *shmem_space_malloc(IN shmem_space_t space, IN size_t size);
void *shmem_space_align(IN shmem_space_t space, IN size_t alignment, IN size_t size); 
void *shmem_space_calloc(IN shmem_space_t space, IN size_t count, IN size_t size);

/* Query operations */
void shmem_get_local_space(IN shmem_team_t team, OUT shmem_space_t *space, OUT int *nspaces);
void shmem_get_parent_space(IN shmem_team_t team, OUT shmem_space_t *space, OUT int *nspaces);
void shmem_get_space(IN void *ptr, OUT shmem_space_t *space);
int shmem_addr_ctx_accessible(shmem_ctx_t ctx, const void *addr, int pe);

For more details on the proposal, please refer the Spaces Wiki link

[naveen-rn]

Closing #361 as it is handled as part of this issue.