Skip to content

Commit

Permalink
Persisent concurrent shared composable element pools
Browse files Browse the repository at this point in the history
By popular demand, this provides a code generator for persistent
concurrent shared composable elements pools.  This is a building block
for more advanced concurrent data structures that have been requested.
From fd_pool_para.c:

  Generate prototypes, inlines and/or implementations for concurrent
  persistent element pools.  A pool can hold a practically unbounded
  number of elements.  Acquiring an element from and releasing an
  element to a pool are typically fast O(1) time.  Requires small O(1)
  space per element.

  The current implementation is based on a lockfree stack.  Acquire and
  release are done via atomic compare-and-swap of the stack top.  As
  such, concurrent usage requires FD_HAS_ATOMIC support (this can still
  be used on platforms without FD_HAS_ATOMIC support but it will not be
  safe for concurrent usage).  Stack top versioning is used to handle
  ABA.  Versioning has been been tweaked to support locking global pool
  operations like initialization (and thus, this can also be used
  without changes as a more conventional spin lock based concurrent
  stack).  Unsurprisingly, the current implementation is equally usable
  as a concurrent element stack (though the implementation may be
  changed in the future to better support ultra high contention ultra
  high concurrency like fd_alloc).

  The current implementation is optimized for pools with a moderate
  number of reasonably localized users (e.g. a handful of cores and
  memory on the same NUMA node).  Various operations are slightly more
  optimal when the size of a pool element is an integer power of 2.
  Operations do much internal integrity checking / bounds checking for
  use in high reliability / high security environments.

  This API is designed for tight and flexible composition with treaps,
  heaps, lists, maps, etc.  Further, a pool can be persisted beyond the
  lifetime of the creating process, be used inter-process, be relocated
  in memory, be naively serialized/deserialized, be moved between hosts,
  use index compression for cache and memory bandwidth, etc.

See fd_pool_para.c for more details.
  • Loading branch information
kbowers-jump authored and lidatong committed Dec 7, 2024
1 parent 0bb75ca commit 16b28c2
Show file tree
Hide file tree
Showing 3 changed files with 1,138 additions and 2 deletions.
6 changes: 4 additions & 2 deletions src/util/tmpl/Local.mk
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
$(call add-hdrs,fd_bplus.c fd_deque.c fd_deque_dynamic.c fd_dlist.c fd_heap.c fd_map.c fd_map_chain.c fd_map_dynamic.c fd_map_giant.c fd_pool.c fd_prq.c fd_queue.c fd_queue_dynamic.c fd_redblack.c fd_set.c fd_set_dynamic.c fd_smallset.c fd_sort.c fd_stack.c fd_treap.c fd_vec.c fd_voff.c)
$(call add-hdrs,fd_bplus.c fd_deque.c fd_deque_dynamic.c fd_dlist.c fd_heap.c fd_map.c fd_map_chain.c fd_map_dynamic.c fd_map_giant.c fd_pool.c fd_pool_para.c fd_prq.c fd_queue.c fd_queue_dynamic.c fd_redblack.c fd_set.c fd_set_dynamic.c fd_smallset.c fd_sort.c fd_stack.c fd_treap.c fd_vec.c fd_voff.c)
$(call make-unit-test,test_bplus,test_bplus,fd_util)
$(call make-unit-test,test_deque,test_deque,fd_util)
$(call make-unit-test,test_deque_dynamic,test_deque_dynamic,fd_util)
Expand All @@ -10,11 +10,12 @@ $(call make-unit-test,test_map_chain_multi,test_map_chain_multi,fd_util)
$(call make-unit-test,test_map_dynamic,test_map_dynamic,fd_util)
$(call make-unit-test,test_map_giant,test_map_giant,fd_util)
$(call make-unit-test,test_map_giant_mem,test_map_giant_mem,fd_util)
ifdef FD_HAS_HOSTED
ifdef FD_HAS_HOSTED # FIXME: HMMM ....
$(call make-unit-test,test_map_giant_concur,test_map_giant_concur,fd_util)
endif
$(call make-unit-test,test_map_perfect,test_map_perfect,fd_util)
$(call make-unit-test,test_pool,test_pool,fd_util)
$(call make-unit-test,test_pool_para,test_pool_para,fd_util)
$(call make-unit-test,test_prq,test_prq,fd_util)
$(call make-unit-test,test_queue,test_queue,fd_util)
$(call make-unit-test,test_queue_dynamic,test_queue_dynamic,fd_util)
Expand All @@ -40,6 +41,7 @@ $(call run-unit-test,test_map_dynamic,)
$(call run-unit-test,test_map_giant,)
$(call run-unit-test,test_map_giant_mem,)
$(call run-unit-test,test_pool,)
$(call run-unit-test,test_pool_para,)
$(call run-unit-test,test_prq,)
$(call run-unit-test,test_queue,)
$(call run-unit-test,test_queue_dynamic,)
Expand Down
Loading

0 comments on commit 16b28c2

Please sign in to comment.