Pagecache: use an objcache as page heap #1757
Conversation
There was a problem hiding this comment.
I'm wondering if some of the i/o objcache cleaners (maybe net drivers and pagecache?) should retain a small minimum number of objects that do not get freed so that in a low memory condition critical resources can still function. If the operation requiring lots of memory requires i/o of some sort in order to free memory, starving those devices completely of memory could deadlock the system. What do you think?
Yes, I think it makes sense to provide a way for a caching heap to retain a configurable amount of memory when a drain is requested; e.g. by adding a |
francescolavra
force-pushed
the
feature/caching_heap
branch
from
6789ea7
to
7fe44ea
Compare
|
I added a |
With this change, it is possible to register different "memory cleaner" closures, which will be invoked to free up memory when the system is low on available physical memory. The virtio balloon deflater and the pagecache drain function have been converted to use this interface. Partially addresses #1130.
This macro is being amended so that it is possible to remove a page list element from within the loop that walks the list. This will be needed to implement the cache drain functionality.
This change introduces a new heap sub-type, called caching_heap, that exports an additional callback to drain memory that has been cached. The objcache heap is now a caching_heap. In addition, the objcache heap now implements locking, which allows safe use of the drain functionality in concurrently accessed heaps without the need for an external lock. Memory cleaner instances have been added in various places where an objcache is used, so that caching heaps can be drained when the system is low on memory. Partially addresses #1494.
In the pagecache code, allocating 4KB pages directly from the linear backed heap creates excessive fragmentation, which in low-memory conditions may cause other allocations in the kernel (which typically use objcaches with a 2MB page size) to fail even when pagecache drain requests are fully executed. This commit addresses the fragmentation issue by changing the pagecache code to use an objcache (with a 2MB page size, and with the linear backed heap as parent heap) as page heap; this allows a pagecache drain to release memory in large chunks that are then available for other allocation requests throughout the kernel. In addition, the pagecache completion heap is now drained if a drain request could not be fully executed by draining the page heap.
francescolavra
force-pushed
the
feature/caching_heap
branch
from
7fe44ea
to
22add9b
Compare
In the pagecache code, allocating 4KB pages directly from the linear backed heap creates excessive fragmentation, which in
low-memory conditions may cause other allocations in the kernel (which typically use objcaches with a 2MB page size) to fail even when pagecache drain requests are fully executed.
This changeset addresses the fragmentation issue by changing the pagecache code to use an objcache (with a 2MB page size, and with the linear backed heap as parent heap) as page heap; this allows a pagecache drain to release memory in large chunks that are then available for other allocation requests throughout the kernel.
As part of this changeset, a generic memory cleaner interface is being implemented, which allows various parts of the kernel to register a closure to be invoked to release cached memory when the system is low on memory (#1130). In addition, a new heap sub-type, called caching_heap, is being defined; a caching_heap exports an additional callback to drain memory that has been
cached (#1494); the objcache heap is now a caching_heap.