Reduce the device bubble introduced by heavy loop synchronization in coalesced fetch/release(z3_leaf_module)

[inkcherry]

depend on #6649

When performing fetch/release operations on Z3 leaf modules, the loop time is excessively long in fine-grained module. Compared to non-leaf modules, Z3 leaf modules may include a larger number of parameters. Although each loop unit does not consume much time, the overall loop length can be significant.

The fetch time is impacted by:

Post-allgather operations (narrow， slice ，cat, difficult to avoid)
Memory pressure（record_stream/fetch event create&sync）
The release time is impacted by:
slice
Free parameter record_stream

Considering the fine-grained leaf modules, where each parameter is relatively small, we can treat the parameters within each leaf module as a unified entity to handle memory pressure. This approach can approximately halve the CPU time required for fetch/release operations.

[inkcherry]

Hi , @tjruwase. Some models use a large number of MoE experts, but the total number of parameters for MoE remains unchanged. This PR handles such cases as a whole to optimize memory management and reduce bubbles caused by long loops. when zero_module_granularity_threshold is enabled.

Do you have any suggestions?