Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

use BlockingQueue in LazyMemorySegmentPool #119

Open
wants to merge 1 commit into
base: main
Choose a base branch
from

Conversation

0rrange
Copy link

@0rrange 0rrange commented Dec 4, 2024

Purpose

Linked issue: close #90 #136

use BlockingQueue instead of List with lock in LazyMemorySegmentPool

Tests

API and Format

com.alibaba.fluss.memory.LazyMemorySegmentPool#nextSegment
com.alibaba.fluss.memory.LazyMemorySegmentPool#waitForSegment
com.alibaba.fluss.memory.LazyMemorySegmentPool#getAllCachePages

Documentation

@CLAassistant
Copy link

CLAassistant commented Dec 5, 2024

CLA assistant check
All committers have signed the CLA.

@loserwang1024
Copy link
Collaborator

Could you please give more explaination why use use BlockingQueue instead of List with lock in LazyMemorySegmentPool?

It's thread safe If one thread only produces into BlockingQueue while another thread only consumes. However, in many places, use multiple operations on cachePages happens. for example in com.alibaba.fluss.memory.LazyMemorySegmentPool#nextSegment includes cachePages.isEmpty(), cachePages.add() and cachePages.remove. Can these operations be thread safe?

@0rrange 0rrange force-pushed the LazyMemorySegmentPoolImprove branch from 7d52985 to 7273a9c Compare December 6, 2024 19:17
@0rrange
Copy link
Author

0rrange commented Dec 6, 2024

Could you please give more explaination why use use BlockingQueue instead of List with lock in LazyMemorySegmentPool?

It's thread safe If one thread only produces into BlockingQueue while another thread only consumes. However, in many places, use multiple operations on cachePages happens. for example in com.alibaba.fluss.memory.LazyMemorySegmentPool#nextSegment includes cachePages.isEmpty(), cachePages.add() and cachePages.remove. Can these operations be thread safe?

Blocking queues are thread-safe, which can be used directly without additional synchronisation mechanisms, and then it comes with a blocking operation that doesn't require the wait logic to be implemented via Condition.
I used BlockingQueue's poll instead of cachePages.remove() and condition.await(). cachePages.isEmpty() and cachePages.add() I didn't change, and I think that locking control is still needed in a concurrent environment.

@0rrange
Copy link
Author

0rrange commented Dec 6, 2024

@swuferhong I've submitted the code, please review, thx

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants