Skip to content

Commit

Permalink
[ENH] Add Segment Interfaces (#1853)
Browse files Browse the repository at this point in the history
## Description of changes

*Summarize the changes made by this PR.*
 - Improvements & Bug fixes
- Adds an interface type for segments. We can add flush etc here in the
future. I will move hnsw to use this in a stacked PR.
 - New functionality
	 - None.

## Test plan
*How are these changes tested?*
No logical changes. These are just interfaces.
- [x] Tests pass locally with `pytest` for python, `yarn test` for js,
`cargo test` for rust

## Documentation Changes
None
  • Loading branch information
HammadB authored Mar 14, 2024
1 parent 97c7539 commit 79f0cd1
Show file tree
Hide file tree
Showing 2 changed files with 13 additions and 0 deletions.
1 change: 1 addition & 0 deletions rust/worker/src/segment/mod.rs
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,7 @@ pub(crate) mod config;
mod distributed_hnsw_segment;
mod segment_ingestor;
mod segment_manager;
mod types;

pub(crate) use segment_ingestor::*;
pub(crate) use segment_manager::*;
12 changes: 12 additions & 0 deletions rust/worker/src/segment/types.rs
Original file line number Diff line number Diff line change
@@ -0,0 +1,12 @@
use crate::types::EmbeddingRecord;

trait SegmentWriter {
fn begin_transaction(&self);
fn write_records(&self, records: Vec<Box<EmbeddingRecord>>, offset_ids: Vec<u32>);
fn commit_transaction(&self);
fn rollback_transaction(&self);
}

trait OffsetIdAssigner: SegmentWriter {
fn assign_offset_ids(&self, records: Vec<Box<EmbeddingRecord>>) -> Vec<u32>;
}

0 comments on commit 79f0cd1

Please sign in to comment.