Skip to content

Commit

Permalink
[BUG] Log service N+1 query (#1923)
Browse files Browse the repository at this point in the history
## Description of changes

*Summarize the changes made by this PR.*
 - Improvements & Bug fixes
- The rust log service makes an additional query when it doesn't need to
if the timestamp is provided and we read less than a batch size. This
means that we are done reading the log and don't need to fetch again.
 - New functionality
	 - None

## Test plan
*How are these changes tested?*
- [x] Tests pass locally with `pytest` for python, `yarn test` for js,
`cargo test` for rust

## Documentation Changes
None
  • Loading branch information
HammadB authored Mar 24, 2024
1 parent ba7b52e commit 93b1f2a
Showing 1 changed file with 7 additions and 0 deletions.
7 changes: 7 additions & 0 deletions rust/worker/src/execution/operators/pull_log.rs
Original file line number Diff line number Diff line change
Expand Up @@ -123,6 +123,13 @@ impl Operator<PullLogsInput, PullLogsOutput> for PullLogsOperator {
offset += batch_size as i64;
result.append(&mut logs);

// We used a a timestamp and we didn't get a full batch, so we have retrieved
// the last batch of logs relevant to our query
if input.end_timestamp.is_some() && num_records_read < batch_size as usize {
break;
}

// We have read all the records up to the size we wanted
if input.num_records.is_some()
&& num_records_read >= input.num_records.unwrap() as usize
{
Expand Down

0 comments on commit 93b1f2a

Please sign in to comment.