Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add batch insert API + types #24

Open
NeilLuno opened this issue Nov 12, 2024 · 0 comments · May be fixed by #25
Open

Add batch insert API + types #24

NeilLuno opened this issue Nov 12, 2024 · 0 comments · May be fixed by #25
Assignees
@NeilLuno

Comments

@NeilLuno
Copy link
Contributor

Some of our transactions insert many events, and it's inefficient for each query to do two round trips to the server to prepare the statement then execute it. Mysql docs recommend inserting using a single INSERT with multiple VALUES tuples to speed up inserts.

I think we can achieve this by introducing a manyInserter type:

type EventToInsert struct {
  ForeignID string
  Type reflex.Type
  Metadata []byte
}

type manyInserter func(ctx context.Context, tx *sql.Tx, events []EventToInsert) error

and an API to insert many events:

func (t *EventsTable) InsertMany(ctx context.Context, tx *sql.Tx, events []EventToInsert) (NotifyFunc, error) {
    // Check noops...
	err := t.manyInserter(ctx, tx, events)
	if err != nil {
		return noopFunc, err
	}

	return t.notifier.Notify, nil
}

We can provide a default implementation like we do for inserter.
@NeilLuno NeilLuno self-assigned this Nov 12, 2024
@NeilLuno NeilLuno linked a pull request Nov 12, 2024 that will close this issue
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@NeilLuno NeilLuno

Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

rsql: Add manyInserter and InsertMany API luno/reflex
1 participant
@NeilLuno