Skip to content

Commit

Permalink
chore(pglite/sync): Bump the version of the Electric sync client (#459)
Browse files Browse the repository at this point in the history
* Bump the version of the Electric sync client

* Fix types
  • Loading branch information
samwillis authored Dec 8, 2024
1 parent fa13714 commit 515a155
Show file tree
Hide file tree
Showing 5 changed files with 73 additions and 52 deletions.
5 changes: 5 additions & 0 deletions .changeset/young-bikes-battle.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
---
'@electric-sql/pglite-sync': patch
---

Bump the version of the Electric sync client
42 changes: 9 additions & 33 deletions docs/docs/sync.md
Original file line number Diff line number Diff line change
Expand Up @@ -38,7 +38,12 @@ You can then use the `syncShapeToTable` method to sync a table from Electric:

```ts
const shape = await pg.electric.syncShapeToTable({
shape: { url: 'http://localhost:3000/v1/shape', table: 'todo' },
shape: {
url: 'http://localhost:3000/v1/shape',
params: {
table: 'todo',
},
},
table: 'todo',
primaryKey: ['id'],
})
Expand Down Expand Up @@ -69,7 +74,7 @@ The `syncShapeToTable` is a relatively thin wrapper around the Electric [ShapeSt
It takes the following options as an object:

- `shape: ShapeStreamOptions`<br>
The shape stream specification to sync, described by [`ShapeStreamOptions`](#shapestreamoptions).
The shape stream specification to sync, described by the Electric [ShapeStream API](https://electric-sql.com/docs/api/clients/typescript#shapestream) options, see the [ShapeStream API](https://electric-sql.com/docs/api/clients/typescript#options) for more details.

- `table: string`<br>
The name of the table to sync into.
Expand Down Expand Up @@ -100,40 +105,11 @@ The returned `shape` object from the `syncShapeToTable` call has the following m
- `subscribe(cb: () => void, error: (err: FetchError | Error) => void)`<br>
A callback to indicate that the shape caught up to the main Postgres.

- `subscribeMustRefresh(cb: () => void)`<br>
A callback that is called when the stream emits a `must-refresh` message.

- `unsubscribeMustRefresh(cb: () => void)`<br>
Unsubscribe from the `mustRefresh` notification.

- `unsubscribe()`<br>
Unsubscribe from the shape. Note that this does not clear the state that has been synced into the table.

### `ShapeStreamOptions`

- `url: string`<br>
The full URL to where the Shape is hosted. This can either be the Electric server directly, or a proxy. E.g. for a local Electric instance, you might set `http://localhost:3000/v1/shape`

- `table: string`<br>
The name of the table in the remote database to sync from

- `where?: string`<br>
Where clauses for the shape.

- `offset?: Offset`<br>
The "offset" on the shape log. This is typically not set as the ShapeStream will handle this automatically. A common scenario where you might pass an offset is if you're maintaining a local cache of the log. If you've gone offline and are re-starting a ShapeStream to catch-up to the latest state of the Shape, you'd pass in the last offset and shapeId you'd seen from the Electric server so it knows at what point in the shape to catch you up from.

- `shapeId?: string`<br>
The server side `shapeId`, similar to `offset`, this isn't typically used unless you're maintaining a cache of the shape log.

- `backoffOptions`<br>
Options to configure the backoff rules on failure

- `subscribe?: boolean`<br>
Automatically fetch updates to the Shape. If you just want to sync the current shape and stop, pass false.

- `signal?: AbortSignal`<br>
A `AbortSignal` instance to use to abort the sync.
- `stream: ShapeStream`<br>
The underlying `ShapeStream` instance, see the [ShapeStream API](https://electric-sql.com/docs/api/clients/typescript#shapestream) for more details.

## Limitations

Expand Down
2 changes: 1 addition & 1 deletion packages/pglite-sync/package.json
Original file line number Diff line number Diff line change
Expand Up @@ -45,7 +45,7 @@
"dist"
],
"dependencies": {
"@electric-sql/client": "~0.8.0"
"@electric-sql/client": "~0.9.0"
},
"devDependencies": {
"@electric-sql/pglite": "workspace:*",
Expand Down
65 changes: 52 additions & 13 deletions packages/pglite-sync/test/sync.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -52,7 +52,10 @@ describe('pglite-sync', () => {
}))

const shape = await pg.electric.syncShapeToTable({
shape: { url: 'http://localhost:3000/v1/shape', table: 'todo' },
shape: {
url: 'http://localhost:3000/v1/shape',
params: { table: 'todo' },
},
table: 'todo',
primaryKey: ['id'],
})
Expand Down Expand Up @@ -121,7 +124,10 @@ describe('pglite-sync', () => {
}))

const shape = await pg.electric.syncShapeToTable({
shape: { url: 'http://localhost:3000/v1/shape', table: 'todo' },
shape: {
url: 'http://localhost:3000/v1/shape',
params: { table: 'todo' },
},
table: 'todo',
primaryKey: ['id'],
})
Expand Down Expand Up @@ -201,7 +207,10 @@ describe('pglite-sync', () => {
const numResumes = 3
for (let i = 0; i < numResumes; i++) {
const shape = await pg.electric.syncShapeToTable({
shape: { url: 'http://localhost:3000/v1/shape', table: 'todo' },
shape: {
url: 'http://localhost:3000/v1/shape',
params: { table: 'todo' },
},
table: 'todo',
primaryKey: ['id'],
shapeKey: 'foo',
Expand Down Expand Up @@ -270,7 +279,10 @@ describe('pglite-sync', () => {

const numInserts = 100
const shape = await pg.electric.syncShapeToTable({
shape: { url: 'http://localhost:3000/v1/shape', table: 'todo' },
shape: {
url: 'http://localhost:3000/v1/shape',
params: { table: 'todo' },
},
table: 'todo',
primaryKey: ['id'],
shapeKey: 'foo',
Expand Down Expand Up @@ -334,7 +346,10 @@ describe('pglite-sync', () => {

// resuming should
const resumedShape = await pg.electric.syncShapeToTable({
shape: { url: 'http://localhost:3000/v1/shape', table: 'todo' },
shape: {
url: 'http://localhost:3000/v1/shape',
params: { table: 'todo' },
},
table: 'todo',
primaryKey: ['id'],
shapeKey: 'foo',
Expand Down Expand Up @@ -375,7 +390,10 @@ describe('pglite-sync', () => {
const altTable = 'bar'

const shape1 = await pg.electric.syncShapeToTable({
shape: { url: 'http://localhost:3000/v1/shape', table: 'todo' },
shape: {
url: 'http://localhost:3000/v1/shape',
params: { table: 'todo' },
},
table: table,
primaryKey: ['id'],
})
Expand All @@ -384,15 +402,21 @@ describe('pglite-sync', () => {
await expect(
async () =>
await pg.electric.syncShapeToTable({
shape: { url: 'http://localhost:3000/v1/shape', table: 'todo_alt' },
shape: {
url: 'http://localhost:3000/v1/shape',
params: { table: 'todo_alt' },
},
table: table,
primaryKey: ['id'],
}),
).rejects.toThrowError(`Already syncing shape for table ${table}`)

// should be able to sync shape into other table
const altShape = await pg.electric.syncShapeToTable({
shape: { url: 'http://localhost:3000/v1/shape', table: 'bar' },
shape: {
url: 'http://localhost:3000/v1/shape',
params: { table: 'bar' },
},
table: altTable,
primaryKey: ['id'],
})
Expand All @@ -403,7 +427,10 @@ describe('pglite-sync', () => {
shape1.unsubscribe()

const shape2 = await pg.electric.syncShapeToTable({
shape: { url: 'http://localhost:3000/v1/shape', table: 'todo_alt' },
shape: {
url: 'http://localhost:3000/v1/shape',
params: { table: 'todo_alt' },
},
table: table,
primaryKey: ['id'],
})
Expand All @@ -420,7 +447,10 @@ describe('pglite-sync', () => {
}))

const shape = await pg.electric.syncShapeToTable({
shape: { url: 'http://localhost:3000/v1/shape', table: 'todo' },
shape: {
url: 'http://localhost:3000/v1/shape',
params: { table: 'todo' },
},
table: 'todo',
primaryKey: ['id'],
})
Expand Down Expand Up @@ -506,7 +536,10 @@ describe('pglite-sync', () => {
expect(result0.rows[0]).toEqual({ current_setting: 'false' })

const shape = await pg.electric.syncShapeToTable({
shape: { url: 'http://localhost:3000/v1/shape', table: 'test_syncing' },
shape: {
url: 'http://localhost:3000/v1/shape',
params: { table: 'test_syncing' },
},
table: 'test_syncing',
primaryKey: ['id'],
})
Expand Down Expand Up @@ -548,7 +581,10 @@ describe('pglite-sync', () => {
}))

const shape = await pg.electric.syncShapeToTable({
shape: { url: 'http://localhost:3000/v1/shape', table: 'todo' },
shape: {
url: 'http://localhost:3000/v1/shape',
params: { table: 'todo' },
},
table: 'todo',
primaryKey: ['id'],
useCopy: true,
Expand Down Expand Up @@ -624,7 +660,10 @@ describe('pglite-sync', () => {
}))

const shape = await pg.electric.syncShapeToTable({
shape: { url: 'http://localhost:3000/v1/shape', table: 'todo' },
shape: {
url: 'http://localhost:3000/v1/shape',
params: { table: 'todo' },
},
table: 'todo',
primaryKey: ['id'],
useCopy: true,
Expand Down
11 changes: 6 additions & 5 deletions pnpm-lock.yaml

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

0 comments on commit 515a155

Please sign in to comment.